A camera extrinsic parameter estimation method and device suitable for escalator

Abstract

The application discloses a camera extrinsic parameter estimation method and device suitable for escalators, and the method comprises the following steps: S1, selecting a plurality of mark points on the escalator, determining an origin and a coordinate axis, and measuring initial world coordinates of each calibration point relative to the origin; S2, acquiring an escalator picture containing the calibration points by using a camera; S3, starting the escalator so that the calibration points move with the escalator; S4, calculating the positions of the calibration points in the motion, and obtaining a world coordinate system of the calibration points; S5, according to a shooting sequence, corresponding to the positions of the calibration points in the image coordinate system in the motion, estimating the pose relationship between the camera and the escalator, re-projecting the world coordinates of the calibration points on the shot image, and calculating the average re-projection error of all re-projected points as the average re-projection error of the estimated pose for iterative processing.

Description

Technical Field

[0001] This invention relates to the field of camera extrinsic parameter estimation technology, and in particular to a method and apparatus for estimating camera extrinsic parameters applicable to escalators. Background Technology

[0002] Video surveillance of escalators plays a crucial role in ensuring passenger safety. Using depth cameras, the position of passengers relative to the camera in the surveillance video can be acquired and converted to their position relative to the ground. This allows for analysis of passenger status based on their real-world location. For cameras with known intrinsic parameters, converting passenger position information from relative to the camera to relative to the ground requires obtaining the camera's attitude relative to the ground through extrinsic parameter calibration.

[0003] One of the fundamental tasks of computer vision is to reconstruct the geometric information of objects in three-dimensional space from the object projection information acquired by a camera, and thereby reconstruct and recognize the objects. The relationship between the three-dimensional geometric position of a point on the surface of a spatial object and its corresponding point in the image is determined by the geometric model of the camera imaging, which is determined by camera parameters. Under most conditions, these parameters must be obtained through experimentation and calculation, i.e., camera calibration using marker points to find the relationship between world points and image projection points. Accurate camera calibration is a prerequisite for subsequent work; improving calibration accuracy and efficiency is key to scientific research and its application in industrial production.

[0004] Therefore, calibrating the camera's extrinsic parameters and determining the positional relationship between the monitoring camera and the escalator model is a crucial step in obtaining passenger position information relative to the ground, and the calibration results directly affect the accuracy of the camera's output.

[0005] Traditional calibration methods mainly include DUT, P3P, and EPnP, which are described in detail below:

[0006] In this method, the DUT directly solves for the unknowns of the extrinsic parameter matrix using at least six calibration points. However, this method ignores the relationships between the extrinsic parameter matrices and solves them as ordinary matrices. The resulting solution may not satisfy the constraints of the extrinsic parameter matrix, and the accuracy of extrinsic parameter estimation using the DUT faces significant challenges when dealing with large, irregular objects such as escalators.

[0007] P3P employs three calibration points and uses the cosine triangle rule to construct an equation relating the camera's optical center to the three spatial points and their corresponding angles. This equation is then transformed into a quadratic equation in two variables, solved using Wu's elimination method to obtain the coordinates of the three matched spatial points in the camera coordinate system. Finally, the camera's extrinsic parameters are obtained using the ICP method. However, the P3P algorithm can only utilize information from three points. When there are more than three pairs of given points, it struggles to utilize more information and is highly susceptible to noise, which can cause the algorithm to fail.

[0008] The EPnP method represents multiple 3D spatial points as a combination of four control points using at least three calibration points. It then calculates the camera pose based on the distance difference between the control points in the image and in the world. Other calibration methods, such as UPnP and DLS, also use at least four calibration points to calculate the camera pose. However, the denser and more dispersed the calibration points are in the image, the more realistic the calibrated camera pose becomes. Conversely, a smaller density of calibration points reduces accuracy, makes the results unsuitable for the entire image, and affects subsequent position calculations. For large objects like escalators, a large number of marker points are needed to estimate extrinsic parameters. However, the irregular shape of escalators, the non-fixed positions of the escalator steps, and the unevenness of the steps significantly reduce the efficiency and accuracy of measuring the world position of the marker points, resulting in very low calibration efficiency. Summary of the Invention

[0009] To overcome the shortcomings of the existing technology, the present invention aims to provide a camera extrinsic parameter estimation method and device suitable for escalators, so as to solve the problems of measurement efficiency and accuracy of marker points on escalators, and solve the problem of insufficient calibration accuracy caused by sparse points.

[0010] To achieve the above and other objectives, this invention proposes a camera extrinsic parameter estimation method suitable for escalators, comprising the following steps:

[0011] Step S1: Select several marker points on the escalator and attach them to the escalator, determine the origin and coordinate axes, and measure the initial world coordinates of each marker point relative to the origin.

[0012] Step S2: Continuously acquire escalator images containing calibration points using a camera;

[0013] Step S3: Start the escalator so that the calibration point moves with the escalator;

[0014] Step S4: Calculate the position of each calibration point during the movement and obtain the world coordinate system of each calibration point;

[0015] Step S5: According to the shooting order, calculate the image coordinate system position of the calibration point in motion, and estimate the pose relationship between the camera and the escalator based on the world coordinates and image coordinates of the calibration point. Based on the estimated pose relationship, reproject the world coordinates of the calibration point onto the captured image, and calculate the average reprojection error of all reprojected points as the average reprojection error of the estimated pose. Iterative processing is performed based on the obtained average reprojection error of the estimated pose.

[0016] Preferably, in step S1, no more than 3 calibration points are selected at any location on the escalator and affixed to any location on the escalator steps and handrail.

[0017] Preferably, step S1 further includes:

[0018] Step S100: Using the intersection line L0 between the inner ends of the two handrail belts, the outer ends of the comb plate teeth, and the steps as the reference line, and the midpoint of L0 as the origin, the key parameters of the escalator are obtained using measuring tools.

[0019] Step S101: Select a first calibration point P0 on the escalator step plate. The first calibration point P0 is located at the origin mentioned in step S100.

[0020] Step S102: Select the second calibration point P1 and the third calibration point P2 on the center lines of the two handrails, respectively located at the intersection of the intersection line L0 and the center line of the handrails.

[0021] Preferably, in step S2, the position of the known internal parameter camera is fixed, the camera frame rate is set, and the camera is started to continuously acquire escalator images containing calibration points.

[0022] Preferably, step S4 further includes:

[0023] Step S401: Calculate the camera working time based on the camera frame rate and the number of photos taken;

[0024] Step S402: Determine the calibration point movement time based on the obtained camera working time and escalator working time;

[0025] Step S403: Calculate the escalator running angle θ based on the determined calibration point movement time and escalator running speed;

[0026] Step S404: Determine the location of the calibration point based on the obtained escalator running angle.

[0027] Preferably, in step S403, the escalator running angle θ is calculated as follows:

[0028]

[0029] Where θ0 is the maximum operating inclination angle, R0 is the radius of curvature of the escalator guide rail, and ve Let t be the escalator's running speed, and t be the time it takes for the calibration point to move. t = t c -t e , t e For the working time of the escalator, t c For the camera's working time, t c =v c n, v c is the camera's frame rate, and n is the number of photos taken.

[0030] Preferably, in step S404, the positions of each calibration point are calculated as follows:

[0031]

[0032] Where, x e z is the longitudinal displacement distance of the calibration point. e x represents the vertical displacement height of the calibration point. e0 The initial longitudinal position of the calibration point is z e0 This is the initial vertical position of the calibration point.

[0033] Preferably, step S5 further includes:

[0034] Step S500: According to the shooting order in step S2, obtain the image position of the calibration point in each image;

[0035] Step S501: Estimate the pose relationship between the world coordinate system and the image coordinate system based on the world coordinates and image coordinates of the calibration points;

[0036] Step S502: Reproject the world coordinates of the calibration points onto the captured image according to the estimated pose relationship;

[0037] Step S503: Calculate the Euclidean distance between the reprojection point and the acquired image coordinate point, and use it as the reprojection error of each point. Then, average the reprojection errors of all points to get the average reprojection error of the estimated pose.

[0038] Step S504: Iterative processing is performed based on the average reprojection error of the estimated pose and the average reprojection error tolerance threshold.

[0039] Preferably, in step S504, an average reprojection error tolerance threshold is set. When the average reprojection error exceeds the threshold, the markers with the largest reprojection error that are not image edges are removed, and the process returns to step S500 for iteration. When the average reprojection error is lower than the error tolerance threshold or the set maximum number of iterations is reached, the iteration stops.

[0040] To achieve the above objectives, the present invention also provides a camera extrinsic parameter estimation device suitable for escalators, comprising:

[0041] The calibration point determination unit is used to select several marker points on the escalator, attach them to the escalator, determine the origin and coordinate axes, and measure the initial world coordinates of each calibration point relative to the origin.

[0042] The image acquisition unit is used to continuously acquire images of escalators containing calibration points using a camera.

[0043] The calibration point position calculation unit is used to start the escalator, so that the calibration point moves with the escalator, calculate the position of each calibration point during the movement, and obtain the world coordinate system of each calibration point during the movement;

[0044] The pose relationship determination unit is used to calculate the image coordinate system position of the calibration point in motion according to the shooting order, estimate the pose relationship between the camera and the escalator according to the world coordinates and image coordinates of the calibration point, reproject the world coordinates of the calibration point onto the captured image according to the estimated pose relationship, calculate the average reprojection error of all reprojection points as the average reprojection error of the estimated pose, and perform iterative processing based on the obtained average reprojection error of the estimated pose.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] 1. This invention cleverly utilizes the movement of escalators and the shooting patterns of cameras. By selecting only a few key points on the escalator as calibration points, the external parameters of the camera and the entire escalator can be estimated, greatly improving the efficiency of calibration.

[0047] 2. This invention calculates the world coordinates of the calibration point based on the law of motion, which is simpler than measuring directly on the escalator, eliminates measurement errors, and makes the calculation of external parameters more accurate.

[0048] 3. By continuously collecting the position information of the marker points during movement, this invention can obtain dense and evenly distributed calibration points on the escalator. Compared with selecting sparse calibration points, the external parameter calculation is more accurate. Attached Figure Description

[0049] Figure 1 This is a flowchart illustrating the steps of a camera extrinsic parameter estimation method applicable to escalators according to the present invention.

[0050] Figure 2 This is a system structure diagram of a camera extrinsic parameter estimation device applicable to escalators according to the present invention;

[0051] Figure 3 This is a flowchart of an embodiment of the present invention;

[0052] Figure 4a and Figure 4b This is a schematic diagram showing the selection of key points in an embodiment of the present invention. Detailed Implementation

[0053] The following describes the embodiments of the present invention through specific examples and in conjunction with the accompanying drawings. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific examples, and various details in this specification can also be modified and changed based on different viewpoints and applications without departing from the spirit of the present invention.

[0054] Figure 1 This is a flowchart illustrating the steps of a camera extrinsic parameter estimation method applicable to escalators according to the present invention. Figure 1 As shown, the present invention provides a method for estimating the extrinsic parameters of a camera applicable to escalators, comprising the following steps:

[0055] Step S1: Select several marker points on the escalator and attach them to the escalator. Determine the origin and coordinate axes, and measure the initial world coordinates of each marker point relative to the origin.

[0056] In this invention, no more than three calibration points are selected at any location on the escalator and affixed to any location on the escalator steps and handrail.

[0057] Specifically, step S1 further includes:

[0058] In step S100, the intersection line L0 between the inner ends of the two handrails, the outer ends of the comb plate teeth, and the steps is used as the reference line, and the midpoint of L0 is used as the origin. The key parameters of the escalator are measured using a ruler and a laser rangefinder, wherein the distance between the center lines of the two handrails is d0.

[0059] Step S101: Select calibration point P0 on the escalator step plate, which is located at the origin mentioned in step S100;

[0060] Step S102: Select two calibration points P1 and P2 on the center lines of the two handrails (each handrail has one center line, which falls on the handrail itself), located at the intersection of L0 and the center line of the handrail respectively. Since the handrails have a certain width, the calibration points need to be on the center line.

[0061] Step S2: Continuously acquire escalator images containing calibration points using a camera.

[0062] Specifically, the position of the camera is known by fixing the internal parameters, the camera frame rate is set, the camera is started, and images of the escalator containing the calibration points are continuously taken.

[0063] Step S3: Start the escalator so that the calibration points move with the escalator. That is, during the calibration process, each calibration point moves with the started escalator.

[0064] Step S4: Calculate the position of each calibration point during the movement.

[0065] In a specific embodiment of the present invention, the world coordinate system of the calibration point during movement is calculated based on the escalator's motion law and the camera's shooting speed. Specifically, step S4 further includes:

[0066] Step S401: Calculate the camera working time based on the camera frame rate and the number of photos taken.

[0067] In a specific embodiment of the present invention, the formula for calculating the camera's working time is as follows:

[0068] t c =v c n (Formula 1)

[0069] Among them, t c v is the time the camera is in operation. c is the camera's frame rate, and n is the number of photos taken.

[0070] Step S402: Determine the calibration point movement time based on the obtained camera working time and escalator working time.

[0071] Specifically, the time for the calibration point to move is calculated using the following formula:

[0072] t = t c -t e (Formula 2)

[0073] Where t is the time it takes for the calibration point to move, t e The time spent working on the escalator.

[0074] Step S403: Calculate the escalator running angle θ based on the determined calibration point movement time and escalator running speed.

[0075] In a specific embodiment of the present invention, the formula for calculating the escalator's inclination angle θ is as follows:

[0076]

[0077] Where θ0 is the maximum operating inclination angle, R0 is the radius of curvature of the escalator guide rail, and v e This refers to the escalator's operating speed.

[0078] Step S404: Determine the position of each calibration point based on the obtained escalator running angle.

[0079] In a specific embodiment of the present invention, the calculation formula for the position of each calibration point is as follows:

[0080]

[0081] Where, x e z is the longitudinal displacement distance of the calibration point.e x represents the vertical displacement height of the calibration point. e0 The initial longitudinal position of the calibration point is z e0 This is the initial vertical position of the calibration point.

[0082] Step S5: According to the shooting order, calculate the image coordinate system position of the calibration point in motion, and estimate the pose relationship between the camera and the escalator based on the world coordinates and image coordinates of the calibration point. Based on the estimated pose relationship, reproject the world coordinates of the calibration point onto the captured image, and calculate the average reprojection error of all reprojected points as the average reprojection error of the estimated pose. Iterative processing is performed based on the obtained average reprojection error of the estimated pose.

[0083] Specifically, step S5 further includes:

[0084] Step S500: Based on the shooting order in step S2, obtain the image positions of the calibration points in each image.

[0085] Step S501: Based on the world coordinates and image coordinates of the calibration points, estimate the pose relationship between the world coordinate system and the image coordinate system (i.e., the camera coordinate system). In a specific embodiment of the present invention, the EPnP algorithm is used to estimate the pose relationship between the world coordinate system and the camera coordinate system.

[0086] Step S502: Reproject the world coordinates of the calibration points onto the captured image according to the estimated pose relationship;

[0087] Step S503: Calculate the Euclidean distance between the reprojection point and the acquired image coordinate point, and use it as the reprojection error of each reprojection point. Then, average the reprojection errors of all reprojection points to get the average reprojection error of the estimated pose.

[0088] Step S504: Set the average reprojection error tolerance threshold. When the average reprojection error exceeds the threshold, remove the markers with the largest reprojection error that are not image edges, and return to step S500 for iteration.

[0089] Step S505: When the average reprojection error is lower than the error tolerance threshold or the maximum number of iterations is reached, the iteration stops. That is, a maximum number of iterations is set, and the iteration stops when the average reprojection error in step S503 is lower than the error tolerance threshold or the maximum number of iterations is reached.

[0090] Figure 2 This is a system structure diagram of a camera extrinsic parameter estimation device applicable to escalators according to the present invention. Figure 2 As shown, the present invention provides a camera extrinsic parameter estimation device suitable for escalators, comprising:

[0091] The calibration point determination unit 201 is used to select several marker points on the escalator and attach them to the escalator, determine the origin and coordinate axes, and measure the initial world coordinates of each calibration point relative to the origin.

[0092] In this invention, no more than three calibration points are selected at any location on the escalator and affixed to any location on the escalator steps and handrail.

[0093] Specifically, the calibration point determination unit 201 further includes:

[0094] The coordinate system determination module is used to measure the key parameters of the escalator using a ruler and a laser rangefinder, with the intersection line L0 between the inner ends of the two handrails, the outer ends of the comb plate teeth, and the steps as the reference line and the midpoint of L0 as the origin. The distance between the centerlines of the two handrails is d0.

[0095] The first calibration point determination module is used to select calibration point P0 on the escalator step plate, which is located at the origin mentioned in step S100.

[0096] Other calibration point determination modules are used to select two calibration points P1 and P2 on the center lines of the two handrail belts, located at the intersection of L0 and the center line of the handrail belts respectively.

[0097] Image acquisition unit 202 is used to continuously acquire escalator images containing calibration points using a camera.

[0098] Specifically, the position of the camera is known by fixing the internal parameters, the camera frame rate is set, the camera is started, and images of the escalator containing the calibration points are continuously taken.

[0099] The calibration point position calculation unit 203 is used to start the escalator so that the calibration points move with the escalator and calculate the position of each calibration point during the movement. That is, during the calibration process, each calibration point moves with the started escalator and the position of each calibration point during the movement is calculated.

[0100] In a specific embodiment of the present invention, the world coordinate system of the calibration point during movement is calculated based on the escalator's motion law and the camera's shooting speed. Specifically, the calibration point position calculation unit 203 further includes:

[0101] The camera working time calculation module is used to calculate the camera working time based on the camera frame rate and the number of photos taken.

[0102] In a specific embodiment of the present invention, the formula for calculating the camera's working time is as follows:

[0103] t c =v c n (Formula 1)

[0104] Among them, t c v is the time the camera is in operation. cis the camera's frame rate, and n is the number of photos taken.

[0105] The calibration point movement time determination module is used to determine the calibration point movement time based on the obtained camera working time and escalator working time.

[0106] Specifically, the time for the calibration point to move is calculated using the following formula:

[0107] t = t c -t e (Formula 2)

[0108] Where t is the time it takes for the calibration point to move, t e The time spent working on the escalator.

[0109] The escalator tilt angle calculation module is used to calculate the escalator tilt angle θ based on the determined calibration point travel time and escalator running speed.

[0110] In a specific embodiment of the present invention, the formula for calculating the escalator's inclination angle θ is as follows:

[0111]

[0112] Where θ0 is the maximum operating inclination angle, R0 is the radius of curvature of the escalator guide rail, and v e This refers to the escalator's operating speed.

[0113] The calibration point position calculation module is used to determine the position of each calibration point based on the obtained escalator running angle, that is, to obtain the world coordinate system of each calibration point during movement.

[0114] In a specific embodiment of the present invention, the calculation formula for the position of each calibration point is as follows:

[0115]

[0116] Where, x e z is the longitudinal displacement distance of the calibration point. e x represents the vertical displacement height of the calibration point. e0 The initial longitudinal position of the calibration point is z e0 This is the initial vertical position of the calibration point.

[0117] The pose relationship determination unit 204 is used to calculate the image coordinate system position of the calibration point in motion according to the shooting order, estimate the pose relationship between the camera and the escalator according to the world coordinates and image coordinates of the calibration point, reproject the world coordinates of the calibration point onto the captured image according to the estimated pose relationship, calculate the average reprojection error of all reprojection points as the average reprojection error of the estimated pose, and perform iterative processing based on the obtained average reprojection error of the estimated pose.

[0118] Specifically, the pose relationship determination unit 204 further includes:

[0119] The calibration point image position determination module is used to obtain the image position of the calibration point in each image according to the shooting order.

[0120] The pose relationship determination module is used to estimate the pose relationship between the world coordinate system and the image coordinate system (i.e., the camera coordinate system) based on the world coordinates and image coordinates of the calibration points, and uses the EPnP algorithm to estimate the pose relationship between the world coordinate system and the camera coordinate system.

[0121] The reprojection module is used to reproject the world coordinates of the calibration points onto the captured image according to the estimated pose relationship;

[0122] The average reprojection error calculation module calculates the Euclidean distance between the reprojection point and the acquired image coordinate point, which is used as the reprojection error of each point. The average reprojection error of all points is then used as the average reprojection error for the estimated pose.

[0123] The iterative processing module is used to set the average reprojection error tolerance threshold. When the average reprojection error exceeds the threshold, the marker points with the largest reprojection error that are not image edges are removed, and the process returns to the calibration point image position determination module for iteration. When the average reprojection error is lower than the error tolerance threshold or the maximum number of iterations is reached, the iteration stops. That is, the maximum number of iterations is set, and the iteration stops when the average reprojection error in the average reprojection error calculation module is lower than the error tolerance threshold or the maximum number of iterations is reached.

[0124] Example

[0125] In this embodiment, as Figure 3 As shown, a method for estimating camera extrinsic parameters suitable for escalators is described, with the following steps:

[0126] Step 1, as follows Figure 4a , Figure 4b The top and side views of the escalator shown are based on the intersection line L0 between the inner ends of the two handrails, the outer ends of the comb plate teeth, and the steps, with the midpoint of L0 as the origin. Key parameters of the escalator were measured using a ruler and a laser rangefinder, with the distance between the centerlines of the two handrails defined as d0.

[0127] Step 2, as follows Figure 4a , Figure 4b As shown, the specific method for selecting calibration points is as follows:

[0128] Step 2.1: Select calibration point P0 on the escalator step, located at the origin mentioned in step 1;

[0129] Step 2.2: Select two calibration points P1 and P2 on the center lines of the two handrails, located at the intersections of L0 and the center lines of the handrails respectively;

[0130] Step 3, as follows Figure 4a , Figure 4b As shown, a camera with known internal parameters is fixed 300cm above point P2, and the camera frame rate is set to 20fps. The camera continuously takes pictures of the escalator containing the calibration point. In this embodiment, the image size is 640*480 pixels and the focal length is 300mm.

[0131] Step 4: Set the escalator speed to 0.5 m / s, start the escalator, and make the calibration point move with the escalator;

[0132] Step 5: Calculate the world coordinate system of the escalator's motion reference point based on the escalator's motion pattern and the camera's shooting speed;

[0133] Step 6: Calculate the image coordinate system position of the corresponding motion calibration point according to the shooting order;

[0134] Step 7: Based on the world coordinates and image coordinates of the calibration points, the pose relationship between the world coordinate system and the camera coordinate system is estimated using the EPnP algorithm;

[0135] Step 8: Reproject the world coordinates of the calibration points onto the captured image according to the estimated pose relationship;

[0136] Step 9: Calculate the Euclidean distance between the reprojection point and the acquired image coordinate point, and use it as the reprojection error of each point. Then, average the reprojection errors of all points to get the average reprojection error of the estimated pose.

[0137] Step 10: Set the average reprojection error tolerance threshold to 1.0. When the average reprojection error exceeds the threshold, remove the markers with the largest reprojection error that are not image edges, and repeat steps 6.1, 6.2, and 6.3.

[0138] Step 11: Set the maximum number of iterations to 3. Stop iterating when the average reprojection error is lower than the error tolerance threshold or the maximum number of iterations is reached.

[0139] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can make modifications and changes to the above embodiments without departing from the spirit and scope of the invention. Therefore, the scope of protection of the present invention should be as set forth in the claims.

Claims

1. A method for estimating camera extrinsic parameters applicable to escalators, comprising the following steps: Step S1: Select several marker points on the escalator and attach them to the escalator, determine the origin and coordinate axes, and measure the initial world coordinates of each marker point relative to the origin. Step S2: Use a camera to acquire an image of the escalator containing the calibration points; Step S3: Start the escalator so that the calibration point moves with the escalator; Step S4: Calculate the position of each calibration point during the movement and obtain the world coordinate system of each calibration point; Step S5: According to the shooting order, calculate the image coordinate system position of the calibration point in motion, and estimate the pose relationship between the camera and the escalator based on the world coordinates and image coordinates of the calibration point. Based on the estimated pose relationship, reproject the world coordinates of the calibration point onto the captured image, and calculate the average reprojection error of all reprojected points as the average reprojection error of the estimated pose. Iterative processing is performed based on the obtained average reprojection error of the estimated pose. Step S1 further includes: Step S100: Using the intersection line L0 of the inner ends of the two handrail belts, the outer ends of the comb plate teeth, and the steps as the reference line, and the midpoint of L0 as the origin, the key parameters of the escalator are obtained using measuring tools. Step S101: Select a first calibration point P0 on the escalator step plate. The first calibration point P0 is located at the origin mentioned in step S100. Step S102: Select the second calibration point P1 and the third calibration point P2 on the center lines of the two handrails, respectively located at the intersection of the intersection line L0 and the center line of the handrails; Step S4 further includes: Step S401: Calculate the camera working time based on the camera frame rate and the number of photos taken; Step S402: Determine the calibration point movement time based on the obtained camera working time and escalator working time; Step S403: Calculate the escalator running angle based on the determined calibration point movement time and escalator running speed. ; Step S404: Determine the location of the calibration point based on the obtained escalator running angle; In step S403, the escalator running angle The calculation is as follows: in For the maximum operating tilt angle, Let be the radius of curvature of the guide rail of the escalator. For the escalator's operating speed, The time taken for the calibration point to move. , For the working hours of the escalator, For the camera's working time, , For the camera's frame rate, This represents the number of photos taken. In step S404, the positions of each calibration point are calculated as follows: in, The longitudinal displacement distance of the calibration point. The vertical displacement height of the calibration point. The initial longitudinal position of the calibration point. This is the initial vertical position of the calibration point.

2. The camera extrinsic parameter estimation method for escalators as described in claim 1, characterized in that, In step S2, the position of the known camera is fixed, the camera frame rate is set, and the camera is started to continuously acquire escalator images containing calibration points.

3. The camera extrinsic parameter estimation method for escalators as described in claim 1, characterized in that, Step S5 further includes: Step S500: According to the shooting order in step S2, obtain the image position of the calibration point in each image; Step S501: Estimate the pose relationship between the world coordinate system and the image coordinate system based on the world coordinates and image coordinates of the calibration points; Step S502: Reproject the world coordinates of the calibration points onto the captured image according to the estimated pose relationship; Step S503: Calculate the Euclidean distance between the reprojection point and the acquired image coordinate point, and use it as the reprojection error of each point. Then, average the reprojection errors of all points to get the average reprojection error of the estimated pose. Step S504: Iterative processing is performed based on the average reprojection error of the estimated pose and the average reprojection error tolerance threshold.

4. The camera extrinsic parameter estimation method for escalators as described in claim 3, characterized in that: In step S504, an average reprojection error tolerance threshold is set. When the average reprojection error exceeds the threshold, the markers with the largest reprojection error that are not image edges are removed, and the process returns to step S500 for iteration. The iteration stops when the average reprojection error falls below the error tolerance threshold or the maximum number of iterations is reached.

5. A camera extrinsic parameter estimation device suitable for escalators, comprising: The calibration point determination unit is used to select several marker points on the escalator, attach them to the escalator, determine the origin and coordinate axes, and measure the initial world coordinates of each calibration point relative to the origin. The image acquisition unit is used to acquire images of escalators containing calibration points using a camera. The calibration point position calculation unit is used to start the escalator, so that the calibration points move with the escalator, calculate the position of each calibration point during the movement, and obtain the world coordinate system of each calibration point during the movement; The pose relationship determination unit is used to calculate the image coordinate system position of the calibration point in motion according to the shooting order, estimate the pose relationship between the camera and the escalator according to the world coordinates and image coordinates of the calibration point, reproject the world coordinates of the calibration point onto the captured image according to the estimated pose relationship, calculate the average reprojection error of all reprojection points as the average reprojection error of the estimated pose, and perform iterative processing based on the obtained average reprojection error of the estimated pose. The calibration point determination unit includes: The coordinate system determination module is used to measure the key parameters of the escalator using a ruler and a laser rangefinder, with the intersection line L0 between the inner ends of the two handrails, the outer ends of the comb plate teeth, and the steps as the reference line and the midpoint of L0 as the origin. The distance between the centerlines of the two handrails is d0. The first calibration point determination module is used to select a first calibration point P0 on the escalator step plate, wherein the first calibration point P0 is located at the origin. Other calibration point determination modules are used to select the second calibration point P1 and the third calibration point P2 on the center line of the two handrails, which are located at the intersection of the intersection line L0 and the center line of the handrails, respectively. The calibration point location calculation unit includes: The camera working time calculation module is used to calculate the camera working time based on the camera frame rate and the number of photos taken. The calibration point movement time determination module is used to determine the calibration point movement time based on the obtained camera working time and escalator working time. The escalator tilt angle calculation module is used to calculate the escalator tilt angle θ based on the determined calibration point travel time and escalator running speed. The calibration point location calculation module is used to determine the location of the calibration point based on the obtained escalator running angle; The escalator's operating angle The calculation is as follows: in For the maximum operating tilt angle, Let be the radius of curvature of the guide rail of the escalator. For the escalator's operating speed, The time taken for the calibration point to move. , For the working hours of the escalator, For the camera's working time, , For the camera's frame rate, This represents the number of photos taken. The locations of each calibration point are calculated as follows: in, The longitudinal displacement distance of the calibration point. The vertical displacement height of the calibration point. The initial longitudinal position of the calibration point. This is the initial vertical position of the calibration point.

Images