Underground cavern real scene modeling image shooting pose planning method and other device

Abstract

The present application relates to the field of image data acquisition in real scene three-dimensional reconstruction technology, and provides a kind of underground chamber real scene modeling image shooting pose planning method, device and storage medium, by shooting section in current chamber, control gimbal shooting equipment is sequentially executed horizontal alignment and adaptive vertical sequence shooting to two side walls, complete the cross coverage shooting of opposite side wall;According to the field of view angle of camera and the preset image overlap rate requirement, dynamically calculate the target position of next shooting section and move gimbal shooting equipment;Repeat the above process until the target image coverage quantity requirement of excavation wall surface any point is reached.The present application ensures that the image overlap rate meets the three-dimensional reconstruction requirement by adaptive pose planning, dynamically calculates the pitch angle and forward distance, realizes the automatic control of image shooting process, can save manpower, greatly improves the shooting efficiency, can also improve the image shooting quality, avoids image blur and other problems.

Description

Technical Field

[0001] This invention relates to the field of image data acquisition technology in real-scene 3D reconstruction technology, and in particular to a method, device and storage medium for planning the pose of real-scene modeling images of underground caverns. Background Technology

[0002] 3D reality modeling technology has been deeply applied to the survey, construction, and design of water conservancy and hydropower projects. Through effective image data acquisition, a digital twin model of the construction excavation face can be constructed, realistically and accurately presenting the actual excavation situation. This provides a data foundation for calculating the engineering excavation volume and geological digital logging operations, and also provides valuable data assets for the construction and operation and maintenance phases of the project.

[0003] To ensure the accuracy of the 3D model, multi-angle imagery of the excavation face of the cavern is required. Each location must be covered by at least three images from different viewpoints to guarantee successful modeling; otherwise, problems such as model holes and incomplete modeling will occur. Furthermore, the excavation face of underground engineering projects often requires real-time cement spraying to protect exposed rock masses and reduce safety hazards such as rockfalls. This limits the opportunities for image data collection to only 1-2 times, increasing the difficulty of image acquisition. Moreover, when acquiring images of small underground caverns (including exploration tunnels and gravity drainage tunnels) with a diameter of 2-3 meters, the narrow diameter results in a narrower field of view for each image, leading to an exponential increase in the number of images acquired while maintaining a certain overlap rate. Workers also need to crouch down to photograph the ceiling and other parts of the narrow cavern, significantly increasing their physical exertion.

[0004] In view of this, it is necessary to propose a method, device and storage medium for planning the pose of real-scene modeling images of underground caverns to solve or at least alleviate the above-mentioned defects. Summary of the Invention

[0005] The main objective of this invention is to provide a method, device, and storage medium for planning the shooting pose of real-scene modeling images of underground caverns, in order to solve the technical problems in the prior art where image acquisition in underground caverns with limited space and time is difficult to guarantee the integrity of data acquisition and has low shooting efficiency due to unreasonable shooting pose planning.

[0006] To achieve the above objectives, the present invention provides a method for planning the pose of real-scene modeling images of underground caverns, comprising the following steps:

[0007] S1, at the current cavern shooting section, execute the first sidewall shooting process and the second sidewall shooting process in sequence; wherein, when the gimbal shooting device is close to the first sidewall of the cavern, first control the gimbal shooting device to perform horizontal alignment so that the shooting direction is perpendicular to the central axis of the cavern; then control the gimbal shooting device to perform adaptive vertical sequence shooting starting from the initial pitch angle to complete the shooting of the opposing second sidewall;

[0008] S2, after moving the gimbal shooting device close to the second side wall facing the opposite direction, first control the gimbal shooting device to perform horizontal alignment again, and then control the gimbal shooting device to perform adaptive vertical sequence shooting from the initial pitch angle to complete the shooting of the first side wall facing the opposite direction.

[0009] S3. Based on the camera field of view parameters of the gimbal shooting device and the preset image overlap rate requirements, calculate the target shooting position of the gimbal shooting device near the first side wall of the next cavern shooting section, and control the gimbal shooting device to move to the target shooting position.

[0010] S4. Repeat steps S1 to S3 until any point on the excavated wall of the target cavern section reaches the required number of target image coverage points.

[0011] Preferably, step S1, controlling the gimbal to perform adaptive vertical sequence shooting starting from the initial pitch angle, includes the following steps:

[0012] S11, first control the gimbal shooting device to rotate to the initial pitch angle, and shoot towards the opposite second side wall at the initial pitch angle;

[0013] S12, acquire the pose data of this shooting, and construct a first ray from the optical center of the camera to the midpoint of the upper edge of the image sensor and a second ray to the midpoint of the lower edge of the image sensor based on the pose data and the camera optical parameters.

[0014] S13, calculate the first intersection point of the first ray and the second sidewall, and the second intersection point of the second ray and the second sidewall, and then obtain the sidewall line segment or broken line between the first intersection points based on the first intersection point and the second intersection point;

[0015] S14, according to the preset image overlap rate requirement, select the first reference point of the next shooting position on the side wall line segment or broken line between the first intersection points;

[0016] S15, based on the spatial geometric relationship between the first reference point and the optical center of the camera, calculate the next target pitch angle required to achieve the preset image overlap rate requirement, and control the gimbal shooting device to rotate to the next target pitch angle;

[0017] S16. Repeat steps S12 to S15 until the camera pitch angle reaches the preset end pitch angle, thus completing the adaptive vertical sequence shooting starting from the initial pitch angle.

[0018] Preferably, step S3 includes the following steps:

[0019] S31, after completing the shooting of the current cavern section, control the camera of the gimbal shooting device to maintain a horizontal attitude and acquire the current pose data;

[0020] S32, based on the current pose data and camera optical parameters, construct a third ray from the camera optical center to the midpoint of the left edge of the image sensor, and a fourth ray from the camera optical center to the midpoint of the right edge of the image sensor.

[0021] S33, calculate the third intersection point of the third ray and the first sidewall, and the fourth intersection point of the fourth ray and the first sidewall, and then obtain the sidewall line segment or arc segment between the second intersection points based on the third intersection point and the fourth intersection point;

[0022] S34, according to the preset image overlap rate requirement, select the second reference point for the next shooting position on the side wall line segment or arc segment between the second intersection points;

[0023] S35, based on the spatial geometric relationship between the second reference point and the camera's optical center, calculate the target shooting position of the gimbal shooting device on the next shooting section of the cave, near the first side wall of the cave, use a laser pointer mounted on the gimbal to mark the next target shooting position, and control the gimbal shooting device to move to the target shooting position.

[0024] Preferably, obtaining the pose data for this shooting in step S12 includes the following steps:

[0025] S121. Establish a two-dimensional coordinate system with the transverse center of the bottom plate of the current tunnel section as the origin O, the direction from the center of the bottom plate to the second side wall as the positive X-axis, and the direction from the center of the bottom plate to the top arch as the positive Y-axis.

[0026] S122, obtain the distance between the camera of the gimbal shooting device and the second side wall, the camera height and the cross-sectional width of the cave, and calculate the coordinates of the camera optical center in the two-dimensional coordinate system;

[0027] S123, based on the current pitch angle of the camera, the focal length of the camera, the size of the image sensor, and the vertical viewing angle of the line connecting the midpoint of the upper edge of the image sensor to the optical center of the camera, and the vertical viewing angle of the line connecting the midpoint of the lower edge of the image sensor to the optical center of the camera, calculate the coordinates of the midpoint of the upper edge of the image sensor and the midpoint of the lower edge of the image sensor in the two-dimensional coordinate system.

[0028] Preferably, in step S123, the coordinates of the midpoint of the upper edge of the image sensor in the two-dimensional coordinate system are (Dw / 2-DL). cos( ), H+L sin( The coordinates of the midpoint of the lower edge of the image sensor in the two-dimensional coordinate system are (Dw / 2 - Df, Hh / 2); where Dw is the cross-sectional width of the cavern, D is the object distance between the gimbal and the second sidewall, and L is the distance from the optical center of the camera to the midpoint of the upper edge of the image sensor. The current pitch angle, The vertical viewing angle is f, the camera focal length is H, the camera optical center height is h, and the image sensor height dimension is h.

[0029] Preferably, the initial pitch angle is -45° and the preset final pitch angle is 75°; wherein, the pitch angle is the angle between the pitch angle and the positive X-axis, with counterclockwise being positive and clockwise being negative.

[0030] Preferably, the preset image overlap rate is 60%.

[0031] The present invention also provides a device for planning the pose of real-scene modeling images of underground caverns, comprising:

[0032] The first shooting execution unit is used to sequentially execute the first sidewall shooting process and the second sidewall shooting process on the current cavern shooting section. Specifically, when the gimbal shooting device is close to the first sidewall of the cavern, the gimbal shooting device is first controlled to perform horizontal alignment so that the shooting direction is perpendicular to the central axis of the cavern. Then, the gimbal shooting device is controlled to perform adaptive vertical sequence shooting starting from the initial pitch angle to complete the shooting of the opposing second sidewall.

[0033] The second shooting execution unit is used to move the gimbal shooting device close to the second side wall facing the opposite direction, first control the gimbal shooting device to perform horizontal alignment again, and then control the gimbal shooting device to perform adaptive vertical sequence shooting starting from the initial pitch angle, so as to complete the shooting of the first side wall facing the opposite direction.

[0034] The forward control unit is used to calculate the target shooting position of the gimbal shooting device near the first side wall of the next cavern shooting section based on the camera field of view parameters and preset image overlap rate requirements of the gimbal shooting device, and control the gimbal shooting device to move to the target shooting position.

[0035] The process control unit is used to repeat steps S1 to S3 until any point on the excavated wall of the target cavern section reaches the target image coverage requirement.

[0036] The present invention also provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described method for planning the pose of a real-scene modeling image of an underground cavern.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] This invention provides a method, device, and storage medium for planning the pose of real-scene modeling images of underground caverns. It realizes automated control of the image shooting process, which can save manpower, greatly improve shooting efficiency, improve image shooting quality, avoid image blurring and other problems, and ensure the quality of real-scene 3D model reconstruction. Specifically, this application implements adaptive pose planning. By dynamically calculating the target pitch angle required for the next image rotation based on the coverage area of ​​the current image on the cave wall, it ensures that the overlap rate between adjacent images always meets the preset image overlap rate requirement during vertical sequence shooting. By calculating the coverage area of ​​the current image in the horizontal direction and based on the preset image overlap rate requirement, it automatically calculates the distance that the gimbal shooting device should advance along the cave axis, ensuring the quality of image data acquisition and providing a reliable data foundation for 3D reconstruction. By using a simple gimbal shooting device that can be controlled by a program, it realizes the automatic rotation and shooting of the camera in the horizontal and vertical directions, thereby completely replacing the traditional manual handling, aiming and shooting operations with an automated acquisition process, freeing up manpower and greatly improving shooting efficiency. Shooting based on the gimbal shooting device can ensure the camera's autofocus function, ensure image shooting quality, and reduce the problems of voids and low quality in the modeling process. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0040] Figure 1 This is a schematic flowchart of one embodiment of the present invention;

[0041] Figure 2 This is a three-dimensional schematic diagram of the overall structure of the gimbal shooting device in one embodiment of the present invention;

[0042] Figure 3 This is a cross-sectional schematic diagram of the overall structure of the gimbal shooting device in one embodiment of the present invention;

[0043] Figure 4 This is a schematic diagram of the structure of the first arm and the second arm in one embodiment of the present invention;

[0044] Figure 5 This is a schematic diagram of the structure of the support platform in one embodiment of the present invention;

[0045] Figure 6 This is a schematic diagram of the structure of the first rotating device in one embodiment of the present invention;

[0046] Figure 7 This is a bottom view of the base in one embodiment of the present invention;

[0047] Figure 8 This is a schematic diagram of the gimbal imaging device in one embodiment of the present invention, showing the process of image acquisition towards the second sidewall from a position close to the first sidewall.

[0048] Figure 9 This is a schematic diagram of the gimbal imaging device in one embodiment of the present invention, showing the process of image acquisition towards the first sidewall from a position near the second sidewall.

[0049] Figure 10 This is a top view of the underground cavern image capture path planning in one embodiment of the present invention;

[0050] Figure 11 This is a schematic diagram of a three-dimensional model of an underground cavern reconstructed from images captured by a gimbal camera in one embodiment of the present invention.

[0051] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings.

[0052] Explanation of icon numbers:

[0053] 1. First rotating device; 11. First fixed part; 12. First rotating part; 2. Second support arm; 21. Bearing; 3. Second rotating device; 4. First support arm; 5. Bearing platform; 51. Drive plate; 52. Bearing plate; 53. Support plate; 6. Distance measuring device; 7. Horizontal positioning device; 8. Angle measuring device; 9. Base; 10. Turntable. Detailed Implementation

[0054] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0056] In this invention, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. Furthermore, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. If the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed by this invention.

[0057] Please refer to Figures 1 to 11 The present invention provides a method for planning the pose of real-scene modeling images of underground caverns, comprising the following steps:

[0058] S1, at the current cavern imaging section, the first sidewall imaging process and the second sidewall imaging process are executed sequentially; wherein, when the gimbal imaging device is close to the first sidewall of the cavern, the gimbal imaging device is first controlled to perform horizontal alignment so that the imaging direction is perpendicular to the central axis of the cavern; then the gimbal imaging device is controlled to perform adaptive vertical sequence imaging starting from the initial pitch angle to complete the imaging of the opposing second sidewall; it is worth noting that the cavern imaging section defined in this application is not the cross-section of the cavern, but an imaging working surface planned along the axial direction of the cavern for systematic data acquisition, providing a clear target working area for the gimbal imaging device.

[0059] like Figures 2 to 7 As shown, the gimbal shooting device includes a turntable 10, a first rotating device 1, a second rotating device 3, a first support arm 4, a second support arm 2, and a support platform 5 for mounting a camera and auxiliary equipment. The first support arm 4 and the second support arm 2 each include a first end (not shown) and a second end (not shown). The first end of the first support arm 4 and the first end of the second support arm 2 are both fixedly connected to the turntable 10. The first support arm 4 and the second support arm 2 are symmetrical with respect to the central axis of the turntable 10. The second rotating device 3 is connected to the turntable 10 to drive the turntable 10 to rotate in the horizontal direction.

[0060] The first support arm 4 and the second support arm 2 are hollow inside. The second end of the first support arm 4 and the second end of the second support arm 2 are rotatably connected to the bearing platform 5. The first rotating device 1 is disposed inside the second end of the first support arm 4 and is connected to the bearing platform 5 to drive the bearing platform 5 to rotate in the vertical direction.

[0061] This embodiment utilizes a turntable 10 and a symmetrically distributed double-arm structure to evenly distribute the weight of the carrying platform 5, camera, and auxiliary equipment across the two arms and turntable 10, avoiding excessive stress at a single point and significantly improving the device's load-bearing capacity for heavy shooting equipment. The hollow arm structure reduces its own weight while maintaining structural strength. Combined with the mechanical balance created by the symmetrical layout, this ensures the device remains stable when carrying heavy equipment, preventing platform deformation and vibration caused by equipment weight and ensuring the structural reliability of the device during shooting.

[0062] The second rotating device 3 drives the turntable 10 to rotate horizontally, realizing 360° horizontal angle adjustment of the bearing platform 5. The first rotating device 1 drives the bearing platform 5 to rotate vertically, realizing precise adjustment of the pitch angle. The synergistic effect of the two rotating devices enables the camera shooting angle and posture to be precisely controlled according to preset parameters, avoiding blurry and distorted photos caused by hand shake and angle deviation during manual shooting, and significantly improving the clarity, consistency and accuracy of the photos.

[0063] The combination of symmetrical support arms and dual rotating devices forms a stable multi-degree-of-freedom adjustment mechanism, which can effectively counteract the slight vibrations that may exist in the underground cavern environment (such as personnel movement, equipment operation, etc.), reduce shaking during shooting, further ensure the stability of photo quality, and avoid fluctuations in shooting quality caused by environmental interference.

[0064] Both the horizontal rotation of the turntable 10 and the vertical rotation of the support platform 5 can be adjusted according to the method steps of this application. In this embodiment, there is no need for manual hand-holding of the camera or manual adjustment of the shooting posture, which reduces the steps of manual intervention and significantly improves the shooting speed. The support platform 5 can carry the camera and auxiliary equipment at the same time, eliminating the need for manual operation of multiple devices. This enables collaborative operation of shooting, lighting, positioning and other functions, reduces the time cost of equipment switching and debugging, and improves the overall work efficiency of underground cavern real scene modeling.

[0065] In this embodiment, the first rotating device 1 further includes a first motor, a first fixing part 11, and a first rotating part 12. The first fixing part 11 is fixedly connected to the first support arm 4, and the first rotating part 12 is fixedly connected to the bearing platform 5. The first rotating part 12 is connected to the first motor. The first rotating part 12 has a protrusion, and the first fixing part 11 has a hole. The protrusion extends into the hole of the first fixing part 11, and the first rotating part 12 is rotatably connected to the first fixing part 11. The protrusion of the first rotating part 12 extends into the hole of the first fixing part 11 to form a shaft-hole rotatable connection. This embedded fit greatly limits the radial displacement (left-right swaying) during rotation. Compared with ordinary surface contact rotatable connections, the structure has higher strength and more uniform force distribution. For heavy cameras and auxiliary equipment, this structure can effectively distribute the weight load transmitted by the bearing platform 5, avoid deformation or loosening at the rotating connection due to uneven force distribution, further improve the device's load-bearing compatibility with heavy shooting equipment, and ensure that it can maintain a stable rotation state even when carrying heavy equipment.

[0066] Furthermore, in this embodiment, the first rotating device 1 and the second rotating device 3 have the same structure. The second rotating device 3 includes a second motor, a second fixed part, and a second rotating part. The first motor is the same as the second motor. The second rotating part is fixedly connected to the turntable 10 and connected to the second motor. The second rotating part has a protrusion the same as that of the first rotating device 1, and the second fixed part has a hole the same as that of the first rotating device 1. The protrusion extends into the hole of the second fixed part, and the second rotating part is rotatably connected to the second fixed part.

[0067] In one embodiment, a horizontal positioning device 7 is further included for aligning the camera horizontally with the underground cavern wall. The horizontal positioning device 7 is fixed to the turntable 10 and connected to the second rotating device 3. Preferably, the horizontal positioning device 7 in this embodiment employs a two-dimensional lidar, which allows for horizontal alignment.

[0068] In one embodiment, the support platform 5 includes a drive plate 51, a support plate 52, and a support plate 53. The drive plate 51, the support plate 52, and the support plate 53 are connected in sequence to form a U-shaped space. The U-shaped space is used to fix a camera. The drive plate 51 is connected to the first rotating device 1, and the support plate 53 is connected to the second support arm 2.

[0069] In this embodiment, the camera and auxiliary equipment are embedded in a U-shaped space. The camera is constrained on three sides by the bearing plate 52, drive plate 51, and support plate 53, preventing the camera from shifting or falling off due to vibration or inertia during rotation and movement. The drive plate 51 is connected to the first rotating device 1, and the support plate 53 is connected to the second support arm 2, forming a double-support structure that works mechanically in synergy with the symmetrical layout of the double support arms. The weight of the bearing platform 5 is transferred to the turntable 10 through dual paths, completely avoiding the stress concentration caused by single-point support, making the load distribution more even, further improving the device's compatibility with heavy equipment, reducing wear at rotating connections, and extending the device's service life.

[0070] As a further preferred embodiment, the second support arm 2 includes a bearing 21, the outer ring of which is fixedly connected to the second support arm 2, and the inner ring of which is connected to the support plate 53. The first support arm 4 and the second support arm 2 are at the same height.

[0071] As a further preferred embodiment, two mounting parts for mounting auxiliary equipment are provided below the support plate 52, and the mounting parts are symmetrically arranged with respect to the central axis of the support platform 5.

[0072] As a further preferred embodiment, the system also includes a ranging device 6 and an angle measuring device 8, both of which are fixed below the support plate 52. The ranging device 6 and the angle measuring device 8 are respectively connected to different mounting parts. In this embodiment, the ranging device 6 is a point laser module used to measure the distance from the camera to the underground cavern wall, and the angle measuring device 8 is an IMU 9-axis sensor used to measure the camera's pitch and horizontal angles.

[0073] The mounting unit is located below the support plate 52, which does not occupy the camera installation area in the U-shaped space, nor does it interfere with the connection between the drive plate 51 and the rotating device, or between the support plate 53 and the second support arm 2. While adding auxiliary equipment, it maintains the compact structure of the support platform 5, adapts to the deployment requirements of narrow underground caverns, and avoids operational limitations caused by equipment redundancy.

[0074] In one embodiment, the system further includes a base 9 and a circuit module. The base 9 is hollow inside, and the circuit module is housed within the base 9. A through hole is provided on the upper part of the base 9 so that a communication circuit outside the base 9 can be connected to the circuit module through the through hole. An interface for connecting the circuit module to the outside is provided on the side of the base 9.

[0075] In a further preferred embodiment, the base 9 is also provided with a connecting seat for matching a tripod, and the connecting seat is located at the center of the bottom of the base 9.

[0076] In this embodiment, the circuit module is housed inside the base 9. The enclosed structure of the base 9 effectively prevents dust, water droplets, and minor impacts from the underground cavern from damaging the delicate circuitry, significantly improving the protection level compared to an exposed circuit layout. Simultaneously, the structural strength of the base 9 protects the circuit module from being squeezed by heavy objects or subjected to impacts from drops, extending the device's lifespan, reducing equipment maintenance and operational interruptions caused by environmental factors, and indirectly improving shooting efficiency.

[0077] Furthermore, a support can be set up, preferably a tripod, with a height ranging from 30 to 80 cm. The minimum height of 30 cm is for taking pictures of small underground caverns of about 2 meters, while the maximum height of 80 cm is to facilitate the operation of the equipment by the staff.

[0078] S2, after moving the gimbal camera to a position close to the second side wall facing the opposite direction, first control the gimbal camera to perform horizontal alignment again, and then control the gimbal camera to perform adaptive vertical sequence shooting starting from the initial pitch angle to complete the shooting of the first side wall facing the opposite direction; wherein, manual alignment or other mature alignment methods can be used, such as building a gyroscope and electronic compass into the gimbal camera to enable the gimbal camera to automatically maintain or turn to a certain absolute or relative angle for alignment, which will not be elaborated here.

[0079] By setting up two symmetrical shooting points on the left and right sides within the same cavern's cross-section and performing adaptive vertical sequence shooting covering the opposite sidewalls and the arch, the effect of cross-covering the same cross-sectional area of ​​the cavern from two different perspectives was achieved. This ensured that the entire cross-section of the cavern was effectively covered and provided high-quality image data for subsequent reconstruction of the real-scene 3D model.

[0080] This application achieves horizontal alignment by controlling the rotation axis of the gimbal camera, ensuring that the camera's shooting direction is perpendicular to the central axis of the cavern. This eliminates cumulative errors introduced by equipment movement or initial attitude deviations, ensuring all captured images are on a uniform reference. It also guarantees that vertically sequenced images will not exhibit unpredictable attitude tilt, ensuring the effectiveness of subsequent 3D reconstruction of the real-world model.

[0081] S3. Based on the camera field of view parameters of the gimbal shooting device and the preset image overlap rate requirements, calculate the target shooting position of the gimbal shooting device near the first side wall of the next cavern shooting section, and control the gimbal shooting device to move to the target shooting position. The camera field of view parameters are known calibration parameters inside the camera, mainly including focal length, image sensor size and camera optical center, which lays the foundation for accurately calculating the actual range that a single image can cover at a specific object distance. The target shooting position is not a fixed position, but a precise distance calculated according to the actual situation, thereby ensuring that the cavern can also capture images that meet the preset image overlap rate requirements in the axial direction.

[0082] This application employs an adaptive vertical sequence shooting method based on real-time pose feedback and preset image overlap rate requirements. This method dynamically adjusts the vertical shooting angle according to the actual distance between the camera and the cavern wall, overcoming the inefficiencies or insufficient overlap rates of fixed-angle schemes caused by changes in cavern cross-sectional dimensions or shooting positions. While ensuring vertical image quality, it optimizes shooting efficiency and significantly reduces issues such as missed shots and blurred images that occur during manual shooting. By accurately calculating the target shooting position along the cavern's central axis to the next shooting section based on camera field-of-view parameters and preset image overlap rate requirements, it automates and quantifies the shooting path planning, ensuring that images taken from adjacent sections along the cavern's axis meet the preset overlap rate requirements.

[0083] S4. Repeat steps S1-S3 until any point on the excavated wall of the target tunnel section reaches the required number of target images. The optimal number of target images is at least four to ensure the quality of the subsequent 3D model creation. Figure 11 This is a schematic diagram of a 3D model of an underground cavern reconstructed from images captured by a gimbal camera.

[0084] This application's solution implements adaptive pose planning. By dynamically calculating the target pitch angle required for the next image rotation based on the coverage area of ​​the current image on the cavern wall, it ensures that the overlap rate between adjacent images always meets the preset image overlap rate requirement during vertical sequence shooting. By calculating the coverage area of ​​the current image in the horizontal direction and based on the preset image overlap rate requirement, it automatically calculates the distance that the gimbal shooting device should advance along the cavern axis, ensuring the quality of image data acquisition and providing a reliable data foundation for 3D reconstruction. By using a programmable gimbal shooting device, it realizes automatic rotation and shooting of the camera in the horizontal and vertical directions, thus completely replacing the traditional manual handling, aiming, and shooting operations with an automated acquisition process, freeing up manpower and significantly improving shooting efficiency. Shooting based on the gimbal shooting device can ensure the camera's autofocus function, guarantee image shooting quality, and reduce the problems of voids and low quality in the modeling process.

[0085] In a preferred embodiment, step S1, controlling the gimbal imaging device to perform adaptive vertical sequence shooting starting from the initial pitch angle, includes the following steps:

[0086] S11, first control the gimbal shooting device to rotate to the initial pitch angle, and shoot towards the opposite second side wall at the initial pitch angle;

[0087] S12, acquire the pose data of this shooting, and construct a first ray from the optical center of the camera to the midpoint of the upper edge of the image sensor and a second ray to the midpoint of the lower edge of the image sensor based on the pose data and the camera optical parameters.

[0088] S13, calculate the first intersection point of the first ray and the second sidewall, and the second intersection point of the second ray and the second sidewall, and then obtain the sidewall segment between the first intersection points based on the first intersection point and the second intersection point; in other embodiments, it can also be a broken line obtained based on the first intersection point and the second intersection point.

[0089] S14, according to the preset image overlap rate requirement, select the first reference point of the next shooting position on the side wall line segment or broken line between the first intersection points;

[0090] After each shot, a field-of-view boundary ray is constructed based on the current pose and camera parameters. The actual coverage line segment on the cave wall (the side wall segment between the first intersection points) of this shot is calculated. The reference point for the next shot is determined on the side wall segment between the first intersection points according to the preset image overlap rate requirement. The abstract preset image overlap rate requirement is transformed into a precise spatial geometric constraint. This allows the optimal target pitch angle to be calculated in reverse, ensuring that the overlap rate between adjacent images in the adaptive vertical sequence shooting is always accurately met. This provides a reliable data foundation for the subsequent high-quality reconstruction of the real scene 3D model.

[0091] S15, based on the spatial geometric relationship between the first reference point and the optical center of the camera, calculate the next target pitch angle required to achieve the preset image overlap rate requirement, and control the gimbal shooting device to rotate to the next target pitch angle;

[0092] S16. Repeat steps S12 to S15 until the camera pitch angle reaches the preset end pitch angle, thus completing the adaptive vertical sequence shooting starting from the initial pitch angle.

[0093] like Figure 8 The diagram shows the process of the gimbal camera capturing images from the second side wall near the first side wall. The two blue arrows indicate the shooting range. After back-calculation and adjustment, the next target pitch angle position corresponds to the range of the two green arrows. The adaptive vertical sequence shooting is completed sequentially from bottom to top.

[0094] like Figure 9 The diagram shows the process of the gimbal camera capturing images from the first side wall near the second side wall. The two blue arrows indicate the shooting range. After back-calculation and adjustment, the next target pitch angle position corresponds to the range of the two green arrows. The adaptive vertical sequence shooting is completed sequentially from bottom to top.

[0095] The gimbal camera in this embodiment can dynamically adjust the rotation angle of each step in the vertical shooting sequence according to its real-time relative position with the cave wall. This overcomes the shortcomings of inefficient or incomplete coverage of fixed-angle shooting schemes caused by changes in cave size or differences in camera pose. While ensuring coverage quality, it achieves high efficiency and intelligence in the shooting process, providing a reliable data foundation for subsequent high-quality reconstruction of real-scene 3D models.

[0096] In a preferred embodiment, step S3 includes the following steps:

[0097] S31, after completing the shooting of the current cavern section, control the camera of the pan-tilt shooting device to maintain a horizontal attitude and acquire the current pose data;

[0098] S32, based on the current pose data and camera optical parameters, construct a third ray from the camera optical center to the midpoint of the left edge of the image sensor, and a fourth ray from the camera optical center to the midpoint of the right edge of the image sensor.

[0099] S33, calculate the third intersection point of the third ray and the first sidewall, and the fourth intersection point of the fourth ray and the first sidewall, and then obtain the sidewall segment between the second intersection points based on the third intersection point and the fourth intersection point; in other embodiments, an arc segment can also be obtained based on the third intersection point and the fourth intersection point.

[0100] S34. According to the preset image overlap rate requirement, select a second reference point on the side wall segment between the second intersection points for the next shooting position. After completing the shooting section of the cave, use the left and right boundary rays of the camera's horizontal field of view to intersect with the side wall of the cave to obtain the side wall segment or arc segment between the second intersection points. Based on the preset overlap rate requirement, determine a reference point on the side wall segment or arc segment between the second intersection points as the second reference point, and automatically calculate the optimal shooting position for the next shooting section of the cave.

[0101] S35, based on the spatial geometric relationship between the second reference point and the camera's optical center, calculate the target shooting position of the gimbal shooting device near the first sidewall of the next cavern's shooting section, and control the gimbal shooting device to move to the target shooting position. Further, a laser pointer mounted on the gimbal shooting device can be used to illuminate a marker point on the cavern wall, and the gimbal shooting device can be moved towards the illuminated marker point to the target shooting position near the first sidewall.

[0102] By performing back-calculation based on the spatial geometric relationship between the second reference point and the camera's optical center, it was ensured that the images captured by adjacent cavern sections along the cavern's forward direction could meet the preset image overlap rate requirements, providing a reliable data foundation for subsequent high-quality real-scene 3D model reconstruction.

[0103] like Figure 10The image capture path planning for the underground cavern shown is shown from above. The numerical codes represent the cavern station numbers, the red dashed arrows indicate the direction of travel for the pan-tilt-zoom (PTZ) camera, and the blue dashed lines represent the central axis of the cavern.

[0104] In a preferred embodiment, obtaining the pose data of this shooting in step S12 includes the following steps:

[0105] S121. Establish a two-dimensional coordinate system with the transverse center of the bottom plate of the current cavern shooting section as the origin O, the direction from the center of the bottom plate to the second side wall as the positive X-axis, and the direction from the center of the bottom plate to the top arch as the positive Y-axis. By establishing the two-dimensional coordinate system, an accurate mathematical reference benchmark is provided for the spatial relationship between the camera pose and the cavern geometry, transforming the complex spatial coverage judgment problem into a planar geometric calculation problem.

[0106] S122: Obtain the distance between the camera of the gimbal shooting device and the second side wall, the camera height and the width of the cave cross section, calculate the coordinates of the camera optical center in the two-dimensional coordinate system, and realize the precise positioning of the camera of the gimbal shooting device in the cave shooting cross section.

[0107] S123, based on the current pitch angle of the camera, the focal length of the camera, the size of the image sensor, and the vertical viewing angle of the line connecting the midpoint of the upper edge of the image sensor to the optical center of the camera, and the vertical viewing angle of the line connecting the midpoint of the lower edge of the image sensor to the optical center of the camera, calculate the coordinates of the midpoint of the upper edge of the image sensor and the midpoint of the lower edge of the image sensor in the two-dimensional coordinate system.

[0108] Further, in step S123, the coordinates of the midpoint of the upper edge of the image sensor in the two-dimensional coordinate system are (Dw / 2-DL). cos( ), H+L sin( The coordinates of the midpoint of the lower edge of the image sensor in the two-dimensional coordinate system are (Dw / 2 - Df, Hh / 2); where Dw is the cross-sectional width of the cavern, D is the object distance between the gimbal and the second sidewall, and L is the distance from the optical center of the camera to the midpoint of the upper edge of the image sensor. The current pitch angle, The vertical viewing angle is f, the camera focal length is H, the camera optical center height is h, and the image sensor height dimension is h.

[0109] As a preferred example, the initial pitch angle is -45°, and the preset ending pitch angle is 75°; where the pitch angle is the angle between the camera and the positive X-axis, with counterclockwise being positive and clockwise being negative. This embodiment sets the initial pitch angle and the ending pitch angle to -45° and 75° respectively, allowing the camera's optical axis to scan from a downward tilt position to a significantly upward tilt position. This enables the gimbal's field of view to completely cover the continuous area from the bottom of the opposing side wall, the side wall, and the top arch, effectively eliminating the bottom and top blind spots in the real-world 3D model.

[0110] As a preferred example, the preset image overlap rate is 60%.

[0111] This application proposes using the cavern imaging section as the basic unit, taking images from the bottom of the sidewall upwards within the same cavern imaging section, ensuring a 60% overlap rate between adjacent images; similarly, when taking images in the next cavern imaging section, ensuring a 60% overlap rate between adjacent images on the same horizontal line. This ensures, considering image edge distortion, that at any location on the cavern imaging section, there are at least four high-quality images covering the area. (The image capture principle is to ensure that all areas of the cavern imaging section are covered by at least four images, thereby ensuring that each point has a sufficient number of images for 3D model reconstruction.) In other embodiments, those skilled in the art can also make adaptive settings as needed.

[0112] like Figure 11 The diagram shows a three-dimensional model of an underground cavern reconstructed from images captured by a gimbal camera. The red markings on the cavern wall are cavern station numbers, joint markings, and other identifiers left by geologists during the exploration process (this content is not covered in the technical solution of this invention, so it is not disclosed in the diagram), used to record the results of the exploration process.

[0113] The present invention also provides a device for planning the pose of real-scene modeling images of underground caverns, comprising:

[0114] The first shooting execution unit is used to sequentially execute the first sidewall shooting process and the second sidewall shooting process on the current cavern shooting section. Specifically, when the gimbal shooting device is close to the first sidewall of the cavern, the gimbal shooting device is first controlled to perform horizontal alignment so that the shooting direction is perpendicular to the central axis of the cavern. Then, the gimbal shooting device is controlled to perform adaptive vertical sequence shooting starting from the initial pitch angle to complete the shooting of the opposing second sidewall.

[0115] The second shooting execution unit is used to move the gimbal shooting device close to the second side wall facing the opposite direction, first control the gimbal shooting device to perform horizontal alignment again, and then control the gimbal shooting device to perform adaptive vertical sequence shooting starting from the initial pitch angle, so as to complete the shooting of the first side wall facing the opposite direction.

[0116] The forward control unit is used to calculate the target shooting position of the gimbal shooting device near the first side wall of the next cavern shooting section based on the camera field of view parameters and preset image overlap rate requirements of the gimbal shooting device, and control the gimbal shooting device to move to the target shooting position.

[0117] The process control unit is used to repeat steps S1 to S3 until any point on the excavated wall of the target cavern section reaches the target image coverage requirement.

[0118] The present invention also provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described method for planning the pose of a real-scene modeling image of an underground cavern.

[0119] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0120] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for planning the pose of real-scene modeling images of underground caverns, characterized in that, Includes the following steps: S1, at the current cavern shooting section, execute the first sidewall shooting process and the second sidewall shooting process in sequence; wherein, when the gimbal shooting device is close to the first sidewall of the cavern, first control the gimbal shooting device to perform horizontal alignment so that the shooting direction is perpendicular to the central axis of the cavern; then control the gimbal shooting device to perform adaptive vertical sequence shooting starting from the initial pitch angle to complete the shooting of the opposing second sidewall; S2, after moving the gimbal shooting device close to the second side wall facing the opposite direction, first control the gimbal shooting device to perform horizontal alignment again, and then control the gimbal shooting device to perform adaptive vertical sequence shooting from the initial pitch angle to complete the shooting of the first side wall facing the opposite direction. S3. Based on the camera field of view parameters of the gimbal shooting device and the preset image overlap rate requirements, calculate the target shooting position of the gimbal shooting device near the first side wall of the next cavern shooting section, and control the gimbal shooting device to move to the target shooting position. S4. Repeat steps S1 to S3 until any point on the excavated wall of the target cavern section reaches the target image coverage requirement. Step S1, controlling the gimbal imaging device to perform adaptive vertical sequence shooting starting from the initial pitch angle, includes the following steps: S11, first control the gimbal shooting device to rotate to the initial pitch angle, and shoot towards the opposite second side wall at the initial pitch angle; S12, acquire the pose data of this shooting, and construct a first ray from the optical center of the camera to the midpoint of the upper edge of the image sensor and a second ray to the midpoint of the lower edge of the image sensor based on the pose data and the camera optical parameters. S13, calculate the first intersection point of the first ray and the second sidewall, and the second intersection point of the second ray and the second sidewall, and then obtain the sidewall line segment between the first intersection points based on the first intersection point and the second intersection point; S14, according to the preset image overlap rate requirement, select the first reference point of the next shooting position on the side wall line segment between the first intersection points; S15, based on the spatial geometric relationship between the first reference point and the optical center of the camera, calculate the next target pitch angle required to achieve the preset image overlap rate requirement, and control the gimbal shooting device to rotate to the next target pitch angle; S16. Repeat steps S12 to S15 until the camera pitch angle reaches the preset end pitch angle, thereby completing the adaptive vertical sequence shooting starting from the initial pitch angle.

2. The method for planning the pose of real-scene modeling images of underground caverns according to claim 1, characterized in that, Step S3 includes the following steps: S31, after completing the shooting of the current cavern section, control the camera of the pan-tilt shooting device to maintain a horizontal attitude and acquire the current pose data; S32, based on the current pose data and camera optical parameters, construct a third ray from the camera optical center to the midpoint of the left edge of the image sensor, and a fourth ray from the camera optical center to the midpoint of the right edge of the image sensor. S33, calculate the third intersection point of the third ray and the first sidewall, and the fourth intersection point of the fourth ray and the first sidewall, and then obtain the sidewall line segment between the second intersection points based on the third intersection point and the fourth intersection point; S34, according to the preset image overlap rate requirement, select the second reference point on the side wall line segment between the second intersection points for the next shooting position; S35, based on the spatial geometric relationship between the second reference point and the camera's optical center, calculate the target shooting position of the gimbal shooting device near the first side wall of the next cavern's shooting section, and control the gimbal shooting device to move to the target shooting position.

3. The method for planning the pose of real-scene modeling images of underground caverns according to claim 1, characterized in that, The step S12 of obtaining the pose data for this shooting includes the following steps: S121. Establish a two-dimensional coordinate system with the transverse center of the bottom plate of the current tunnel section as the origin O, the direction from the center of the bottom plate to the second side wall as the positive X-axis, and the direction from the center of the bottom plate to the top arch as the positive Y-axis. S122, obtain the distance between the camera of the gimbal shooting device and the second side wall, the camera height and the cross-sectional width of the cave, and calculate the coordinates of the camera optical center in the two-dimensional coordinate system; S123, based on the current pitch angle of the camera, the focal length of the camera, the size of the image sensor, and the vertical viewing angle of the line connecting the midpoint of the upper edge of the image sensor to the optical center of the camera, and the vertical viewing angle of the line connecting the midpoint of the lower edge of the image sensor to the optical center of the camera, calculate the coordinates of the midpoint of the upper edge of the image sensor and the midpoint of the lower edge of the image sensor in the two-dimensional coordinate system.

4. The method for planning the pose of real-scene modeling images of underground caverns according to claim 3, characterized in that, In step S123, the coordinates of the midpoint of the upper edge of the image sensor in the two-dimensional coordinate system are (Dw / 2-DL). cos( ), H+L sin( The coordinates of the midpoint of the lower edge of the image sensor in the two-dimensional coordinate system are (Dw / 2 - Df, Hh / 2); where Dw is the cross-sectional width of the cavern, D is the object distance between the gimbal and the second sidewall, and L is the distance from the optical center of the camera to the midpoint of the upper edge of the image sensor. The current pitch angle, The vertical viewing angle is f, the camera focal length is H, the camera optical center height is h, and the image sensor height dimension is h.

5. The method for planning the pose of real-scene modeling images of underground caverns according to claim 3, characterized in that, The initial pitch angle is -45°, and the preset final pitch angle is 75°; wherein, the pitch angle is the angle between the pitch angle and the positive X-axis, with counterclockwise being positive and clockwise being negative.

6. The method for planning the pose of real-scene modeling images of underground caverns according to claim 1, characterized in that, The preset image overlap rate is 60%.

7. A device for planning the pose of real-scene modeling images of underground caverns, used to execute the pose planning method for real-scene modeling images of underground caverns as described in claim 1, characterized in that, include: The first shooting execution unit is used to sequentially execute the first sidewall shooting process and the second sidewall shooting process on the current cavern shooting section. Specifically, when the gimbal shooting device is close to the first sidewall of the cavern, the gimbal shooting device is first controlled to perform horizontal alignment so that the shooting direction is perpendicular to the central axis of the cavern. Then, the gimbal shooting device is controlled to perform adaptive vertical sequence shooting starting from the initial pitch angle to complete the shooting of the opposing second sidewall. The second shooting execution unit is used to move the gimbal shooting device close to the opposing second side wall, first control the gimbal shooting device to perform horizontal alignment again, and then control the gimbal shooting device to perform adaptive vertical sequence shooting from the initial pitch angle to complete the shooting of the opposing first side wall. The forward control unit is used to calculate the target shooting position of the gimbal shooting device near the first side wall of the next cavern shooting section based on the camera field of view parameters and preset image overlap rate requirements of the gimbal shooting device, and control the gimbal shooting device to move to the target shooting position. The process control unit is used to repeat steps S1 to S3 until any point on the excavated wall of the target cavern section reaches the target image coverage requirement.

8. A storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the underground cavern real-scene modeling image shooting pose planning method as described in any one of claims 1 to 6.

Images