A dual-camera switching AR lofting method, device, equipment and medium

Abstract

The application discloses a kind of dual camera switching AR lofting method, device, equipment and medium, belong to surveying and mapping engineering and augmented reality field, method is: real-time calculation lofting target and the plane distance between RTK host;Based on the plane distance and preset time threshold, distance threshold, determine the camera currently running, specifically: when the plane distance is greater than the distance threshold and duration is greater than or equal to the time threshold, switch to front view camera;When the plane distance is less than or equal to the distance threshold and duration is greater than or equal to the time threshold, switch to down view camera;Lofting pixel coordinates of the lofting target are calculated using the matching projection mapping model;Lofting mark is superimposed and rendered to the video picture of the camera currently running, and the video picture superimposed with the lofting mark is output to electronic notebook. By implementing the present application, camera frequent shaking switching can be prevented, and the continuity of on-site lofting operation is ensured.

Description

Technical Field

[0001] This invention relates to the fields of surveying engineering and augmented reality, and in particular to a dual-camera switching AR stakeout method, apparatus, equipment, and medium. Background Technology

[0002] In construction projects such as buildings, municipal works, and pipelines, RTK (Real-Time Kinematic) layout is a common method for setting up on-site points. Traditional RTK layout relies solely on the display of coordinates, horizontal distance, and orientation text information on the screen to guide workers to the target point. Operators need to continuously interpret the numbers and repeatedly adjust the position, resulting in poor intuitiveness and low work efficiency.

[0003] With the development of augmented reality technology, AR (augmented reality) lofting solutions are gradually being applied, which intuitively indicate the target points by overlaying lofting marks on real-time video footage.

[0004] Currently, most mainstream AR lofting devices are equipped with only a single camera with a fixed field of view, making it difficult to handle different lofting conditions at different distances: when the lofting target is far away, the field of view of the close-range lens is limited; when the target is close to the instrument, the long-range lens cannot clearly capture the ground points; when using dual cameras, the dual cameras automatically switch based on distance, often facing the problem of repeated image jumps and flickering at critical points. Summary of the Invention

[0005] This invention provides a dual-camera switching AR lofting method, apparatus, device, and medium to solve the technical problem of repeated image jumps and flickering when dual cameras automatically switch based on distance.

[0006] To address the aforementioned technical problems, embodiments of the present invention provide a dual-camera switching AR lofting method, comprising: The planar distance between the stakeout target and the RTK host is calculated in real time; wherein the RTK host is equipped with a forward-looking camera and a downward-looking camera; Based on the planar distance and preset time and distance thresholds, the currently operating camera is determined, specifically: when the planar distance is greater than the distance threshold and the duration is greater than or equal to the time threshold, and the currently operating camera is not a front-view camera, the camera is switched to a front-view camera; when the planar distance is less than or equal to the distance threshold and the duration is greater than or equal to the time threshold, and the currently operating camera is not a downward-view camera, the camera is switched to a downward-view camera. For the currently operating camera, the lofting pixel coordinates of the lofting target are calculated using a matching projection mapping model; Based on the lofting pixel coordinates, the lofting marks are superimposed and rendered onto the video image of the currently running camera, and the video image superimposed with the lofting marks is output to the electronic handheld device.

[0007] Understandably, compared to existing technologies where dual cameras automatically switch based on distance, often face the problem of repeated image jumps and flickering at critical points, this invention achieves automatic camera switching based on the planar distance between the stakeout target and the RTK host, combined with distance and time thresholds. It can adaptively select the appropriate camera according to the working conditions of near and far distances, avoiding frequent camera switching caused by small fluctuations in distance, preventing frequent camera jitter switching, improving image stability, and ensuring the continuity of on-site stakeout operations.

[0008] Furthermore, after determining the currently operating camera, the process also includes performing attitude processing on the currently operating camera, specifically: If the currently running camera is a downward-looking camera, then the conjugate of the original RTK quaternion is obtained to get the attitude quaternion of the downward-looking camera relative to the stationary coordinate system, and the attitude quaternion is converted into Euler angles. If the currently running camera is a forward-looking camera, then the conjugate of the original RTK quaternion is superimposed with the quaternion rotated 90° around the X-axis to obtain the attitude quaternion of the forward-looking camera relative to the stationary coordinate system, and the attitude quaternion is converted into Euler angles.

[0009] Understandably, the different mechanical installation angles of the front-view and downward-view cameras relative to the RTK host can lead to systematic deviations in pose calculation, ultimately causing the lofting mark positions on the AR video screen to not coincide with the actual lofting points, resulting in continuous offset errors. This invention employs differentiated pose compensation for different cameras: for the downward-view camera, only the conjugate of the original RTK quaternions is calculated; for the front-view camera, a quaternion rotated 90° around the X-axis is superimposed on the conjugate of the quaternions, yielding accurate pose quaternions for each camera in the stationary coordinate system, which are then converted to Euler angles. This method accurately compensates for the inherent installation angle deviations of the two types of cameras, eliminating spatial pose errors caused by mechanical assembly offsets. This results in more accurate subsequent conversions from geodetic coordinates to the camera coordinate system, preventing positional shifts of the lofting marks in the electronic handheld device's video screen and effectively ensuring the projection accuracy of AR lofting under dual-camera switching conditions.

[0010] Furthermore, after performing pose processing on the currently operating camera, the process also includes: The heading angle is calculated based on the Euler angles, specifically as follows: when the yaw angle in the Euler angles is greater than 0°, the heading angle is the difference between 360° and the yaw angle; otherwise, the heading angle is the opposite of the yaw angle.

[0011] Calculate the absolute value of the difference between the compass heading angle and the heading angle; when the absolute value is within a preset deviation threshold range, the RTK host issues a warning of heading inaccuracy.

[0012] Understandably, the yaw angle coordinate system output by the inertial measurement unit is inconsistent with the general azimuth coordinate system for engineering layout. Directly using the Euler yaw angle cannot intuitively represent the azimuth relationship relative to true north. After this conversion, on the one hand, the location of the layout target can be intuitively shown to the operators; on the other hand, it ensures the effective implementation of subsequent orientation consistency testing and facilitates the timely identification of abnormal attitude problems of the RTK host.

[0013] Furthermore, after calculating the lofting pixel coordinates of the lofting target using a matching projection mapping model, the process also includes: If the currently operating camera is a front-view camera, obtain the camera depth value of the lofting target; when the camera depth value is less than or equal to 0, determine that the lofting target is behind the device, fix the lofting mark in the center of the image, and at the same time perform magnification processing on the lofting pixel coordinates.

[0014] Understandably, this serves two purposes: firstly, it avoids rendering inconsistencies and invalid coordinates due to the target being positioned behind the machine; secondly, it provides a clear indication to operators that the target is behind the RTK host, guiding them to turn around and locate the point, thus resolving the issue of ineffective lofting markers in scenarios where the target is positioned behind the machine, and improving the lofting guidance capability across all scenarios.

[0015] Furthermore, the step of calculating the lofted target pixel coordinates using a matching projection mapping model for the currently operating camera includes: If the currently running camera is a downward-facing camera, a full-screen linear mapping is performed from the original CCD resolution to the screen resolution of the electronic handbook. Based on the horizontal and vertical coordinates of the lofted target on the CCD image, the horizontal and vertical coordinates of the lofted pixels are calculated. If the currently running camera is a front-view camera, the original image resolution of the front-view camera is mapped to the window display resolution to calculate the horizontal coordinate of the lofted pixels and the vertical coordinate of the base screen. The vertical coordinate of the base screen is then superimposed with a vertical centering offset to obtain the vertical coordinate of the lofted pixels. The window display resolution is the image resolution of the video window in the electronic handheld device, and the vertical centering offset is half the difference between the total screen height and the display height of the front-view image.

[0016] Understandably, independent projection mapping models are configured for the front-view and lower-view cameras due to their different imaging resolutions and display strategies. The lower-view camera uses full-screen linear mapping to adapt to the full-screen size of the electronic handheld device, while the front-view camera incorporates a vertical centering offset based on coordinate conversion to adapt to the windowed display mode. These two independent projection mapping models match the hardware imaging characteristics and rendering rules of the two cameras, completing the conversion of target spatial points to two-dimensional pixel positions on the screen. This avoids coordinate adaptation errors caused by sharing a single conversion model, ensuring continuous and accurate calculation results for lofted pixel coordinates before and after camera switching. This invention improves the projection accuracy of lofted marks by configuring independent projection mapping models.

[0017] Furthermore, determining the currently operating camera includes: If no camera is currently running, start and run the downward-facing camera.

[0018] Understandably, this solution prioritizes using a downward-facing camera for data collection, automatically activating it when no other camera is available. The downward-facing camera is positioned towards the ground layout area, with its field of view aligned with the pole tips and layout points, ensuring stable capture of the ground scene and guaranteeing that the AR layout guidance image remains consistently aligned with the construction points.

[0019] Furthermore, after identifying the currently operating camera, the following is also included: If the camera is switched, the RTK host obtains the updated video stream from the camera and re-establishes the video stream connection with the electronic handheld device.

[0020] Understandably, after the camera completes the switch, the video stream connection between the RTK host and the electronic handheld device is re-established, which can smoothly obtain the real-time image data output by the currently running camera, ensure that the data source of the video screen is normally replaced before the switch, avoid abnormal phenomena such as video stuttering, black screen, and image source disorder, ensure the continuous and stable output of AR video screen, and maintain the continuity of the layout operation.

[0021] Another embodiment of the present invention provides a dual-camera switching AR lofting device, including: a distance calculation module, a camera switching module, a coordinate calculation module, and an AR rendering output module; The distance calculation module is used to calculate the planar distance between the stakeout target and the RTK host in real time; wherein the RTK host is equipped with a forward-looking camera and a downward-looking camera; The camera switching module is used to determine the currently running camera based on the planar distance and a preset time threshold and distance threshold. Specifically, when the planar distance is greater than the distance threshold and the duration is greater than or equal to the time threshold, and the currently running camera is not a front-view camera, the camera is switched to a front-view camera; when the planar distance is less than or equal to the distance threshold and the duration is greater than or equal to the time threshold, and the currently running camera is not a downward-view camera, the camera is switched to a downward-view camera. The coordinate calculation module is used to calculate the lofting pixel coordinates of the lofting target using a matching projection mapping model for the currently running camera. The AR rendering output module is used to overlay the lofting marks onto the video image of the currently running camera according to the lofting pixel coordinates, and output the video image with the overlaying lofting marks to the electronic handheld device.

[0022] Another embodiment of the present invention provides a terminal device, including: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the steps of the dual-camera switching AR lofting method of the present invention.

[0023] Another embodiment of the present invention also provides a computer-readable storage medium item, including: a stored computer program, which, when the computer program is running, controls the device where the computer-readable storage medium is located to perform the steps of the dual-camera switching AR lofting method of the present invention. Attached Figure Description

[0024] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0025] Figure 1 This is a flowchart illustrating a dual-camera switching AR lofting method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a dual-camera switching AR lofting device according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a hysteresis buffering mechanism provided in an embodiment of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the term "comprising" and any variations thereof in the specification, claims and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0028] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0029] Please refer to Figure 1 To address the issue of repeated image jumps and flickering at critical points in existing dual-camera automatic switching based on distance, an embodiment of the present invention provides a dual-camera switching AR lofting method, which includes steps S100 to S400, each step of which is as follows: S100, Real-time calculation of the planar distance between the stakeout target and the RTK host; wherein, the RTK host is equipped with a forward-looking camera and a downward-looking camera; It should be noted that the forward-looking camera is adapted for long-distance layout, with the lens field of view facing the design point in the distance; the layout target is more likely to be kept in the frame under long-distance working conditions; it is suitable for long-distance line layout and long-distance point location; the downward-looking camera is adapted for close-distance layout, with the lens facing the ground working area and the tip of the measuring pole; when fine-tuning layout at close distance, it can stably capture the real scene at the pole tip, which is suitable for fine-tuning of points and close-distance calibration.

[0030] Understandably, real-time calculation of the planar distance between the stakeout target and the RTK host provides accurate and reliable data for subsequent camera switching, improving the continuity and reliability of stakeout guidance at different working distances.

[0031] S200. Based on the planar distance and preset time threshold and distance threshold, determine the currently running camera, specifically: when the planar distance is greater than the distance threshold and the duration is greater than or equal to the time threshold, and the currently running camera is not a front-view camera, switch to the front-view camera; when the planar distance is less than or equal to the distance threshold and the duration is greater than or equal to the time threshold, and the currently running camera is not a downward-view camera, switch to the downward-view camera. like Figure 3 As shown, it should be noted that in this embodiment, the distance threshold is set to 10m and the time threshold is set to 3s. Initially, no camera is running, so the downward-facing camera is started and running. Within 0-4s, the planar distance is less than 10m, so there is no need to switch cameras. At the end of the 4th second, the planar distance begins to exceed 10m, but the duration is less than the time threshold, so no camera switching is performed. Within 4-7s, the planar distance is always greater than 10m, and the currently running camera is the downward-facing camera, so the downward-facing camera is switched to the forward-facing camera.

[0032] It should be noted that the present invention is also equipped with a front-facing camera counter and a lower-facing camera counter. When the planar distance is greater than a distance threshold, the front-facing camera counter is incremented by 1 every 1 second, while the lower-facing camera counter is reset to zero. When the planar distance is less than a distance threshold, the lower-facing camera counter is incremented by 1 every 1 second, while the front-facing camera counter is reset to zero. When the front-facing camera counter is greater than or equal to a time threshold and the currently operating camera is not a front-facing camera, the system switches to the front-facing camera. When the lower-facing camera counter is greater than or equal to a time threshold and the currently operating camera is not a lower-facing camera, the system switches to the lower-facing camera.

[0033] Understandably, during site layout, slight personnel movement, minor ground elevation differences, and measurement noise can cause the planar distance to fluctuate around the switching threshold. This invention constructs a hysteresis buffer mechanism using planar distance, a preset distance threshold, and a time threshold. Camera switching is only performed after the planar distance consistently meets the switching conditions and reaches the distance threshold. This hysteresis buffer mechanism effectively avoids frequent camera switching caused by short-term distance oscillations, preventing frequent video stream reconnection and image flickering, ensuring stable output of AR layout images, and improving the viewing experience. Simultaneously, it differentiates between near and far working conditions based on planar distance: a forward-looking camera is used for far-distance conditions, and a downward-looking camera is used for near-distance conditions, achieving adaptive scheduling of dual cameras to meet both the needs of long-distance point observation and close-range precise ground layout.

[0034] Preferably, if the currently operating camera is a downward-looking camera, the conjugate of the original RTK quaternion is obtained to get the attitude quaternion of the downward-looking camera relative to the stationary coordinate system, and the attitude quaternion is converted into Euler angles. If the currently running camera is a forward-looking camera, then the conjugate of the original RTK quaternion is superimposed with the quaternion rotated 90° around the X-axis to obtain the attitude quaternion of the forward-looking camera relative to the stationary coordinate system, and the attitude quaternion is converted into Euler angles.

[0035] It should be noted that, due to the fixed mechanical mounting angle of 90° around the X-axis between the forward-looking camera and the IMU, the quaternion of rotation around the X-axis of 90° is superimposed to compensate for the attitude deviation caused by the hardware installation. Finally, the attitude quaternion of the forward-looking camera relative to the stationary coordinate system is accurately solved, ensuring the accuracy of subsequent coordinate projection calculations.

[0036] Further, the heading angle is calculated based on the Euler angles, specifically as follows: when the yaw angle in the Euler angles is greater than 0°, the heading angle is the difference between 360° and the yaw angle; otherwise, the heading angle is the opposite of the yaw angle; the absolute value of the difference between the compass heading angle and the heading angle is calculated; when the absolute value is within a preset deviation threshold range, the RTK host issues a warning of heading inaccuracy.

[0037] It is understandable that the construction site environment is susceptible to factors such as magnetic field interference and IMU (Inertial Measurement Unit) measurement drift, which can cause attitude data distortion. Timely alarms when the deviation between two sets of heading data is too large effectively alert operators to unreliable current attitude information, preventing distorted layout pixel coordinates calculated based on incorrect poses and avoiding misaligned layout marks that could mislead construction. This solution performs heading angle standard conversion based on the positive and negative relationship of yaw angles, unifying the angle expression format and avoiding comparison failures caused by angle ranges spanning 0° / 360°. Real-time calculation of heading difference triggers a heading inaccuracy warning when the deviation exceeds the allowable range, enabling construction personnel to promptly recognize significant errors in the RTK host's heading and proactively avoid problems such as distorted AR projection coordinate calculations and misaligned layout marks caused by abnormal attitude data, thus improving the reliability of AR layout operations.

[0038] Preferably, if no camera is currently running, the downward-facing camera is started and operated.

[0039] Furthermore, if the camera is switched, the RTK host obtains the updated video stream from the camera and re-establishes a video stream connection with the electronic handheld device.

[0040] Understandably, when switching cameras, the RTK host obtains the new video stream corresponding to the switched camera and re-establishes a video transmission connection with the electronic handheld device. This ensures that the video data source is updated synchronously during the switching between the front-view and downward-view cameras, avoiding the use of the video data source from the previous camera. It also ensures that the electronic handheld device receives the real-time image from the currently running camera, allowing AR lofting markers to be accurately superimposed on the latest real-world video, preventing misalignment of virtual and real point positions caused by inconsistent image sources.

[0041] S300. For the currently operating camera, calculate the lofting pixel coordinates of the lofting target using a matching projection mapping model; Understandably, this invention uses a dedicated projection mapping model to calculate the pixel coordinates of the stakeout target for the currently operating camera, avoiding coordinate projection deviations caused by mixing camera models; ensuring the accuracy of the mapping relationship between the screen pixel coordinates during the entire process of converting the geodetic coordinates, improving the positioning accuracy of the AR guide markers, preventing guide point offsets and misalignment, and ensuring the accuracy of the AR guide screen after switching between near and far stakeout cameras.

[0042] Furthermore, the three-dimensional coordinates of the target under the camera coordinate system are substituted into the pinhole imaging model. Combined with the camera intrinsic parameters (horizontal focal length, vertical focal length, and principal point coordinates of the image) obtained by the pre-calibration of the current working camera, the CCD pixel coordinates on the photosensitive plane are calculated. If the currently running camera is a front-view camera, it is determined whether the CCD pixel coordinates exceed the original image resolution range of the front-view camera. When the CCD horizontal coordinate is less than 0 and the CCD vertical coordinate is less than 0, or the CCD horizontal coordinate is greater than the original horizontal resolution of the front-view camera CCD and the CCD vertical coordinate is greater than the original vertical resolution of the front-view CCD, it is determined that the projection point corresponding to the target exceeds the video screen of the front-view camera.

[0043] Understandably, the over-screen judgment mechanism can objectively and accurately reflect whether the lofting target is within the actual imaging field of view of the forward-facing camera, avoiding misjudgments caused by using screen coordinates to determine boundaries. Accurate field-of-view over-boundary recognition results can provide a reliable basis for subsequent logic such as solving screen boundary intersections and reverse target guidance, ensuring the accurate triggering of AR lofting guidance strategies and improving the reliability of AR visualization guidance.

[0044] Preferably, if the currently operating camera is a front-view camera, the camera depth value of the lofting target is obtained; when the camera depth value is less than or equal to 0, it is determined that the lofting target is behind the device, the lofting mark is fixed in the center of the image, and the lofting pixel coordinates are magnified.

[0045] Understandably, when the target is behind the equipment, the pinhole camera projection cannot generate valid visible coordinates. This solution fixes the layout mark in the center of the image and magnifies the layout pixel coordinates, providing directional guidance to the operator. This solves the problem of AR marks disappearing and lacking guidance information when the target is behind the equipment, enabling reverse guidance of the rear point. This allows construction personnel to quickly turn the RTK host towards the target point, improving the continuity of the layout operation.

[0046] Furthermore, if the currently operating camera is a downward-facing camera, a full-screen linear mapping is performed from the original CCD resolution to the screen resolution of the electronic handheld device. Based on the horizontal and vertical coordinates of the lofted target on the CCD image, the horizontal and vertical coordinates of the lofted pixels are calculated. If the currently running camera is a front-view camera, the original image resolution of the front-view camera is mapped to the window display resolution to calculate the horizontal coordinate of the lofted pixels and the vertical coordinate of the base screen. The vertical coordinate of the base screen is then superimposed with a vertical centering offset to obtain the vertical coordinate of the lofted pixels. The window display resolution is the image resolution of the video window in the electronic handheld device, and the vertical centering offset is half the difference between the total screen height and the display height of the front-view image.

[0047] Understandably, the downward-facing camera uses a full-screen linear mapping from the CCD's original resolution to the screen resolution, ensuring distortion-free close-range ground layout images and uniform and accurate point mapping, suitable for close-range fine-tuning operations at pole tips. The forward-facing camera, while not displaying full-screen, uses a dedicated window resolution mapping and a fixed vertical centering offset to ensure the forward-facing, distant view is centered and neatly displayed on the screen, effectively eliminating the vertical coordinate offset problem caused by the non-full-screen display of the forward-facing camera. Through independent mapping rules for both cameras, it adapts to the different imaging layouts and display mechanisms of the two types of cameras, avoiding pixel projection deviations caused by a unified mapping model. This significantly improves the matching accuracy between layout marks and actual point locations under different working conditions at varying distances, ensuring stable and misaligned AR guidance positions.

[0048] S400. Based on the lofting pixel coordinates, the lofting marks are superimposed and rendered onto the video image of the currently running camera, and the video image superimposed with the lofting marks is output to the electronic handheld device.

[0049] Understandably, operators can obtain real-time visual guidance by relying on electronic handbooks, which reduces the difficulty of setting out operations and improves the efficiency of setting out operations.

[0050] like Figure 2 As shown, based on the above-described method embodiments, an embodiment of the present invention provides a dual-camera switching AR lofting device, which includes modules S1 to S4, and the specific details of each module are as follows: The distance calculation module S1 is used to calculate the planar distance between the stakeout target and the RTK host in real time; wherein, the RTK host is equipped with a forward-looking camera and a downward-looking camera; The camera switching module S2 is used to determine the currently running camera based on the planar distance and a preset time threshold and distance threshold. Specifically, when the planar distance is greater than the distance threshold and the duration is greater than or equal to the time threshold, and the currently running camera is not a front-view camera, the camera is switched to a front-view camera; when the planar distance is less than or equal to the distance threshold and the duration is greater than or equal to the time threshold, and the currently running camera is not a downward-view camera, the camera is switched to a downward-view camera. The coordinate calculation module S3 is used to calculate the lofting pixel coordinates of the lofting target using a matching projection mapping model for the currently running camera. AR rendering output module S4 is used to overlay and render the lofting marks onto the video image of the currently running camera according to the lofting pixel coordinates, and output the video image with the overlaying lofting marks to the electronic handheld device.

[0051] It is understood that the above-described device embodiments correspond to the method embodiments of the present invention, and can implement the dual-camera switching AR lofting method provided by any of the above-described method embodiments of the present invention.

[0052] It should be noted that the device embodiments described above are merely illustrative, and some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0053] Based on the above-described method embodiments, another embodiment of the present invention provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements a dual-camera switching AR lofting method according to any embodiment of the present invention.

[0054] For example, in this embodiment, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the terminal device.

[0055] The terminal device can be a desktop computer, laptop, handheld computer, or cloud server, etc.

[0056] The terminal device may include, but is not limited to, a processor and a memory.

[0057] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0058] Based on the above-described method embodiments, another embodiment of the present invention provides a computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute a dual-camera switching AR lofting method as described in any of the above-described method embodiments of the present invention.

[0059] The modules / units integrated in the device / terminal equipment, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0060] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A dual-camera switching AR lofting method, characterized in that, include: The planar distance between the stakeout target and the RTK host is calculated in real time; wherein the RTK host is equipped with a forward-looking camera and a downward-looking camera; Based on the planar distance and preset time and distance thresholds, the currently operating camera is determined, specifically: when the planar distance is greater than the distance threshold and the duration is greater than or equal to the time threshold, and the currently operating camera is not a front-view camera, the camera is switched to a front-view camera; when the planar distance is less than or equal to the distance threshold and the duration is greater than or equal to the time threshold, and the currently operating camera is not a downward-view camera, the camera is switched to a downward-view camera. For the currently operating camera, the lofting pixel coordinates of the lofting target are calculated using a matching projection mapping model; Based on the lofting pixel coordinates, the lofting marks are superimposed and rendered onto the video image of the currently running camera, and the video image superimposed with the lofting marks is output to the electronic handheld device.

2. The dual-camera switching AR lofting method as described in claim 1, characterized in that, After identifying the currently operating camera, the process also includes performing attitude processing on the currently operating camera, specifically: If the currently running camera is a downward-looking camera, then the conjugate of the original RTK quaternion is obtained to get the attitude quaternion of the downward-looking camera relative to the stationary coordinate system, and the attitude quaternion is converted into Euler angles. If the currently running camera is a forward-looking camera, then the conjugate of the original RTK quaternion is superimposed with the quaternion rotated 90° around the X-axis to obtain the attitude quaternion of the forward-looking camera relative to the stationary coordinate system, and the attitude quaternion is converted into Euler angles.

3. The dual-camera switching AR lofting method as described in claim 2, characterized in that, After performing pose processing on the currently running camera, the following is also included: The heading angle is calculated based on the Euler angles, specifically as follows: when the yaw angle in the Euler angles is greater than 0°, the heading angle is the difference between 360° and the yaw angle; otherwise, the heading angle is the opposite of the yaw angle. Calculate the absolute value of the difference between the compass heading angle and the heading angle; when the absolute value is within a preset deviation threshold range, the RTK host issues a warning of heading inaccuracy.

4. The dual-camera switching AR lofting method as described in claim 1, characterized in that, After calculating the lofted pixel coordinates of the lofted target using a matching projection mapping model, the process further includes: If the currently operating camera is a front-view camera, obtain the camera depth value of the lofting target; when the camera depth value is less than or equal to 0, determine that the lofting target is behind the device, fix the lofting mark in the center of the image, and at the same time perform magnification processing on the lofting pixel coordinates.

5. The dual-camera switching AR lofting method as described in claim 1, characterized in that, The step of calculating the lofted pixel coordinates of the lofted target using a matching projection mapping model for the currently operating camera includes: If the currently running camera is a downward-facing camera, a full-screen linear mapping is performed from the original CCD resolution to the screen resolution of the electronic handbook. Based on the horizontal and vertical coordinates of the lofted target on the CCD image, the horizontal and vertical coordinates of the lofted pixels are calculated. If the currently running camera is a front-view camera, the original image resolution of the front-view camera is mapped to the window display resolution to calculate the horizontal coordinate of the lofted pixels and the vertical coordinate of the base screen. The vertical coordinate of the base screen is then superimposed with a vertical centering offset to obtain the vertical coordinate of the lofted pixels. The window display resolution is the image resolution of the video window in the electronic handheld device, and the vertical centering offset is half the difference between the total screen height and the display height of the front-view image.

6. The dual-camera switching AR lofting method as described in claim 1, characterized in that, The process of determining the currently operating camera includes: If no camera is currently running, start and run the downward-facing camera.

7. A dual-camera switching AR lofting method as described in any one of claims 1-6, characterized in that, After determining the currently operating camera, the following is also included: If the camera is switched, the RTK host obtains the updated video stream from the camera and re-establishes the video stream connection with the electronic handheld device.

8. A dual-camera switching AR lofting device, characterized in that, include: Distance calculation module, camera switching module, coordinate calculation module, and AR rendering output module; The distance calculation module is used to calculate the planar distance between the stakeout target and the RTK host in real time; wherein, the RTK host is equipped with a forward-looking camera and a downward-looking camera; The camera switching module is used to determine the currently running camera based on the planar distance and a preset time threshold and distance threshold. Specifically, when the planar distance is greater than the distance threshold and the duration is greater than or equal to the time threshold, and the currently running camera is not a front-view camera, the camera is switched to a front-view camera; when the planar distance is less than or equal to the distance threshold and the duration is greater than or equal to the time threshold, and the currently running camera is not a downward-view camera, the camera is switched to a downward-view camera. The coordinate calculation module is used to calculate the lofting pixel coordinates of the lofting target using a matching projection mapping model for the currently running camera. The AR rendering output module is used to overlay the lofting marks onto the video image of the currently running camera according to the lofting pixel coordinates, and output the video image with the overlaying lofting marks to the electronic handheld device.

9. A terminal device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements the dual-camera switching AR lofting method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, include: A stored computer program, wherein, when the computer program is executed, it controls the device containing the computer-readable storage medium to perform the dual-camera switching AR lofting method as described in any one of claims 1-7.

Images