Methods, apparatus, equipment and storage media for remote image stakeout and non-contact point acquisition.

Abstract

This invention discloses a method, apparatus, device, and storage medium for remote image lofting and non-contact point acquisition. The method includes: constructing and caching corresponding spatial registration data for each image frame, the spatial registration data including at least image depth data, scene local 3D point cloud data, device pose data when acquiring the image frame, and coordinate system transformation parameters; determining the target pixel and its pixel coordinates selected by the user in the image frame; determining the target depth of the target pixel based on the image depth data or scene local 3D point cloud data associated with the image frame; determining the target 3D point of the target pixel in the engineering coordinate system based on the pixel coordinates, target depth, pose data, and coordinate system transformation parameters of the target pixel; outputting the coordinates of the target 3D point as the non-contact point acquisition result; and / or comparing the target 3D point coordinates with the design point coordinates to generate deviation data, and outputting lofting guidance information based on the deviation data.

Description

Technical Field

[0001] This invention relates to the fields of engineering surveying, 3D scanning, spatial positioning, AR guidance and intelligent construction technology, and in particular to a method, device, electronic device and storage medium for remote image layout and non-contact point acquisition. Background Technology

[0002] Point acquisition refers to obtaining three-dimensional coordinate data from the target location on site; layout refers to finding the corresponding location of the design coordinates on site and guiding the work. The two are the core links in engineering surveying, construction positioning, and machinery guidance.

[0003] Existing point acquisition and layout methods mostly use equipment such as RTK and total station, which require operators to approach the target point for contact measurement and rely on manual point-by-point operation. However, dangerous / obstructed / high / deep pit scenarios are difficult to reach, making it impossible to achieve non-contact, high-efficiency remote layout and point acquisition, resulting in poor applicability and reliability. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, storage medium, and computer program product for remote image lofting and non-contact point acquisition.

[0005] According to one aspect of the present invention, a method for remote image lofting and non-contact point acquisition is provided, comprising: For each image frame, corresponding spatial registration data is constructed and cached; wherein, the spatial registration data is used to establish the mapping relationship between image points and three-dimensional spatial points, and includes at least: image depth data, local three-dimensional point cloud data of the scene, pose data of the device when acquiring the image frame, and coordinate system transformation parameters; wherein, the image frame includes at least one of real-time image frame, video frame, scan preview frame, and playback image frame; In response to the user's selection of a target object in the image frame, determine the target pixel selected by the user and the pixel coordinates of the target pixel; Acquire spatial registration data associated with image frames, and determine the target depth corresponding to the target pixel based on the image depth data or local 3D point cloud data of the scene in the spatial registration data; Based on the pixel coordinates, target depth, pose data, and coordinate system transformation parameters of the target pixel, determine the target 3D point in the engineering coordinate system; Output the coordinates of the target 3D point as a non-contact point acquisition result; and / or The coordinates of the target 3D point are compared with the coordinates of the target layout point, the current equipment position, the current work point, or the design point in the construction design information to generate deviation data, and the layout guidance information is output based on the deviation data.

[0006] According to another aspect of the present invention, an image remote lofting and non-contact point acquisition device is provided, comprising: The data construction module is used to construct and cache the corresponding spatial registration data for each image frame. The spatial registration data is used to establish the mapping relationship between image points and three-dimensional spatial points, and includes at least: image depth data, local three-dimensional point cloud data of the scene, pose data of the device when acquiring the image frame, and coordinate system transformation parameters. The image frame includes at least one of real-time image frames, video frames, scan preview frames, and playback image frames. The response processing module is used to respond to the user's point selection operation on the target object in the image frame, and to determine the target pixel selected by the user and the pixel coordinates of the target pixel; The data acquisition and processing module is used to acquire spatial registration data associated with image frames, and determine the target depth corresponding to the target pixel based on the image depth data or local 3D point cloud data of the scene in the spatial registration data. The calculation module is used to determine the target 3D point of the target pixel in the engineering coordinate system based on the pixel coordinates of the target pixel, the target depth, the pose data and the coordinate system transformation parameters. The point acquisition output module is used to output the coordinates of the target 3D point as a non-contact point acquisition result; and / or The layout guidance module is used to compare the coordinates of the target 3D point with the coordinates of the target layout point, the current equipment position, the current work point, or the design point in the construction design information, generate deviation data, and output layout guidance information based on the deviation data.

[0007] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and A memory that is communicatively connected to at least one processor; wherein, The memory stores a computer program that can be executed by at least one processor, such that the at least one processor is able to execute the image remote lofting and non-contact point acquisition method of the present invention.

[0008] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions for causing a processor to execute and implement the image remote lofting and non-contact point acquisition method of the present invention.

[0009] According to another aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the steps in the above-described method.

[0010] The technical solution of this invention enables visualized, non-contact, and remote point selection; it supports reviewing point selection without requiring real-time on-site operation; and it improves operational safety in hazardous areas, high-altitude environments, and deep pits. It solves the problems of traditional methods requiring physical contact, low efficiency, and poor applicability to hazardous scenarios. It also overcomes the shortcomings of single-sensor depth measurement and insufficient accuracy, exhibiting strong robustness, wide applicability, ease of operation, and high safety and efficiency.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart illustrating a remote image lofting and non-contact point acquisition method provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating another method for remote image lofting and non-contact point acquisition provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an image remote lofting and non-contact point acquisition device provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device that implements the image remote lofting and non-contact point acquisition method according to embodiments of the present invention. Detailed Implementation

[0014] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0015] Example 1 Figure 1This is a flowchart of an image remote lofting and non-contact point acquisition method provided by an embodiment of the present invention. This embodiment can be applied to scenarios of point acquisition and / or lofting. The method can be executed by an image remote lofting and non-contact point acquisition device, which can be implemented in hardware and / or software and can be configured in an electronic device.

[0016] like Figure 1 As shown, the methods for remote image lofting and non-contact point acquisition include: S101. Construct and cache corresponding spatial registration data for each image frame; wherein, the spatial registration data is used to establish the mapping relationship between image points and three-dimensional spatial points, and includes at least: image depth data, local three-dimensional point cloud data of the scene, pose data of the device when acquiring the image frame, and coordinate system transformation parameters. S102. In response to the user's selection operation of the target object in the image frame, determine the target pixel selected by the user and the pixel coordinates of the target pixel; S103. Obtain spatial registration data associated with image frames, and determine the target depth corresponding to the target pixel based on the image depth data or local three-dimensional point cloud data of the scene in the spatial registration data. S104. Based on the pixel coordinates, target depth, pose data and coordinate system transformation parameters of the target pixel, determine the target three-dimensional point of the target pixel in the engineering coordinate system. S105, Output the coordinates of the target 3D point as a non-contact point acquisition result; and / or S106. Compare the coordinates of the target three-dimensional point with the coordinates of the target layout point, the current equipment position, the current work point, or the design point in the construction design information to generate deviation data, and output layout guidance information based on the deviation data.

[0017] In step S101, an image frame refers to at least one of real-time image frames, video frames, scan preview frames, and playback image frames, used for user interactive selection. Since an image frame can be a playback image frame, this invention supports offline point acquisition operations on cached historical video frames / playback image frames. It eliminates the need to restart the device on-site to capture real-world images; relying on pre-cached spatial registration data for each frame, non-contact point acquisition and stakeout calculations can be completed by directly retrieving historical frame data, enabling offline playback measurement. Spatial registration data is a set of data used to establish the mapping relationship between image points and three-dimensional spatial points. Image depth data is the three-dimensional spatial distance data corresponding to each pixel in the image. Scene local three-dimensional point cloud data is a set of local three-dimensional points obtained by converting effective depth pixels. Device pose data is the position and attitude information of the device when acquiring image frames. Coordinate system transformation parameters include camera intrinsic parameters, camera extrinsic parameters, device pose parameters, and world engineering registration parameters.

[0018] In some embodiments, spatial registration data corresponding to each image frame is constructed, including S1011-S1015: S1011. For a single image frame, obtain the image depth data corresponding to the image frame; wherein, the image depth data is obtained by fusing two types of original depth information, namely, the binocular depth and the dToF depth, corresponding to the imaging of the image frame.

[0019] Specifically, the binocular depth is the pixel distance value obtained by correcting, stereo matching, and calculating parallax from the left and right views of the binocular camera. The dToF depth is the pixel distance value calculated by the time-of-flight method using modulated light emitted and reflected signals received by the dToF module. The preset fusion rule is a strategy that assigns weights to the binocular depth and dToF depth based on distance, texture, and confidence level, and then sums them. It should be noted that depth fusion is used to provide effective depth for image frames. The key innovation of this case lies in the calculation and lofting guidance of image point selection to engineering coordinates based on spatial registration data.

[0020] Specifically, when acquiring the current single-frame image, the unique exposure timestamp t of that frame is recorded immediately. All subsequent sensor data are aligned based on this timestamp to ensure that the data are from the same source and in the same order.

[0021] Binocular depth calculation process: Distortion and epipolar correction are performed on the left and right images acquired by the binocular camera; stereo matching is performed on the corrected images to calculate the disparity value of each pixel; based on the relationship between the camera baseline, focal length, and disparity, the disparity is converted into depth to obtain the binocular depth map. For near-field areas with rich textures, such as walls, floors, and component surfaces, binocular depth sensors can output dense and detailed depth values.

[0022] dToF depth calculation process: The dToF module acquires the raw depth and signal strength; performs multipath suppression, noise filtering, outlier removal, and confidence level selection; outputs a valid dToF depth map. For scenes with weak textures, distant areas, reflective surfaces, and occluded edges, dToF can still maintain stable depth output.

[0023] Timestamp alignment is based on the image frame timestamp t. The closest stereo depth and dToF depth data to t are found from the data queue to complete time synchronization.

[0024] Deep fusion calculation process: Weighted fusion is performed on each pixel (u,v) to obtain image depth data. : ; in, , For adaptive weighting; at close range, binocular details are better, improving... reduce At long distances, dToF is more stable at scale, improving... ,reduce Weighting criteria: distance, texture intensity, reflectivity, depth continuity, and confidence level.

[0025] The output timestamped fused depth map finally yields the image depth data for that image frame.

[0026] The benefits of this step are: it combines the advantages of binoculars and dToF to solve the problems of missing depth, abrupt changes, and high noise in single sensors; and it ensures strict timestamp alignment to guarantee that multi-source data is without misalignment or lag.

[0027] S1012. Based on the image depth data of the image frame, and combined with the camera intrinsic parameters in the coordinate system transformation parameters, the pixels with effective depth values ​​in the frame are converted into three-dimensional points in the camera coordinate system, and the three-dimensional points are summarized to generate the local three-dimensional point cloud data of the scene corresponding to the image frame.

[0028] Among them, the effective depth value is the depth value that meets the preset confidence conditions and has no anomalies. The 3D point position in the camera coordinate system is the coordinate of a 3D point in space with the camera's optical center as the origin. The local 3D point cloud data of the scene is the set of local 3D points obtained by back-projection of the effective depth pixels.

[0029] Specifically, iterate through all pixels in the image frame and filter out pixels with valid depth values. Obtain the camera intrinsic parameter matrix K: ;in , For camera focal length, , The coordinates of the image points are used. The effective depth pixels are back-projected according to the camera intrinsic parameters to calculate the 3D point positions in the camera coordinate system. : ;in, Let be the homogeneous coordinates of the pixel; z is the pixel's fusion depth. The 3D points in the camera coordinate system obtained by backprojecting all effective pixels are summarized to form the local 3D point cloud data of the scene corresponding to this image frame. This transforms the 2D depth map into a set of 3D spatial points, providing the geometric basis for subsequent depth completion and coordinate transformation.

[0030] S1013. Based on the generated local 3D point cloud data of the scene, perform depth padding on pixels that have no effective depth value within the image frame, and construct the overall correspondence between image points within the frame and 3D spatial points.

[0031] Pixels without effective depth values ​​are those whose depth is missing due to weak texture, occlusion, reflection, or distance. Depth padding is the process of estimating the missing depth using point cloud neighborhood information.

[0032] Specifically, the generated local 3D point cloud of the scene is back-projected onto the image plane to establish a one-to-one correspondence between 3D points and pixels. For pixels in the image frame that do not have effective depth values, such as weakly textured walls, smooth reflective components, distant areas, occlusion boundaries, depth holes, and shadow areas, a search is conducted within the neighborhood of the target pixel for matching effective depth points.

[0033] Subsequently, methods such as nearest neighbor interpolation, weighted average interpolation, bilinear interpolation, local plane fitting, or intersection of camera rays and local surfaces are used to estimate and fill in the missing depth. For example, in areas where depth is easily missing, such as corners, trench edges, and component edges, the system fits a local plane to the neighborhood point cloud, intersects the camera ray with the plane, obtains accurate 3D position and depth values, and then constructs a complete correspondence between image points and 3D spatial points.

[0034] It should be noted that during this depth completion step, if a large area of ​​a pixel's neighborhood is in a state of weak texture, strong reflection, long distance, or complete occlusion, and the number of effective point clouds in the neighborhood does not reach the fitting calculation threshold, or if the pixel crosses the edge of a component or is in a suspended area without a solid body, causing the fitting model to fail, the corresponding depth completion residual will be too large and the confidence level will be too low. When the S103 check finds that the confidence level of the target pixel's completion depth is lower than the preset threshold, the completion depth of that point will be judged as invalid depth. Therefore, even after this full-frame pre-completion processing, there will still be objective situations where the target pixel lacks effective depth when the user selects it. For such invalid points, the depth needs to be recalculated.

[0035] The benefits of this step are: it solves problems such as missing depth, depth holes, and discontinuous edges, enabling stable calculation of 3D coordinates at any selected point location; it significantly improves the success rate of point acquisition in complex scenes such as weak texture, reflection, occlusion, and long distance; and it ensures full coverage mapping between the image and 3D space, improving the robustness and applicability of non-contact point acquisition.

[0036] S1014. Combining the device's posture and spatial position information at the moment the image frame was captured, determine the corresponding pose data, and determine the conversion relationship between the camera coordinate system, device coordinate system, world coordinate system, and engineering coordinate system to obtain the coordinate system transformation parameters.

[0037] Among them, the camera coordinate system is a coordinate system with the camera's optical center as the origin. The equipment coordinate system is a coordinate system based on the equipment itself. The world coordinate system is a locally unified coordinate system for equipment operation. The engineering coordinate system is a global absolute coordinate system used for construction, design, and RTK. Coordinate system transformation parameters include: camera intrinsic parameter K and camera extrinsic parameter. Equipment position World Engineering Registration Parameters .

[0038] By combining the device's attitude, position, and orientation information at the moment the current image frame was captured, the device pose data is determined. This data can be obtained through fusion positioning using IMU, SLAM, stereo cameras, and dToF. Simultaneously, a conversion relationship between multiple coordinate systems is established: the conversion from the camera coordinate system to the device coordinate system is achieved through camera extrinsic parameters; the conversion from the device coordinate system to the world coordinate system is achieved through device pose parameters; the conversion parameter from the world coordinate system to the engineering coordinate system is defined as the world engineering registration parameter. This registration parameter can be generated based on on-site control points, known benchmark points, engineering calibration points, or historical project registration data. The conversion from the world coordinate system to the engineering coordinate system is achieved through the world engineering registration parameter. If RTK absolute coordinate constraints are integrated, the system uses the geographic coordinates or engineering coordinates provided by RTK to calibrate the world engineering registration parameter, ensuring that the final coordinates are strictly aligned with the construction design coordinate system. Finally, the camera intrinsic parameters, camera extrinsic parameters, device pose parameters, and world engineering registration parameters are uniformly encapsulated into coordinate system transformation parameters, forming a complete transformation link from image pixels to engineering coordinates.

[0039] The beneficial effects are: establishing a complete conversion path from two-dimensional points in an image to absolute coordinates in an engineering project, enabling the point acquisition results to be directly used in actual operations such as engineering surveying, construction layout, and equipment guidance; supporting RTK global constraints to ensure high coordinate accuracy and strong consistency; and achieving alignment of multiple coordinate systems, facilitating direct comparison with design points, equipment locations, and construction coordinates.

[0040] S1015. Integrate the image depth data, local 3D point cloud data, pose data and coordinate system transformation parameters corresponding to the current image frame to form the spatial registration data of the image frame and complete the cache storage.

[0041] The image depth data of the current frame, the local 3D point cloud data of the scene, the device pose data, and the coordinate system transformation parameters (intrinsic parameters, extrinsic parameters, pose, and world engineering registration parameters) are packaged into one, bound to the image frame and timestamp, and then written to the cache. When the user clicks on a subsequent image frame, the corresponding spatial registration data can be obtained directly based on the timestamp of the image frame clicked by the user.

[0042] In step S102, the selection operation is the user's selection of a target object on the image interface. In this invention, the user can select the target object online in the device's real-time viewfinder or remotely select it offline by retrieving stored historical images or archived video frames. Target objects include: corners, ground points, equipment edges, building components, design outlines, pit edges, construction marks, or other targets to be measured. Target pixels are the image pixels corresponding to the user's selected location. Pixel coordinates are the two-dimensional coordinates of the target pixel in the image. .

[0043] In practice, users select target objects (such as corners, markers, component edges, or pit locations) on real-time images, videos, scan previews, or playback images. The selected location is captured, the corresponding target pixels are extracted, and their two-dimensional pixel coordinates in the image are determined. At the same time, the image frame and timestamp corresponding to the selected point are recorded to match the corresponding spatial registration data.

[0044] The benefits of this step are: enabling visualization, contactless, and remote point selection; supporting review of selected points without requiring real-time on-site operation; and improving operational safety in hazardous areas, high-altitude locations, and deep pits.

[0045] In step S103, a preset depth determination strategy is established: direct depth is used preferentially, and depth is estimated from the point cloud when it is missing. Target depth: the 3D spatial distance value z corresponding to the target pixel.

[0046] In some embodiments, according to a preset depth determination strategy, the target depth corresponding to the target pixel is determined based on image depth data in spatial registration data or local 3D point cloud data of the scene, including: If the target pixel has a valid depth value, it is directly used as the target depth. If the target pixel has no valid depth value or the depth accuracy does not meet the requirements, the local 3D point cloud of the scene is projected onto the image plane to establish the mapping relationship between the point cloud and the pixel. The valid depth point is searched in the neighborhood of the target pixel, and the target depth of the target pixel is estimated by interpolation or fitting.

[0047] Specifically, the spatial registration data of the selected image frame is read. For example, based on the timestamp of the image frame selected by the user, the corresponding spatial registration data is obtained. Then, it is checked whether the image depth data in the spatial registration data includes the depth value of the pixel. If the pixel has a valid depth, it is directly used as the target depth. If the pixel is in a region with no valid depth, such as weak texture, reflection, occlusion, distance, or boundary holes, the local 3D point cloud of the scene is projected onto the image, and a valid depth point is searched in the neighborhood of the target pixel. The target depth is estimated through interpolation, local plane fitting, or ray intersection to ensure that a stable depth is obtained at the selected location.

[0048] The benefits of this step are that reliable depth can be obtained regardless of whether the point is selected in the effective area or the deep void area; and it solves the problem of not being able to select points in complex scenes.

[0049] In step S104, the target 3D point refers to the 3D spatial coordinates of the target pixel in the engineering coordinate system. The camera intrinsic parameter K is the intrinsic parameter matrix composed of focal length and principal point.

[0050] In some embodiments, the target three-dimensional point of the target pixel in the engineering coordinate system is determined based on the pixel coordinates, target depth, pose data, and coordinate system transformation parameters, including S1041-S1043: S1041. Based on the pixel coordinates of the target pixel, the target depth, and the camera intrinsic parameters in the coordinate system transformation parameters, back-project to obtain the three-dimensional point in the camera coordinate system.

[0051] The pixel coordinates of the target pixel are The target depth is z, and the camera intrinsic parameter is K. Therefore, the three-dimensional point in the camera coordinate system is... : ; This step restores two-dimensional image points to three-dimensional spatial points from the camera's perspective, completing the first geometric reconstruction from a plane to space.

[0052] S1042. Based on the pose data and camera extrinsic parameters in the coordinate system transformation parameters, transform the 3D points in the camera coordinate system to the 3D points in the world coordinate system.

[0053] Specifically, three-dimensional points in the world coordinate system : ;in, For camera external parameters, This provides equipment pose data. It transforms local 3D points from the camera into a unified world coordinate system within the equipment's operating space, ensuring that image points captured from different angles and positions can all be integrated into the same spatial system.

[0054] S1043. If RTK absolute coordinate constraints are used, the world engineering registration parameters in the coordinate system transformation parameters are corrected using RTK absolute coordinate constraints, and the three-dimensional points in the world coordinate system are transformed to the engineering coordinate system through the corrected world engineering registration parameters to obtain the target three-dimensional points.

[0055] Among them, RTK absolute coordinate constraints: using geographic / engineering absolute coordinates provided by the RTK module, to eliminate world coordinate system drift. World engineering registration parameters. : Transformation matrix from world coordinate system to engineering coordinate system.

[0056] Correction: Calibration was performed using RTK absolute coordinate pairs to ensure the results are strictly aligned with the construction coordinate system. Engineering coordinate system: The absolute coordinate system used for construction design, CAD / BIM, and layout. Target 3D point : Ultimately, these are the engineering coordinate points that can be used for measurement and layout.

[0057] Specifically, first determine if the system has RTK absolute coordinate constraints enabled; if not, directly use the pre-calibrated ones. Perform the conversion; if already connected, use the absolute coordinates provided by RTK to match the world engineering registration parameters. Make corrections to eliminate accumulated errors. Use the corrected version. , to convert three-dimensional points in the world coordinate system Transform to the engineering coordinate system to obtain the final target 3D point. : Output As a target three-dimensional point in the engineering coordinate system, it can be directly used for layout, comparison, and recording.

[0058] The relative world coordinates are converted into absolute engineering coordinates that can be used in construction, and absolute accuracy is guaranteed by RTK, so that the image point acquisition results can be directly used for engineering operations.

[0059] In step S105, the non-contact point acquisition result is the three-dimensional coordinates of the target point that can be obtained without going to the site.

[0060] In practice, the coordinates of the target 3D point in the engineering coordinate system are output as the point acquisition result, which can be displayed, recorded, stored, or exported for measurement, verification, quantity calculation, construction acceptance, etc. This achieves truly non-contact, long-distance, and safe point acquisition; no personnel need to approach dangerous areas; suitable for scenarios where traditional equipment cannot operate, such as deep pits, high altitudes, electrified areas, and confined spaces.

[0061] In step S106, the construction design information includes at least the coordinates of the design points in CAD / BIM / LandXML. Deviation data: the coordinate difference between the target point and the design point. Layout guidance information: prompts indicating the direction, distance, and deviation of movement.

[0062] In some embodiments, the coordinates of the target three-dimensional point are compared with the coordinates of the design point in the construction design information to generate deviation data, and layout guidance information is output based on the deviation data, including S1061-S1063: S1061. Select a comparison reference point. The reference point is selected from any one or more of the target layout point, current equipment position, current work point, and design point. Convert the target three-dimensional point and the selected reference point to the same engineering coordinate system.

[0063] Retrieve coordinates of four types of benchmark points as needed: Design points: These are the reference coordinates of the engineering coordinate system preset in the drawings, taken from engineering design documents such as CAD, BIM, LandXML, point tables, and construction control points. Target layout points: Coordinates of the points to be laid out on site, obtained by using the design points as a reference and making minor adjustments based on the on-site construction environment; Current device location: The real-time coordinates of the device in the engineering coordinate system are calculated in real time based on the RTK, IMU, and fusion positioning modules on the device. Current work point: The coordinates of the location of the workers and construction equipment are obtained through on-site measurement and data entry or on-site point selection and collection by the equipment; Users can select one or more of the above-mentioned points as comparison benchmarks, either individually or in combination, according to their on-site layout requirements.

[0064] The calculated target 3D point is now within the engineering coordinate system. For the selected reference point, if the reference point coordinates are not in the engineering coordinate system format, they are uniformly converted to the same engineering coordinate system using the previously stored world engineering registration parameters. Only after all points have completed coordinate normalization can the subsequent deviation calculation steps be performed to avoid calculation errors caused by different coordinate systems.

[0065] S1062. Solve for deviation data based on the difference in point coordinates in the same coordinate system. The deviation data includes at least one of horizontal deviation, elevation deviation, azimuth deviation, and remaining distance between two points.

[0066] Among them, horizontal deviation: the difference in distance between the target three-dimensional point and the reference point in the plane direction (east-west, north-south), used to indicate left-right and forward-backward movement; Elevation deviation: The difference in height (up and down) between the target 3D point and the reference point in the vertical direction, used to indicate whether it is raised or lowered; Orientation deviation: The angular deviation between the current orientation of the equipment and the direction of the reference point, used to indicate turning direction; Remaining distance: The straight-line distance from the target 3D point or the current position of the device to the design point, used to determine whether it is close to the target.

[0067] It can output a single deviation or a combination of multiple deviations simultaneously to comprehensively reflect the spatial difference between the current position and the target position.

[0068] S1063. Generate azimuth arrows, distance values, or movement prompts based on deviation data to guide personnel or equipment to adjust and position themselves at the target location.

[0069] In some embodiments, intuitive and executable stakeout guidance information is generated based on the calculated deviation data. Specific implementation methods include: Based on the horizontal deviation, left / right / forward / backward movement prompts are generated. When the operator manually lays out the equipment, the system outputs movement instructions facing the operator, such as "move left 0.2m" or "move forward 0.5m". When docking with automated machinery for automatic laying out, the system generates suggestions for equipment motion control. Based on the elevation deviation, adjustment prompts are generated, such as "downward 0.1m" or "upward 0.05m". Based on the orientation deviation, a turning prompt is generated. In manual scenarios, the personnel turn the command, while in automated scenarios, the target pose parameters of the equipment are output, such as "turn left 15°" or "head toward the target point". Distance prompts are generated based on the remaining distance, such as "0.3m from target" or "Target range reached".

[0070] In some embodiments, based on the completion of deviation calculation and guidance information generation, visual guidance and feedback can also be provided, including the following steps: Overlay at least one of the following information onto the real-time image interface of an AR device, mobile terminal, driver's cab display terminal, or scanning terminal: user-selected target point, determined 3D target point, design point, current device position, target direction arrow, deviation value, target tolerance circle, and trajectory guide line. The layout progress is fed back with different colors according to the magnitude of the deviation. Specifically, green is displayed when the deviation is less than the set threshold, yellow is displayed when it is close to the threshold, and red is displayed when it exceeds the threshold or the direction is wrong. By combining equipment pose and work scenario information, a guide trajectory between the current position of the equipment and the target point is generated, enabling more intuitive, efficient and accurate remote layout guidance in a visual manner.

[0071] Specifically, the system overlays one or more of the following information onto the real-time image interface of AR devices, mobile terminals, cab display terminals, or scanning terminals: the target point selected by the user in the image, the calculated 3D target point, the design point in the construction design information, the current position of the equipment, the directional arrow pointing to the target point, the deviation value between the target 3D point and the design point, the target tolerance circle used to indicate the qualified range of the layout, and the trajectory guide line to guide the movement path, so that the user can intuitively observe the target position, deviation status, and travel route.

[0072] The layout progress is indicated by color based on the magnitude of the deviation. Specifically, different colors are used to mark the layout status based on the comparison between the deviation data and the preset threshold: green is displayed when the deviation is less than the set threshold and the layout requirements are met; yellow is displayed when the deviation is close to the set threshold (i.e., the difference between the deviation and the threshold is less than a certain set value) and fine-tuning is required; and red is displayed when the deviation exceeds the threshold or the direction is incorrect. This provides a clear and intuitive way to indicate the current layout progress and the correctness of the operation to the user.

[0073] The process of generating a guide trajectory from the device to the target point involves combining the device's current pose data, obstacle information in the work scenario, and reachable area information to plan and generate a safe and reasonable guide trajectory from the device's current position to the target point. This trajectory is then overlaid on the image interface as a guide line to guide the user or device to move along the optimal path to the target point, achieving accurate, efficient, and safe remote stakeout.

[0074] The solution of this invention achieves non-contact point acquisition and remote visual layout through the fusion of binocular and dToF depth, point cloud depth completion, multi-level coordinate transformation and RTK absolute coordinate constraints. It effectively solves the problems of traditional measurement and layout requiring contact operations, low efficiency, and poor applicability to dangerous scenarios. At the same time, it overcomes the defects of single sensor lack of depth and insufficient accuracy, and has the advantages of strong robustness, wide applicability, simple operation, safety and efficiency, and high layout accuracy.

[0075] Example 2 Figure 2 This document provides a flowchart of a method for remote image lofting and non-contact point acquisition, as described in an embodiment of the present invention. See also... Figure 2 The method includes the following steps: S201. In response to the user's selection operation, the same target object is tracked in multiple consecutive frames of images before and after the selection through feature matching, optical flow method or target tracking algorithm, and the confidence of the corresponding three-dimensional point and the depth data of each frame is calculated respectively.

[0076] Among them, continuous multi-frame images: multiple time-series images before and after the user's selection, used to improve the stability of point selection. Tracking the same target object: locking the target pixel or feature point selected by the user across multiple frames to maintain target consistency. 3D points: 3D point positions in the engineering coordinate system calculated separately for each frame image. Depth data confidence: a numerical value characterizing the reliability of the depth in that frame, related to texture, distance, and noise.

[0077] In practice, after the user selects a target, the same target point is locked and tracked in multiple consecutive frames of images using feature matching, optical flow, or target tracking algorithms, based on the selected location.

[0078] For each image frame, the calculation can be performed according to steps S103-S104 of the above embodiment to obtain the three-dimensional points corresponding to each image frame. Calculate the confidence level of depth for each frame synchronously. High confidence levels are achieved when depth is effective, texture is rich, and the image is taken at close range; low confidence levels are achieved when depth is lacking, texture is weak, the image is reflective, and the image is taken at a long distance.

[0079] This allows for the output of multiple sets of 3D points. ) and the corresponding depth confidence ( ).

[0080] S202. Based on the confidence level of depth data in each frame, the 3D points calculated based on each image frame are optimized by weighted fusion, robust estimation or confidence filtering to obtain the optimized target 3D points.

[0081] The process includes: weighted fusion: averaging the 3D points from multiple frames based on their confidence level; robust estimation: calculating the mean after removing outliers and noise points; confidence screening: retaining only the results from high-confidence frames for optimization; and optimized target 3D points: the final point selection results after multi-frame stabilization.

[0082] In practice, multiple sets of 3D points are read. ) and the corresponding depth confidence ( ).

[0083] Optimize using one of the following methods: The target 3D points can be obtained by weighted fusion according to the following formula. : Frames with higher confidence levels have greater weights and contribute more strongly to the results.

[0084] Robust estimation: First, remove outliers with excessive bias, then take the mean.

[0085] Confidence filtering: Only frames with a confidence level higher than the threshold are retained, and then averaged.

[0086] After determining the optimal target 3D point through steps S201-S202, point selection and / or stakeout can be performed according to steps S203 and / or S204.

[0087] S203, Output the coordinates of the target 3D point as a non-contact point acquisition result; and / or S204. Compare the coordinates of the target 3D point with the coordinates of the target layout point, the current equipment position, the current work point, or the design point in the construction design information to generate deviation data, and output layout guidance information based on the deviation data.

[0088] The solution of this invention, through multi-frame tracking and confidence-weighted optimization, effectively reduces the impact of single-frame depth noise, point selection error, equipment jitter and sensor fluctuation, significantly improves the stability and accuracy of point acquisition, and makes non-contact point acquisition reliable even in complex construction scenarios.

[0089] Example 3 Figure 3 This is a schematic diagram of an image remote lofting and non-contact point acquisition device provided in an embodiment of the present invention. This device is equipped to execute any of the image remote lofting and non-contact point acquisition methods of the present invention. For example... Figure 3 As shown, the image remote stakeout and non-contact point acquisition device includes: The data construction module 301 is used to construct and cache the corresponding spatial registration data for each image frame; wherein, the spatial registration data is used to establish the mapping relationship between image points and three-dimensional spatial points, and includes at least: image depth data, scene local three-dimensional point cloud data, device pose data when acquiring image frames, and coordinate system transformation parameters; wherein, the image frame includes at least one of real-time image frames, video frames, scan preview frames, and playback image frames; The response processing module 302 is used to respond to the user's point selection operation on the target object in the image frame and determine the target pixel selected by the user and the pixel coordinates of the target pixel; The data acquisition and processing module 303 is used to acquire spatial registration data associated with image frames, and determine the target depth corresponding to the target pixel based on the image depth data or local three-dimensional point cloud data of the scene in the spatial registration data. The calculation module 304 is used to determine the target three-dimensional point of the target pixel in the engineering coordinate system based on the pixel coordinates, target depth, pose data and coordinate system transformation parameters of the target pixel. Point acquisition output module 305 is used to output the coordinates of the target 3D point as a non-contact point acquisition result; and / or The layout guidance module 306 is used to compare the coordinates of the target three-dimensional point with the coordinates of the target layout point, the current equipment position, the current work point, or the design point in the construction design information, generate deviation data, and output layout guidance information based on the deviation data.

[0090] In some embodiments, the data construction module 301 is specifically used for: For a single image frame, the image depth data corresponding to that image frame is obtained; wherein, the image depth data is obtained by fusing two types of original depth information, namely, the binocular depth and the dToF depth, corresponding to the imaging of that image frame; Based on the image depth data of the image frame, combined with the camera intrinsic parameters in the coordinate system transformation parameters, the pixels with effective depth values ​​in the frame are converted into three-dimensional points in the camera coordinate system, and the three-dimensional points are summarized to generate the local three-dimensional point cloud data of the scene corresponding to the image frame. Based on the generated local 3D point cloud data of the scene, depth padding is performed on pixels that have no effective depth value within the image frame to construct the overall correspondence between image points within the frame and 3D spatial points. By combining the device's pose and spatial position information at the moment the image frame was captured, the corresponding pose data is determined, and the conversion relationship between the camera coordinate system, device coordinate system, world coordinate system, and engineering coordinate system is determined to obtain the coordinate system transformation parameters. The image depth data, local 3D point cloud data, pose data, and coordinate system transformation parameters corresponding to the current image frame are integrated to form the spatial registration data of the image frame and cached.

[0091] In some embodiments, the data acquisition and processing module 303 is specifically used for: Retrieve image depth data; if the target pixel has a valid depth value, directly use the valid depth value as the target depth. If the target pixel has no effective depth value or the depth accuracy does not meet the requirements, the local 3D point cloud of the scene is projected onto the image plane to establish the mapping relationship between the point cloud and the pixel. Effective depth points are searched in the neighborhood of the target pixel, and the target depth of the target pixel is estimated by interpolation or fitting.

[0092] In some embodiments, the solving module 304 is specifically used for: Based on the pixel coordinates of the target pixel, the target depth, and the camera intrinsic parameters in the coordinate system transformation parameters, the three-dimensional point in the camera coordinate system is obtained by back projection. Based on pose data and camera extrinsic parameters in coordinate system transformation parameters, transform 3D points in the camera coordinate system to 3D points in the world coordinate system; If RTK absolute coordinate constraints are used, the world engineering registration parameters in the coordinate system transformation parameters are corrected using RTK absolute coordinate constraints, and the 3D points in the world coordinate system are transformed to the engineering coordinate system using the corrected world engineering registration parameters to obtain the target 3D points.

[0093] In some embodiments, an optimization module is also included, comprising: The tracking unit is used to respond to the user's selection operation and track the same target object in multiple consecutive frames of images before and after the selection by feature matching, optical flow method or target tracking algorithm, and respectively calculate the confidence of the corresponding three-dimensional point and the depth data of each frame; The optimization unit is used to optimize the 3D points calculated based on each image frame by weighted fusion, robust estimation or confidence filtering based on the confidence of the depth data of each frame, so as to obtain the optimized target 3D points.

[0094] In some embodiments, the optimization unit is further configured to: The target 3D point is determined according to the following formula. : ;in, Represents the i-th 3D point; This represents the depth confidence of the image frame to which the i-th 3D point belongs.

[0095] In some embodiments, the layout guidance module 306 is specifically used for: Select a comparison reference point, which is selected from any one or more of the target layout point, current equipment position, current work point, and design point; convert the target three-dimensional point and the selected reference point to the same engineering coordinate system; The deviation data is calculated based on the difference in point coordinates in the same coordinate system. The deviation data includes at least one of the following: horizontal deviation, elevation deviation, azimuth deviation, and remaining distance between two points. Based on deviation data, directional arrows, distance values, or movement prompts are generated to guide personnel or equipment to adjust and position themselves at the target location.

[0096] In some embodiments, an AR display module is also included, for: Overlay at least one of the following information onto the real-time image interface of an AR device, mobile terminal, driver's cab display terminal, or scanning terminal: user-selected target point, determined 3D target point, design point, current device position, target direction arrow, deviation value, target tolerance circle, and trajectory guide line. The layout progress is fed back with different colors according to the magnitude of the deviation. Specifically, green is displayed when the deviation is less than the set threshold, yellow is displayed when it is close to the threshold, and red is displayed when it exceeds the threshold or the direction is wrong. By combining equipment pose and operational scenario information, a guidance trajectory between the current position of the equipment and the target point is generated.

[0097] The image remote lofting and non-contact point acquisition device provided in the embodiments of the present invention can execute the image remote lofting and non-contact point acquisition method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0098] According to embodiments of the present invention, the present invention also provides an electronic device, a readable storage medium, and a computer program product.

[0099] Example 4 Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the invention described and / or claimed herein.

[0100] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory 12 or a random access memory 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the read-only memory 12 or loaded from storage unit 18 into the random access memory 13. The random access memory 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, read-only memory 12, and random access memory 13 are interconnected via a bus 14. An input / output interface 15 is also connected to the bus 14.

[0101] Multiple components in electronic device 10 are connected to input / output interface 15, including: input unit 16; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disks, optical disks, etc.; and communication unit 19, such as network interface cards, modems, wireless transceivers, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0102] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, central processing units, graphics processing units, various special-purpose artificial intelligence computing chips, various processors running machine learning model algorithms, digital signal processors, and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as performing remote image lofting and non-contact point acquisition methods.

[0103] In some embodiments, the image remote lofting and non-contact point acquisition method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via read-only memory 12 and / or communication unit 19. When the computer program is loaded into random access memory 13 and executed by processor 11, one or more steps of the image remote lofting and non-contact point acquisition method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the image remote lofting and non-contact point acquisition method by any other suitable means (e.g., by means of firmware).

[0104] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays, application-specific integrated circuits (ASICs), application-specific standard products (ASICs), systems-on-a-chip (SoCs), complex programmable logic devices, computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0105] Computer programs used to implement the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable image remote lofting and non-contact point acquisition device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0106] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, optical fibers, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0107] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device or liquid crystal display for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0108] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet. The computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having client-server relationships with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, a host product within the cloud computing service system, addressing the shortcomings of traditional physical hosts and virtual private servers, such as high management difficulty and weak business scalability.

[0109] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0110] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for remote image lofting and non-contact point acquisition, characterized in that, include: Spatial registration data is constructed and cached for each image frame; wherein the spatial registration data is used to establish the mapping relationship between image points and three-dimensional spatial points, and includes at least: image depth data, local three-dimensional point cloud data of the scene, pose data of the device when the image frame is acquired, and coordinate system transformation parameters; wherein the image frame includes at least one of real-time image frame, video frame, scan preview frame, and playback image frame. In response to a user's selection of a target object in the image frame, the user-selected target pixel and the pixel coordinates of the target pixel are determined. Obtain the spatial registration data associated with the image frame, and determine the target depth corresponding to the target pixel based on the image depth data or the local three-dimensional point cloud data of the scene in the spatial registration data; Based on the pixel coordinates of the target pixel, the target depth, the pose data, and the coordinate system transformation parameters, determine the target three-dimensional point of the target pixel in the engineering coordinate system; Output the coordinates of the target 3D point as a non-contact point acquisition result; and / or The coordinates of the target 3D point are compared with the coordinates of the target layout point, the current equipment position, the current work point, or the design point in the construction design information to generate deviation data, and layout guidance information is output based on the deviation data.

2. The method according to claim 1, characterized in that, The step of constructing corresponding spatial registration data for each image frame includes: For a single image frame, the image depth data corresponding to that image frame is obtained; wherein, the image depth data is obtained by fusing two types of original depth information, namely, the binocular depth and the dToF depth, corresponding to the imaging of that image frame; Based on the image depth data of the image frame, and combined with the camera intrinsic parameters in the coordinate system transformation parameters, the pixels with effective depth values ​​in the frame are converted into three-dimensional points in the camera coordinate system, and the three-dimensional points are summarized to generate the scene local three-dimensional point cloud data corresponding to the image frame. Based on the generated local 3D point cloud data of the scene, depth padding is performed on pixels that have no effective depth value within the image frame to construct the overall correspondence between image points within the frame and 3D spatial points. By combining the device's pose and spatial position information at the moment the image frame was captured, the corresponding pose data is determined, and the conversion relationship between the camera coordinate system, device coordinate system, world coordinate system, and engineering coordinate system is determined to obtain the coordinate system transformation parameters. The image depth data, local 3D point cloud data, pose data, and coordinate system transformation parameters corresponding to the current image frame are integrated to form the spatial registration data of the image frame and cached.

3. The method according to claim 1, characterized in that, Based on the image depth data or the local 3D point cloud data of the scene in the spatial registration data, the target depth corresponding to the target pixel is determined, including: Retrieve the image depth data; if the target pixel has a valid depth value, directly use the valid depth value as the target depth. If the target pixel has no effective depth value or the depth accuracy does not meet the requirements, the local 3D point cloud of the scene is projected onto the image plane to establish a mapping relationship between the point cloud and the pixel. Effective depth points are searched in the neighborhood of the target pixel, and the target depth of the target pixel is estimated by interpolation or fitting.

4. The method according to claim 1, characterized in that, The step of determining the target 3D point of the target pixel in the engineering coordinate system based on the pixel coordinates of the target pixel, the target depth, the pose data, and the coordinate system transformation parameters includes: Based on the pixel coordinates of the target pixel, the target depth, and the camera intrinsic parameters in the coordinate system transformation parameters, a three-dimensional point in the camera coordinate system is obtained by back projection. Based on the pose data and the camera extrinsic parameters in the coordinate system transformation parameters, the 3D points in the camera coordinate system are transformed to 3D points in the world coordinate system. If RTK absolute coordinate constraints are used, the world engineering registration parameters in the coordinate system transformation parameters are corrected using the RTK absolute coordinate constraints, and the three-dimensional points in the world coordinate system are transformed to the engineering coordinate system using the corrected world engineering registration parameters to obtain the target three-dimensional points.

5. The method according to claim 1, characterized in that, Also includes: In response to the user's selection operation, the same target object is tracked in multiple consecutive frames of images before and after the selection by feature matching, optical flow method or target tracking algorithm, and the confidence of the corresponding three-dimensional point and the depth data of each frame is calculated respectively. Based on the confidence level of depth data in each frame, the 3D points calculated from each image frame are optimized by weighted fusion, robust estimation, or confidence filtering to obtain the optimized target 3D points.

6. The method according to claim 5, characterized in that, Based on the confidence level of depth data from each frame, the 3D points calculated from each image frame are optimized using a weighted fusion method to obtain the optimized target 3D points, including: The target 3D point is determined according to the following formula. : ;in, Represents the i-th 3D point; This represents the depth confidence of the image frame to which the i-th 3D point belongs.

7. The method according to claim 1, characterized in that, The step involves comparing the coordinates of the target 3D point with the coordinates of the target layout point, the current equipment position, the current work point, or the design point in the construction design information to generate deviation data, and outputting layout guidance information based on the deviation data, including: Select a comparison reference point, which is selected from any one or more of the target layout point, current equipment position, current work point, and design point; convert the target three-dimensional point and the selected reference point to the same engineering coordinate system; The deviation data is calculated based on the difference in point coordinates in the same coordinate system. The deviation data includes at least one of the following: horizontal deviation, elevation deviation, azimuth deviation, and remaining distance between two points. Based on deviation data, directional arrows, distance values, or movement prompts are generated to guide personnel or equipment to adjust and position themselves at the target location.

8. The method according to claim 1, characterized in that, Also includes: Overlay at least one of the following information onto the real-time image interface of an AR device, mobile terminal, driver's cab display terminal, or scanning terminal: user-selected target point, determined 3D target point, design point, current device position, target direction arrow, deviation value, target tolerance circle, and trajectory guide line. The layout progress is fed back with different colors according to the magnitude of the deviation. Specifically, green is displayed when the deviation is less than the set threshold, yellow is displayed when it is close to the threshold, and red is displayed when it exceeds the threshold or the direction is wrong. By combining equipment pose and operational scenario information, a guidance trajectory between the current position of the equipment and the target point is generated.

9. A device for remote image lofting and non-contact point acquisition, characterized in that, include: The data construction module is used to construct and cache corresponding spatial registration data for each image frame; wherein, the spatial registration data is used to establish the mapping relationship between image points and three-dimensional spatial points, and includes at least: image depth data, scene local three-dimensional point cloud data, device pose data when acquiring the image frame, and coordinate system transformation parameters; wherein, the image frame includes at least one of real-time image frames, video frames, scan preview frames, and playback image frames; The response processing module is used to respond to the user's point selection operation on the target object in the image frame, and to determine the target pixel selected by the user and the pixel coordinates of the target pixel; The data acquisition and processing module is used to acquire the spatial registration data associated with the image frame, and determine the target depth corresponding to the target pixel based on the image depth data or the scene local three-dimensional point cloud data in the spatial registration data. The calculation module is used to determine the target three-dimensional point of the target pixel in the engineering coordinate system based on the pixel coordinates of the target pixel, the target depth, the pose data and the coordinate system transformation parameters; The point acquisition output module is used to output the coordinates of the target 3D point as a non-contact point acquisition result; and / or The layout guidance module is used to compare the coordinates of the target three-dimensional point with the coordinates of the target layout point, the current equipment position, the current work point, or the design point in the construction design information, generate deviation data, and output layout guidance information based on the deviation data.

10. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the method of any one of claims 1-8.

Images