Water resource bulletin achievement display method based on augmented reality

Abstract

The invention relates to the technical field of virtual reality, in particular to a water resource bulletin achievement display method based on augmented reality. The method comprises the following steps: acquiring environmental spatial perception information through a camera, an inertial measurement unit and a positioning unit of an augmented reality terminal, and extracting and correcting scene feature points to form a spatial feature data set; a virtual reference coordinate system is generated in combination with GPS positioning, optical anchoring is performed on a real scene target area, and a first display scene is generated; and when it is detected that a real object carrier exists in the scene, dynamically adjusting layer display of the scene by using the attitude and position parameters, realizing fusion display of the virtual information and the real object carrier, and generating a second display scene under augmented reality. According to the method and the system, high-precision alignment and natural fusion of a virtual water resource result in a real environment are realized through fusion of space perception positioning, optical anchoring and a real object carrier depth identification technology, and the stability, the reality sense and the interactivity of augmented reality display are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of virtual reality technology, and in particular to a method for displaying water resources bulletin results based on augmented reality. Background Technology

[0002] Existing methods for presenting water resources reports primarily employ two-dimensional charts, geographic information system visualizations, or video animations. While these methods can present some hydrological spatial distribution information, they generally lack a sense of integration with the real environment and fail to achieve a spatial correspondence between virtual data and real-world scenes. In recent years, augmented reality (AR) technology has been increasingly applied to the field of natural resource visualization, used to overlay virtual water resources information onto real-world images.

[0003] However, existing AR display solutions generally suffer from minor defects such as inaccurate optical anchoring, large virtual-real alignment errors, virtual element misalignment caused by viewpoint drift, and insufficient recognition of physical carriers, making it difficult to maintain stable display effects in complex geographical environments or outdoor scenes. Summary of the Invention

[0004] Therefore, it is necessary to provide a method for displaying water resources bulletin results based on augmented reality to solve at least one of the aforementioned technical problems.

[0005] To achieve the above objectives, a method for displaying water resources bulletin results based on augmented reality is proposed, the method comprising the following steps: Step S1: Acquire spatial perception information of the user's observed environment through the sensor module of the augmented reality display terminal, wherein the sensor module includes a camera, an inertial measurement unit and a positioning unit; Step S2: Extract scene feature points based on the environmental images acquired by the camera, and perform time synchronization correction on the scene feature points in conjunction with the attitude change data of the inertial measurement unit to obtain a matching spatial feature dataset; Step S3: Generate a virtual reference coordinate system in the terminal display interface based on the spatial feature dataset and GPS positioning information, and perform optical anchoring on the target area in the real scene based on the virtual reference coordinate system to obtain the first display scene; Step S4: In response to the detection of a physical carrier in the first display scene, the layer display of the physical carrier is adjusted by the acquired posture and position parameters to obtain the second display scene.

[0006] The present invention has the following beneficial effects: I. By integrating multi-layered processing mechanisms such as spatial perception, optical anchoring, and virtual-real fusion, the visualization accuracy and immersive experience of water resource data in real-world environments are significantly improved. Firstly, this method utilizes the collaborative work of cameras, inertial measurement units, and positioning units to collect real-time spatial perception information of the user's observed environment and construct a high-precision virtual reference coordinate system, achieving accurate matching between virtual information and the real environment. Through feature point extraction and time-synchronized correction, visual offsets caused by changes in terminal posture can be eliminated, ensuring stable display of virtual elements in complex geographical scenes and greatly improving spatial alignment accuracy and display stability in augmented reality scenarios.

[0007] Second, through spatial distribution analysis and projection mapping of 3D boundary points, each boundary point in the virtual scene can be accurately projected onto the camera's field of view coordinate plane. Furthermore, based on the brightness gradient and pixel matching relationship, segment-by-segment translation correction and pixel fusion calibration are performed, thereby achieving a seamless and consistent display of virtual elements and real-world images. This process not only eliminates problems such as misalignment and blurred boundaries in traditional AR displays but also maintains a highly consistent visual experience regardless of user perspective movement or lighting changes, providing reliable support for the three-dimensional visualization of water resource spatial results.

[0008] Third, by introducing a physical carrier detection and spatial depth estimation mechanism, it is possible to automatically identify physical carrier areas with geometric contour features and stable shapes in augmented reality environments, and to perform spatial conversion of depth trends based on the camera's field of view and focal length parameters, thereby achieving accurate overlay and display of virtual data on the surface of the physical carrier. This mechanism allows virtual information to adaptively attach to real carriers, such as sand tables, models, or geographic landmarks, achieving a natural fusion effect of "real carrier + virtual augmentation". Attached Figure Description

[0009] Figure 1 This is a flowchart illustrating the steps of a method for displaying water resources bulletin results based on augmented reality. Figure 2 for Figure 1 A detailed flowchart illustrating the implementation steps of step S3. Figure 3 This is a flowchart illustrating the workflow of a method for displaying water resources bulletin results based on augmented reality, as proposed in this application. The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0010] The technical method of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0011] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor methods and / or microcontroller methods.

[0012] It should be understood that although the terms "first," "second," etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are used merely to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0013] To achieve the above objectives, please refer to Figures 1 to 3 A method for displaying water resources bulletin results based on augmented reality, the method comprising the following steps: Step S1: Acquire spatial perception information of the user's observed environment through the sensor module of the augmented reality display terminal, wherein the sensor module includes a camera, an inertial measurement unit and a positioning unit; In one embodiment, the augmented reality display terminal includes a camera module, an inertial measurement unit (IMU), and a positioning unit. The camera module is used to acquire environmental image data in the user's viewing direction in real time, and the sampling frequency can be set to 30-60 frames / second to ensure the continuity of visual information and the complete capture of the spatial scene.

[0014] An inertial measurement unit (IMU) is used to acquire information about the terminal's attitude changes in space, including parameters such as acceleration, angular velocity, and pitch angle. An IMU can consist of a three-axis accelerometer and a three-axis gyroscope, and uses data fusion algorithms (such as complementary filtering or Kalman filtering) to calculate the terminal's spatial attitude and orientation vector in real time.

[0015] The positioning unit is used to obtain the terminal's location information in a geographic coordinate system. It can use the Global Positioning System (GPS) or an indoor positioning module (such as UWB or Wi-Fi positioning) for real-time positioning. The positioning accuracy is preferably less than 0.5 meters to ensure accurate alignment between virtual content and real geographical location when augmented reality is displayed.

[0016] During data acquisition, the control unit synchronizes the image frames captured by the camera module with the attitude data output by the inertial measurement unit in time, and combines this with the position information output by the positioning unit to form spatial perception information of the user's observed environment. This spatial perception information includes three parts: environmental image data, attitude angle data, and geographical location information.

[0017] In a preferred embodiment, the control unit can further preprocess the aforementioned spatial perception information, including distortion correction, image enhancement, and coordinate alignment operations, thereby forming a spatial perception dataset consistent with the augmented reality display coordinate system, providing accurate environmental reference for subsequent virtual-real fusion rendering.

[0018] Step S2: Extract scene feature points based on the environmental images acquired by the camera, and perform time synchronization correction on the scene feature points in conjunction with the attitude change data of the inertial measurement unit to obtain a matching spatial feature dataset; In one embodiment, after acquiring an environmental image, the augmented reality display terminal first performs preprocessing on the image by an image processing module, including image denoising, contrast enhancement, and distortion correction, in order to improve the accuracy of subsequent feature extraction.

[0019] Subsequently, the control unit performs scene feature point extraction based on the preprocessed environmental image. Specifically, feature detection algorithms (such as ORB, SIFT, or SURF) are used to identify feature regions with significant texture changes or brightness gradient changes in the image, and their feature descriptors are calculated. Approximately 2000 to 5000 feature points can be extracted from each frame of the image. The feature point information includes image coordinates, grayscale gradient direction, and local descriptor vector.

[0020] To improve the spatial consistency of feature points, attitude change data acquired by the inertial measurement unit (IMU) is used to perform time synchronization correction on the feature points. The IMU outputs the terminal's acceleration data, angular velocity data, and attitude angle parameters (pitch, yaw, and roll) in each time period. The control unit aligns this attitude data with the image acquisition timestamps and obtains the precise attitude state of the terminal at the moment of each image acquisition through time interpolation and attitude fusion calculations.

[0021] Based on this posture state, the control unit performs coordinate transformation on the feature points extracted at different time points, mapping them from the two-dimensional coordinate system of the image to a unified spatial reference coordinate system, thereby achieving temporal synchronization and spatial alignment of feature points across frames. This method eliminates feature point offsets caused by terminal motion, thus obtaining a stable and consistent spatial feature representation.

[0022] In a preferred embodiment, the control unit further performs spatial matching on the time-synchronized feature point set, filters out duplicate feature points by calculating feature descriptor similarity (e.g., cosine similarity or Hamming distance), and performs deduplication and averaging fusion to finally form a matched spatial feature dataset. The spatial feature dataset includes the three-dimensional spatial coordinates of the feature points, feature direction vectors, and their time label information, which can serve as the basic data for subsequent 3D scene reconstruction and augmented reality registration.

[0023] Step S3: Generate a virtual reference coordinate system in the terminal display interface based on the spatial feature dataset and GPS positioning information, and perform optical anchoring on the target area in the real scene based on the virtual reference coordinate system to obtain the first display scene; In one embodiment, after acquiring the spatial feature dataset, the augmented reality display terminal first uses the control unit to call the terminal's positioning unit to obtain real-time GPS positioning information, which includes longitude, latitude, and altitude parameters. The positioning unit performs continuous positioning sampling within a time interval of 0.5 to 1 second to ensure the synchronization and consistency between the spatial location data and the image acquisition time series.

[0024] The control unit establishes a global geographic coordinate reference system based on GPS positioning information, using the geographic origin as the initial point of the virtual reference coordinate system. In this reference system, the X-axis corresponds to the east-west direction, the Y-axis corresponds to the north-south direction, and the Z-axis corresponds to the vertical direction. Simultaneously, the spatial feature dataset obtained in step S2 is projected onto this virtual reference coordinate system, and a coordinate transformation matrix is ​​used to achieve a consistent mapping between the spatial coordinates of the feature points and their geographic coordinates. This mapping process includes solving for the rotation matrix and correcting the translation vector, used to correct spatial deviations caused by changes in terminal attitude.

[0025] After establishing the virtual reference coordinate system, an optical anchoring process is performed. Specifically, the augmented reality display terminal uses real-time captured environmental images from a camera to identify target areas in the real-world scene (such as water monitoring points, building locations, equipment positions, etc.) through a feature matching algorithm, and binds the corresponding spatial feature points in the virtual reference coordinate system to the image positions of these target areas. The binding method involves calculating the perspective projection matrix of the two sets of coordinates and the camera's intrinsic parameters (focal length, principal point coordinates) to precisely map the virtual coordinate points onto the image plane, achieving optical overlap between the virtual space and the real-world image.

[0026] In a preferred embodiment, to improve the stability and accuracy of anchoring, a dynamic attitude compensation mechanism based on an inertial measurement unit (IMU) is introduced during the optical anchoring process. By acquiring real-time acceleration and angular velocity change data of the terminal, the camera's field of view offset is predicted, and the projection position of the anchored point in the virtual reference coordinate system is finely adjusted to ensure that the virtual element can still be stably attached to the target area when the user moves or the terminal shakes slightly.

[0027] Ultimately, the augmented reality display terminal presents virtual coordinate information superimposed on the real scene in the user's field of vision, forming the first display scene. In this display scene, the user can visually observe the alignment effect between the virtual coordinate axes, spatial markers, and reference frames and the actual object positions, thus providing a spatial basis for subsequent water resource data loading and visualization overlay.

[0028] Step S4: In response to the detection of a physical carrier in the first display scene, the layer display of the physical carrier is adjusted by the acquired posture and position parameters to obtain the second display scene.

[0029] In one embodiment, after generating the first display scene, the augmented reality display terminal first detects whether there is a physical object, such as a water sampler, measuring instrument, or other identifying object, in the user's field of view using a camera and sensor module. The detection method includes spatial point cloud acquisition based on a depth camera, as well as feature matching and template recognition based on RGB images.

[0030] Once a physical carrier is confirmed to exist within the field of view, the control unit reads its real-time attitude data (including pitch, roll, and yaw angles) and position coordinates (X, Y, Z) in space. The control unit then performs coordinate transformation and attitude mapping on this attitude and position data with a virtual reference coordinate system, ensuring that the virtual layer of the carrier is correctly overlaid on the augmented reality terminal's display interface. During the mapping process, the precise positioning of virtual information is achieved by aligning the 3D model or virtual icon of the physical carrier with its actual spatial location.

[0031] The control unit then performs layer adjustment operations. Specific operations include: rotating the virtual layer according to the carrier's real-time posture to align it with the actual orientation of the physical carrier; translating the virtual layer according to the carrier's spatial position to ensure it remains superimposed on the carrier's shape and position on the terminal display interface; and automatically adjusting the layer scaling ratio according to the viewing distance to ensure the virtual layer is clearly discernible at different viewing distances without obscuring key areas.

[0032] To further ensure display stability, the position and orientation of the carrier layer are filtered in real time. For example, the carrier parameters of consecutive frames are smoothed using moving average or Kalman filtering methods to eliminate jitter caused by user hand tremors or sensor noise.

[0033] Ultimately, the augmented reality display terminal presents a second display scene in the user's field of vision. In this scene, the virtual layer of the physical carrier has been dynamically adjusted according to its posture and position parameters. The user can clearly see the spatial position of the carrier in the real environment, and related virtual information can be overlaid, providing a spatial reference basis for subsequent water resources bulletin data overlay and operation.

[0034] In another embodiment, such as Figure 3 As shown, the second display scene obtained is the scene after spatial registration and alignment. When a user requests and calls water resource data, it can generate and render an AR visualization model in the corresponding display scene, while waiting for and responding to other user interaction commands, such as gestures and voice.

[0035] As an example of the present invention, reference is made to... Figure 2 As shown, step S3 in this example includes: Step S31: Determine the set of three-dimensional boundary points of the target area based on the spatial feature dataset, and calculate the precise position of each three-dimensional boundary point in the geographic coordinate system using the GPS positioning information extracted by spatial perception information to obtain the precise position of the boundary points; Step S32: Analyze the spatial distribution characteristics of the boundary points based on the three-dimensional boundary point set and establish a virtual reference coordinate system; use the virtual reference coordinate system to synchronously map the precise positions of the boundary points to obtain boundary point mapping data; Step S33: In the terminal display interface, the boundary points in the virtual reference coordinate system are projected onto the camera's field of view coordinate plane through the boundary point mapping data to obtain a virtual superimposed view that is consistent with the real image view. Step S34: Optically anchor the target area using a virtual overlay perspective to obtain the first display scene.

[0036] In one embodiment, a sequence of environmental image frames in the user's viewing direction is first continuously acquired using a camera, and feature extraction and depth estimation are performed on the image frames. By identifying feature points at the boundary of the target area and combining them with the terminal's posture data, spatial compensation is performed on the feature points to eliminate parallax errors caused by hand shake or rotation. Subsequently, the relative depth of the feature points is combined with GPS data, and each three-dimensional boundary point is mapped to a geographic coordinate system through coordinate transformation to form a three-dimensional boundary point set, and the spatial coordinates and timestamp of each point are recorded.

[0037] Next, a spatial distribution analysis of the 3D boundary point set is performed. First, the minimum bounding volume of the boundary points is determined, the center position of the target region and the principal direction vector are calculated, and then the boundary morphology is determined based on the boundary point density. The center of the target region is taken as the origin, the Z-axis direction is defined by the principal direction vector, and the perpendicular direction is taken as the XY plane, thus establishing a virtual reference coordinate system. Subsequently, the geographic coordinates of the boundary points are transformed to this reference coordinate system, and low-confidence points are interpolated and their positions corrected to ensure that the boundary point distribution is continuous, closed, and without isolated points, forming boundary point mapping data.

[0038] Subsequently, the boundary point mapping data is projected onto the camera's field of view coordinate plane. Based on the camera's focal length, pixel size, optical center, and terminal posture, the three-dimensional coordinates of each boundary point are converted into two-dimensional pixel positions. Linear interpolation and Gaussian filtering are then used to smooth the boundary contours, ensuring continuous projection curves and smooth edges, thereby generating virtual overlay view data consistent with the real-world scene perspective.

[0039] Finally, in the terminal display interface, the virtual overlay perspective data is rendered in real time to form the first display scene. As the user moves, the registration relationship between the virtual boundary and the real environment is updated according to the change in posture, and the display position of the virtual boundary is corrected in real time through an inter-frame differential optical anchoring algorithm. When a recognizable physical object is detected in the real environment, the rotation angle and scaling ratio of the virtual boundary are adjusted according to the spatial position of the physical object, so that the outline of the virtual boundary is precisely aligned with the surface features of the physical object, thereby achieving augmented reality spatial anchoring display of the target area.

[0040] Preferably, step S33 includes the following steps: Step S331: Import the boundary point mapping data into the projection mapping unit in the terminal display interface to confirm the virtual scene; Step S332: Project the three-dimensional spatial coordinates of each boundary point onto the camera's field of view coordinate plane using homogeneous coordinate transformation to obtain the two-dimensional projected coordinates of the boundary points; Step S333: Reconstruct the virtual scene contour based on the two-dimensional projection coordinates of each boundary point to generate a virtual contour consistent with the camera's field of view; Step S334: By aligning the virtual and real boundaries, the virtual contour is spatially superimposed with the boundary features of the environment image to obtain a virtual superimposed viewpoint consistent with the viewpoint of the real image.

[0041] In one embodiment, boundary point mapping data is imported into the display interface to generate the basic geometric information of the virtual scene. For each boundary point, based on its three-dimensional spatial coordinates and timestamp, combined with the camera's intrinsic parameters (including focal length, principal point coordinates, and pixel size) and pose information, the three-dimensional coordinates are projected onto the camera's field of view coordinate plane through homogeneous coordinate transformation to obtain the two-dimensional projected coordinates corresponding to each boundary point. During the projection process, the depth of the boundary points is normalized to ensure the correct occlusion relationship, and real-time compensation is performed for any jitter that may occur during motion.

[0042] Subsequently, the virtual scene contour is reconstructed based on two-dimensional projected coordinates. Specifically, the boundary points are sorted according to the spatial connectivity of the projected coordinates, and a continuous and smooth contour line is generated using curve interpolation methods (such as B-spline interpolation). At the same time, Gaussian filtering and local smoothing are applied to the contour line to eliminate jagged edges caused by projection discretization, ensuring the continuity and natural transition between the contour line and the boundary features in the actual image.

[0043] Next, by aligning the virtual contour with the boundary features in the environmental image, the virtual and real boundaries are superimposed. First, the corresponding target boundary feature points in the environmental image are extracted. The registration relationship between the virtual contour and the image boundary is calculated using affine transformation or homography matrix. Then, the virtual contour is rotated, translated, and scaled to ensure its spatial position matches the boundary features of the environmental image. As the user moves or the camera view changes, the projection position of the virtual contour is updated in real time to ensure continuous alignment between the virtual contour and the boundary features of the real image, thus obtaining a virtual overlay viewpoint consistent with the real image's perspective. Finally, after the above processing, the virtual contour can accurately match the boundary features of the real scene, supporting target area visualization and spatial anchoring in augmented reality, providing reliable basic data for the subsequent generation of a second display scene.

[0044] Preferably, the method for obtaining the camera's field of view coordinate plane includes: Obtain camera intrinsic and extrinsic parameters, where the camera intrinsic parameters are used to characterize the internal imaging characteristics of the camera, and the camera extrinsic parameters are used to characterize the camera's attitude and position in real space; The camera's focal length, optical center coordinates, and pixel ratio are determined based on the camera's internal parameter data. The rotation and translation vector parameters of the camera in three-dimensional space are determined by using the camera's extrinsic parameter data. The camera's field of view coordinate plane is determined based on focal length parameters, optical center coordinate parameters, pixel ratio parameters, rotation vector parameters, and translation vector parameters.

[0045] In one embodiment, the camera's intrinsic and extrinsic parameters are obtained. The camera's intrinsic parameters characterize the camera's internal imaging characteristics, including focal length, optical center coordinates, and pixel ratio; the camera's extrinsic parameters characterize the camera's attitude and position in three-dimensional space, including rotation and translation vectors.

[0046] Next, based on the camera's intrinsic parameter data, the camera's focal length, optical center coordinates, and pixel ratio are determined. The focal length describes the camera lens's magnification capability; the optical center coordinates determine the position of the imaging center on the image plane; and the pixel ratio converts physical spatial units to pixel units to ensure the accuracy of spatial point projection.

[0047] Simultaneously, the camera's attitude and position in three-dimensional space are determined using camera extrinsic parameter data. Specifically, rotation vector parameters represent the camera's direction and orientation, while translation vector parameters determine the camera's coordinate position in real space. The combination of rotation and translation vectors describes the complete transformation relationship of the camera from the world coordinate system to the camera coordinate system.

[0048] Finally, based on the aforementioned focal length parameters, optical center coordinate parameters, pixel ratio parameters, rotation vector parameters, and translation vector parameters, the camera's field of view coordinate plane is determined. By mapping target points in three-dimensional space to the two-dimensional image plane according to intrinsic and extrinsic parameters, precise positioning of spatial points within the camera's field of view is achieved, providing a reliable geometric basis for subsequent virtual contour projection and augmented reality display.

[0049] Preferably, determining the camera's field of view coordinate plane based on focal length parameters, optical center coordinate parameters, pixel ratio parameters, rotation vector parameters, and translation vector parameters includes: Based on the focal length parameter, optical center coordinate parameter, and pixel ratio parameter in the camera's intrinsic parameters, the positional relationship of the image plane is determined; Based on the rotation vector parameters and translation vector parameters in the camera's extrinsic parameter data, the spatial pose of the camera relative to the virtual reference coordinate system is determined. The positional relationship of the image plane is combined with the spatial pose of the camera to determine the camera's field of view coordinate plane.

[0050] In one embodiment, the positional relationship of the camera's image plane is determined based on the camera's intrinsic parameter data, including focal length parameters, optical center coordinate parameters, and pixel ratio parameters. The focal length parameter describes the imaging magnification capability of the camera lens, the optical center coordinate parameters determine the position of the imaging center on the image plane, and the pixel ratio parameters convert physical spatial units to pixel units, thereby ensuring the accuracy and proportional relationship of the mapping of spatial points to the image plane.

[0051] Secondly, based on the rotation and translation vector parameters in the camera's extrinsic parameter data, the spatial pose of the camera relative to the virtual reference coordinate system in three-dimensional space is determined. Specifically, the camera's direction and orientation are determined by the rotation vector, and the camera's position in the reference coordinate system is determined by the translation vector, thus realizing the spatial mapping relationship between the camera coordinate system and the virtual reference coordinate system.

[0052] Finally, the positional relationships of the image planes are combined with the spatial pose of the camera to determine the camera's field of view coordinate plane. This field of view coordinate plane allows for precise projection of target points in three-dimensional space onto the camera's two-dimensional image plane, providing a reliable geometric basis for the spatial alignment of virtual scenes with the real environment in augmented reality displays.

[0053] Preferably, step S34 includes: Determine the mapping position of the boundary points of the target area in the camera's field of view coordinate plane based on the virtual overlay view data; The target area's two-dimensional pixel grid is segmented and covered using the mapped location, and the pixel coordinates of each segment are matched one by one with the corresponding virtual boundary points; Apply segment-by-segment translation correction to each matched pixel coordinate to ensure that the virtual boundary points remain continuously aligned with the target area of ​​the environment image from a visual perspective. The virtual boundary points, which have been corrected by segment-by-segment translation, are superimposed with the pixel information of the target area to form virtual elements after optical anchoring. The first display scene is generated based on the continuity of view and the consistency of boundaries of each virtual element.

[0054] In one embodiment, the mapping positions of the boundary points of the target area in the camera's field of view coordinate plane are determined based on virtual overlay view data. During the mapping process, the boundary points in three-dimensional space are projected through the camera's field of view coordinate plane to obtain the two-dimensional pixel coordinates corresponding to each boundary point, providing basic geometric information for subsequent virtual element overlay.

[0055] Subsequently, the pixel grid of the target region is segmented and covered using the mapped two-dimensional pixel coordinates. Specifically, the target region is divided into several consecutive pixel segments according to the arrangement of the boundary points, and the pixel coordinates of each segment are matched one by one with the corresponding virtual boundary points, thereby establishing the correspondence between virtual boundary points and real image pixels.

[0056] For each matching pixel coordinate segment, a segment-by-segment translation correction operation is applied. By calculating the deviation between the virtual boundary point and the target region boundary in the environmental image, the pixel segment is slightly translated along the horizontal and vertical directions to ensure that the virtual boundary point is continuously aligned with the actual target region from the viewpoint, thus ensuring the geometric continuity and visual consistency of the boundary.

[0057] After segment-by-segment translation and correction, the virtual boundary points are superimposed with the pixel information of the target area to form optically anchored virtual elements. During the superposition process, not only are the spatial relationships of the virtual elements preserved, but also the local pixel features of the target area are incorporated to achieve high-precision fusion of the virtual elements with the real-world image.

[0058] Finally, a first display scene is generated based on the continuity of the virtual elements in terms of viewpoint and the consistency of their boundaries. This display scene can achieve accurate overlap between virtual elements and real-world target areas in the augmented reality interface, ensuring the spatial perception accuracy and visual coherence of the user's observation.

[0059] Preferably, overlaying the virtual boundary points, which have undergone segment-by-segment translation correction, with the pixel information of the target region includes: The target pixel position of each virtual boundary point in the camera's field of view coordinate plane is determined based on the virtual boundary point data after segment-by-segment translation correction. Based on the target pixel location, extract the corresponding pixel grayscale value and color channel data from the pixel information of the target area to generate target pixel attribute data; Pixel fusion calibration processing is performed on the virtual boundary points and target pixel attribute data, including: Using the spatial coordinates of the virtual boundary points as a reference, weighted smoothing calculations are performed on the corresponding pixel grayscale values. The color of pixels at the boundary intersection is dynamically transitioned based on the brightness gradient changes of adjacent pixel channels. The mapping offset of the virtual boundary points is adjusted based on the changes in the brightness gradient to obtain the virtual boundary points after fusion calibration. The virtual boundary points processed by fusion calibration are superimposed with the pixel information of the target area to generate virtual element data after optical anchoring.

[0060] In one embodiment, the target pixel position of each virtual boundary point in the camera's field of view coordinate plane is determined based on the virtual boundary point data corrected by segment-by-segment translation. This process achieves the spatial correspondence between virtual boundary points and real image pixels by projecting the corrected boundary points in three-dimensional space onto a two-dimensional pixel plane.

[0061] Subsequently, based on the target pixel location, the corresponding grayscale value and color channel data are extracted from the pixel information of the target region to generate target attribute data for each pixel. This target pixel attribute data includes the pixel's brightness, color information, and gradient change characteristics of neighboring pixels, providing basic information for the subsequent fusion of virtual boundary points and real pixels.

[0062] Next, pixel fusion calibration is performed on the virtual boundary points and target pixel attribute data. Specifically, using the spatial coordinates of the virtual boundary points as a reference, the corresponding pixel grayscale values ​​are weighted and smoothed to achieve a continuous grayscale transition in the boundary region. Simultaneously, based on the brightness gradient changes of adjacent pixel channels, the pixel colors at the boundary intersection are dynamically adjusted and transitioned to ensure a smooth visual connection between virtual elements and real pixels. Furthermore, the mapping offset of the virtual boundary points is fine-tuned according to the brightness gradient changes to obtain the fused and calibrated virtual boundary points.

[0063] Finally, the virtual boundary points, after fusion calibration, are superimposed with the pixel information of the target area to form optically anchored virtual element data. This virtual element data can achieve high-precision, continuous, and natural superposition with the real target area in augmented reality displays, thereby ensuring spatial positioning accuracy and visual continuity for the user.

[0064] Preferred methods for detecting physical carriers include: Candidate regions with geometric contour features are extracted from environmental images, and the candidate regions are filtered for brightness and texture consistency to generate candidate regions for physical carriers. Estimate the spatial depth of candidate regions for carrying objects to determine the depth distribution information of the candidate regions; The morphological stability parameter is calculated based on the depth distribution information, and then compared with the morphological stability parameter by a preset physical morphological stability threshold. When the morphological stability parameter is greater than or equal to the preset physical morphological stability threshold, it is determined that a physical carrier exists.

[0065] In one embodiment, based on an environmental image acquired by an augmented reality terminal, preliminary detection is performed on potential physical carriers within the image. This detection process includes extracting candidate regions with geometric contour features from the environmental image, such as identifying rectangular, circular, or irregular contour regions in the image through edge detection, contour extraction, or morphological methods. Subsequently, these candidate regions are filtered for brightness and texture consistency to eliminate false contours caused by background noise or lighting variations, thereby generating candidate regions for physical carriers.

[0066] Next, spatial depth estimation is performed on each candidate region of the physical carrier to obtain depth distribution information of the candidate region. Depth estimation can be based on monocular vision depth estimation, stereo vision matching, or combined with parallax information from the inertial measurement unit to determine the distance and spatial distribution characteristics of the candidate region relative to the observer.

[0067] Then, morphological stability parameters are calculated based on the depth distribution information of the candidate regions to evaluate the morphological reliability and stability of the candidate regions in three-dimensional space. These morphological stability parameters may include indicators such as regional volume uniformity, boundary continuity, and depth gradient variation. The calculated morphological stability parameters are compared with a preset physical morphological stability threshold. When the morphological stability parameters are greater than or equal to the threshold, it is determined that the candidate region indeed contains a physical carrier.

[0068] Preferably, estimating the spatial depth of the candidate region for the physical object includes: Disparity comparison is performed on image pixels within the candidate region of the object, and the relative depth trend of the candidate region is estimated based on the disparity change of adjacent pixels. Based on the preset field of view and focal length parameters in the camera, the spatial depth trend is spatially converted to confirm the spatial depth of the candidate area for carrying physical objects.

[0069] In one embodiment, spatial depth estimation is performed on each candidate region of the physical object. First, disparity comparison is performed on the image pixels within the candidate region. Specifically, the disparity change of each pixel can be calculated by comparing the grayscale values ​​of the same pixel in adjacent frames or left and right view images, feature point matching results, or other similarity indicators. Based on the disparity change between adjacent pixels, the relative depth trend of the candidate region is estimated, reflecting the distance distribution of the region relative to the observer.

[0070] Subsequently, by combining the camera's preset field of view and focal length parameters, the aforementioned relative depth trend is spatially converted to obtain the actual depth information of the candidate region in real space. This process includes mapping the parallax between pixels to actual distances, calibrating the depth using focal length, imaging size, and field of view parameters, and finally obtaining the spatial depth value and distribution characteristics of the candidate region.

[0071] Preferably, the spatial conversion of relative depth trends based on preset field of view and focal length parameters in the camera includes: The corresponding imaging geometry is determined based on the preset field of view and focal length parameters in the camera. The pixel position changes in the relative depth trend are correlated with the imaging geometry to calculate the viewing projection direction corresponding to each pixel; Based on the changing pattern of the angle between the line of sight projection direction and the camera's main axis, the spatial ratio of the relative depth values ​​of each pixel is calculated to obtain the spatial depth of the candidate region.

[0072] In one embodiment, spatial conversion is performed on the relative depth trend of the candidate region of the physical object to obtain the spatial depth information of the region. First, the imaging geometry of the camera is established based on the preset field of view angle (e.g., a horizontal field of view of 90° and a vertical field of view of 60°) and focal length parameters (e.g., f=4.0mm). This geometry clarifies the spatial projection correspondence between each pixel on the pixel plane and the optical center of the camera, thus providing a basis for subsequent depth conversion.

[0073] Next, the relative depth trend of pixels within the candidate area of ​​the physical object is correlated with the imaging geometry. Specifically, for each pixel within the candidate area, its disparity value in consecutive frames is extracted, and the corresponding viewing direction is calculated based on the camera focal length and pixel pitch parameters. By combining the pixel's coordinate offset on the image plane with the camera focal length, the ray direction passing through the pixel from the camera's optical center is determined, and a three-dimensional viewing projection vector is generated for each pixel.

[0074] Then, based on the variation of the angle between the gaze projection vector and the camera's principal optical axis, the relative depth values ​​of the pixels are spatially scaled. For example, for the pixel spacing corresponding to the parallax Δd, the actual distance ΔZ is calculated using a formula, and then the three-dimensional coordinates (X, Y, Z) of the pixel in space are calculated by combining the ray angle θ. Pixels throughout the entire candidate region are depth-mapped using this method, ultimately generating a spatial depth distribution map.

[0075] This spatial depth distribution map accurately determines the depth variation trend and morphological characteristics of candidate regions, providing fundamental data for subsequent calculation of morphological stability parameters and adjustment of the physical carrier's position in augmented reality displays. This method is applicable to physical carriers of different sizes and distances, and can be adapted to depth estimation needs in different environments by adjusting camera focal length, pixel pitch, and field of view parameters.

[0076] Of particular importance, step S4 includes: Extract the attitude and position parameters of the physical carrier in the first display scene, including pitch angle, yaw angle and translation position information; Based on the correspondence between attitude parameters and position parameters, the spatial position of the first display scene is adjusted to obtain the adjusted first display scene; The adjusted first display scene is used to overlay layers, and the overlaid layers are gradually replaced with the first display scene to obtain the second display scene.

[0077] In one embodiment, the attitude and position parameters of the physical carrier in the first display scene are extracted. Specifically, the attitude state of the physical carrier in the current field of view is identified through collaborative calculation by the built-in camera and inertial sensor of the augmented reality terminal. The attitude parameters include pitch, yaw, and roll, and the position parameters include translational position information (x, y, z) along the three-dimensional spatial coordinate axes. Through this step, a complete spatial pose description of the physical carrier in the current display scene can be obtained.

[0078] Next, based on the correspondence between the extracted attitude parameters and position parameters, the spatial position of the first display scene is adjusted. Specifically, the virtual viewpoint is rotated according to changes in pitch and yaw angles, and the coordinate system of the display scene is translated and compensated based on translation position information to achieve geometric alignment between the virtual and real spaces. Through these adjustments, the display perspective of the first display scene is kept consistent with the actual position of the physical carrier, thereby avoiding perspective deviations or depth misalignments when images are superimposed.

[0079] Subsequently, after adjusting the spatial position, layers are overlaid based on the adjusted first display scene. Specifically, the preset virtual display content is used as the foreground layer, and its transparency is matched and pixel-level overlaid with the real scene in the adjusted first display scene. The occlusion relationship is corrected according to the depth information to ensure the continuity and realism of virtual and real elements in space.

[0080] Finally, the superimposed layers gradually replace the original images in the first display scene to obtain the second display scene. This replacement process can be achieved using a frame-level transition method, that is, by gradually changing the transparency of several consecutive frames, the virtual layer is smoothly integrated into the real scene, thereby generating a second display scene containing virtual augmented information and achieving a natural and continuous augmented reality display effect.

[0081] Most importantly, based on the correspondence between attitude parameters and position parameters, the spatial position adjustment of the first display scene includes: Based on the changing trends of pitch angle, yaw angle and roll angle in the attitude parameters, the rotation direction and rotation amplitude of the virtual viewpoint in the first display scene are determined. Based on the translational displacement represented by the position parameters, the translational offset of the first display scene in spatial coordinates is calculated, and the relative displacement compensation processing is performed on the display elements in the scene accordingly. By establishing the coupling relationship between attitude and position parameters, a spatial correspondence mapping between the virtual viewpoint and the physical carrier is established, and relative displacement compensation is performed on the first display scene to generate the adjusted first display scene.

[0082] In one embodiment, after detecting a change in the attitude of the physical carrier, three-axis rotational attitude parameters, including pitch, yaw, and roll angles, are extracted in real time, and the trend of change of each angle compared to the previous frame is calculated. For example, when the pitch angle increases, an upward tilt of the physical carrier is identified; when the yaw angle changes positively, a rightward rotation trend of the physical carrier is identified; when the roll angle changes, rotation of the physical carrier around the front and rear axes is identified, and rotation control commands for the virtual camera are generated accordingly. Each angle change corresponds to a specific rotation direction and amplitude to drive the virtual viewpoint in the first display scene to rotate synchronously. During this process, the rotation changes are smoothed through interpolation to avoid jumps or jitters in the image when the attitude changes abruptly.

[0083] Secondly, based on the detected position parameters, the displacement of the physical carrier in three-dimensional space is estimated in real time. Position parameters typically include the translational distance of the carrier in the horizontal and vertical directions. By comparing the feature point positions of the carrier in different image frames frame by frame, its direction of movement and displacement amplitude in space are calculated. When the carrier is detected moving forward or backward, the depth of field of the virtual scene is adjusted accordingly; when the carrier is detected moving left or right, the lateral offset position of the virtual elements is dynamically adjusted. In this way, translational offset data of the scene can be generated, and based on this, the coordinates of all virtual elements in the displayed screen can be batch-translated, achieving dynamic compensation of the overall space.

[0084] Subsequently, by leveraging the interaction between attitude and position parameters, a spatial mapping between the virtual viewpoint and the physical object is constructed. This mapping is determined by the rotation direction, rotation amplitude, and translation offset. Based on these three sets of data, the spatial viewpoint position and orientation of the virtual camera are dynamically updated, ensuring that the virtual scene maintains the same true proportions and orientation as the physical object in the displayed image.

[0085] For example, when the carrier rotates at a certain angle and translates to the right, the rotation trend and translation direction are fused together to determine the rotation center and translation compensation path of the virtual scene, thereby ensuring that the virtual image maintains a smooth and continuous spatial transition effect on the screen.

[0086] Finally, the first display scene after the above posture and position corrections is output as the spatial adjustment result, resulting in the adjusted first display scene. The adjusted display scene presents a virtual perspective that moves synchronously with the physical carrier in the user's field of vision, making the virtual object visually fixed on the surface or boundary of the physical object, and preventing it from shifting, floating, or misaligning due to the movement or rotation of the carrier, thereby improving the spatial stability and realism of the augmented reality display.

[0087] Therefore, the embodiments should be considered as exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of the equivalents of the application are intended to be included within the invention.

[0088] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.

Claims

1. A method for displaying water resources bulletin results based on augmented reality, characterized in that, Includes the following steps: Step S1: Acquire spatial perception information of the user's observed environment through the sensor module of the augmented reality display terminal, wherein the sensor module includes a camera, an inertial measurement unit and a positioning unit; Step S2: Extract scene feature points based on the environmental images acquired by the camera, and perform time synchronization correction on the scene feature points in conjunction with the attitude change data of the inertial measurement unit to obtain a matching spatial feature dataset; Step S3: Generate a virtual reference coordinate system in the terminal display interface based on the spatial feature dataset and GPS positioning information, and perform optical anchoring on the target area in the real scene based on the virtual reference coordinate system to obtain the first display scene; Step S4: In response to the detection of a physical carrier in the first display scene, the layer display of the physical carrier is adjusted by the acquired posture and position parameters to obtain the second display scene.

2. The method for displaying water resources bulletin results based on augmented reality according to claim 1, characterized in that, Step S3 includes the following steps: Step S31: Determine the set of three-dimensional boundary points of the target area based on the spatial feature dataset, and calculate the precise position of each three-dimensional boundary point in the geographic coordinate system using the GPS positioning information extracted by spatial perception information to obtain the precise position of the boundary points; Step S32: Analyze the spatial distribution characteristics of the boundary points based on the three-dimensional boundary point set and establish a virtual reference coordinate system; use the virtual reference coordinate system to synchronously map the precise positions of the boundary points to obtain boundary point mapping data; Step S33: In the terminal display interface, the boundary points in the virtual reference coordinate system are projected onto the camera's field of view coordinate plane through the boundary point mapping data to obtain a virtual superimposed view that is consistent with the real image view. Step S34: Optically anchor the target area using a virtual overlay perspective to obtain the first display scene.

3. The method for displaying water resources bulletin results based on augmented reality according to claim 2, characterized in that, Step S33 includes the following steps: Step S331: Import the boundary point mapping data into the projection mapping unit in the terminal display interface to confirm the virtual scene; Step S332: Project the three-dimensional spatial coordinates of each boundary point onto the camera's field of view coordinate plane using homogeneous coordinate transformation to obtain the two-dimensional projected coordinates of the boundary points; Step S333: Reconstruct the virtual scene contour based on the two-dimensional projection coordinates of each boundary point to generate a virtual contour consistent with the camera's field of view; Step S334: By aligning the virtual and real boundaries, the virtual contour is spatially superimposed with the boundary features of the environment image to obtain a virtual superimposed viewpoint consistent with the viewpoint of the real image.

4. The method for displaying water resources bulletin results based on augmented reality according to claim 3, characterized in that, Methods for obtaining the camera's view coordinate plane include: Obtain camera intrinsic and extrinsic parameters, where the camera intrinsic parameters are used to characterize the internal imaging characteristics of the camera, and the camera extrinsic parameters are used to characterize the camera's attitude and position in real space; The camera's focal length, optical center coordinates, and pixel ratio are determined based on the camera's internal parameter data. The rotation and translation vector parameters of the camera in three-dimensional space are determined by using the camera's extrinsic parameter data. The camera's field of view coordinate plane is determined based on focal length parameters, optical center coordinate parameters, pixel ratio parameters, rotation vector parameters, and translation vector parameters.

5. The method for displaying water resources bulletin results based on augmented reality according to claim 4, characterized in that, The camera's field of view coordinate plane is determined based on focal length parameters, optical center coordinate parameters, pixel ratio parameters, rotation vector parameters, and translation vector parameters, including: Based on the focal length parameter, optical center coordinate parameter, and pixel ratio parameter in the camera's intrinsic parameters, the positional relationship of the image plane is determined; Based on the rotation vector parameters and translation vector parameters in the camera's extrinsic parameter data, the spatial pose of the camera relative to the virtual reference coordinate system is determined. The positional relationship of the image plane is combined with the spatial pose of the camera to determine the camera's field of view coordinate plane.

6. The method for displaying water resources bulletin results based on augmented reality according to claim 3, characterized in that, Step S34 includes: Determine the mapping position of the boundary points of the target area in the camera's field of view coordinate plane based on the virtual overlay view data; The target area's two-dimensional pixel grid is segmented and covered using the mapped location, and the pixel coordinates of each segment are matched one by one with the corresponding virtual boundary points; Apply segment-by-segment translation correction to each matched pixel coordinate to ensure that the virtual boundary points remain continuously aligned with the target area of ​​the environment image from a visual perspective. The virtual boundary points, which have been corrected by segment-by-segment translation, are superimposed with the pixel information of the target area to form virtual elements after optical anchoring. The first display scene is generated based on the continuity of view and the consistency of boundaries of each virtual element.

7. The method for displaying water resources bulletin results based on augmented reality according to claim 6, characterized in that, The process of overlaying the virtual boundary points, which have undergone segment-by-segment translation correction, with the pixel information of the target region includes: The target pixel position of each virtual boundary point in the camera's field of view coordinate plane is determined based on the virtual boundary point data after segment-by-segment translation correction. Based on the target pixel location, extract the corresponding pixel grayscale value and color channel data from the pixel information of the target area to generate target pixel attribute data; Pixel fusion calibration processing is performed on the virtual boundary points and target pixel attribute data, including: Using the spatial coordinates of the virtual boundary points as a reference, weighted smoothing calculations are performed on the corresponding pixel grayscale values. The color of pixels at the boundary intersection is dynamically transitioned based on the brightness gradient changes of adjacent pixel channels. The mapping offset of the virtual boundary points is adjusted based on the changes in the brightness gradient to obtain the virtual boundary points after fusion calibration. The virtual boundary points processed by fusion calibration are superimposed with the pixel information of the target area to generate virtual element data after optical anchoring.

8. The method for displaying water resources bulletin results based on augmented reality according to claim 1, characterized in that, The detection methods for physical carriers include: Candidate regions with geometric contour features are extracted from environmental images, and the candidate regions are filtered for brightness and texture consistency to generate candidate regions for physical carriers. Estimate the spatial depth of candidate regions for carrying objects to determine the depth distribution information of the candidate regions; The morphological stability parameter is calculated based on the depth distribution information, and then compared with the morphological stability parameter by a preset physical morphological stability threshold. When the morphological stability parameter is greater than or equal to the preset physical morphological stability threshold, it is determined that a physical carrier exists.

9. The method for displaying water resources bulletin results based on augmented reality according to claim 8, characterized in that, Estimating the spatial depth of candidate regions for physical objects includes: Disparity comparison is performed on image pixels within the candidate region of the object, and the relative depth trend of the candidate region is estimated based on the disparity change of adjacent pixels. Based on the preset field of view and focal length parameters in the camera, the spatial depth trend is spatially converted to confirm the spatial depth of the candidate area for carrying physical objects.

10. The method for displaying water resources bulletin results based on augmented reality according to claim 9, characterized in that, Spatial conversion of relative depth trends based on preset field of view and focal length parameters in the camera includes: The corresponding imaging geometry is determined based on the preset field of view and focal length parameters in the camera. The pixel position changes in the relative depth trend are correlated with the imaging geometry to calculate the viewing projection direction corresponding to each pixel; Based on the changing pattern of the angle between the line of sight projection direction and the camera's main axis, the spatial ratio of the relative depth values ​​of each pixel is calculated to obtain the spatial depth of the candidate region.

Images