Screen observation-based game real-time two-dimensional map positioning method and device, and medium

Abstract

The application provides a game real-time two-dimensional map positioning method and device based on screen observation and a medium. The method comprises the following steps: performing screen acquisition on a target game window under the authorization of an operating system to obtain an image sequence; positioning a small map area from the image sequence and identifying a direction mark and a feature point, matching the direction mark and the feature point with corresponding structures in a preset or online constructed global two-dimensional map to select an anchor point; estimating a relative displacement and a rotation angle between adjacent frames to form motion observation; constructing an estimation model taking a position and an orientation as state variables, fusing the position observation and the motion observation, and outputting a target two-dimensional coordinate and an orientation of a player; and using the target two-dimensional coordinate and the orientation to generate a two-dimensional map layer containing a current position and orientation in an external rendering module and calling the two-dimensional map layer for display. The application can realize zero-invasion acquisition, unified coordinate system, stable and continuous real-time positioning of a player's position and orientation, and generation of a two-dimensional map.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus and medium for real-time two-dimensional map positioning in games based on screen observation. Background Technology

[0002] With the increasing popularity of large-scale open-world and multiplayer online games, game scenes are vast, complex, and quests are frequent. Players urgently need stable real-time location cues and path guidance during exploration and travel. Factors such as resolution, window scaling, interface themes, and interactive panel switching mean that relying solely on a visual minimap or temporary markers is insufficient for continuous positioning and global navigation. Obtaining usable real-time location data without modifying the game program has become a crucial technical requirement in this field.

[0003] Existing technologies mainly include the following: First, manual identification and screenshot annotation based on the embedded small map in the client; Second, tools that directly obtain coordinate data through reverse analysis, memory reading or module injection; Third, interface solutions that rely on official plugins, playback files or telemetry exports; Fourth, pure vision methods that use general computer vision for first-view odometer or template matching.

[0004] However, the aforementioned existing technologies still have the following problems: the embedded minimap solution lacks unified coordinates and robust recognition, and is easily affected by occlusion, scaling, and theme changes; reverse engineering and injection solutions have compliance and security risks, are vulnerable across versions, and are prone to conflicts with anti-cheat systems; official interfaces have inconsistent coverage, limited timeliness and data granularity, and are missing for some games; pure visual solutions are prone to scale drift, lack anchor point matching and loop constraints based on minimap and global map, and are difficult to align when map assets are incomplete; existing external visualizations are mostly static overlays, lacking rendering and caching mechanisms that coordinate with positioning results. Summary of the Invention

[0005] In view of this, embodiments of this application provide a game real-time two-dimensional map positioning method, device and medium based on screen observation, to solve the problems of existing technologies, such as difficulty in stably obtaining real-time position and orientation without modifying the game, unstable positioning due to lack of unified coordinate mapping between small map and global map, and scale drift and poor positioning continuity caused by single observation.

[0006] A first aspect of this application provides a real-time two-dimensional map positioning method for games based on screen observation, comprising: capturing the target game window screen under the authorization of the operating system to obtain an image sequence; locating a small map region from the image sequence and identifying directional markers and feature points; matching the directional markers and feature points with corresponding structures in a preset or online constructed global two-dimensional map to select anchor points and establish a geometric mapping relationship from small map pixels to a unified coordinate system; estimating the relative displacement and rotation angle between adjacent frames based on the main view frame in the image sequence to form motion observation; constructing an estimation model with position and orientation as state variables; fusing the position observation and motion observation obtained from the geometric mapping relationship to output the target two-dimensional coordinates and orientation of the player in the unified coordinate system; and using the target two-dimensional coordinates and orientation to generate a two-dimensional map layer containing the current position and orientation in an external rendering module for display.

[0007] A second aspect of this application provides a real-time two-dimensional map positioning device for games based on screen observation, comprising: an acquisition module, used to acquire images of a target game window under the authorization of the operating system to obtain an image sequence; a matching module, used to locate a small map area from the image sequence and identify directional markers and feature points, match the directional markers and feature points with corresponding structures in a preset or online constructed global two-dimensional map to select anchor points, and establish a geometric mapping relationship from small map pixels to a unified coordinate system; an estimation module, used to estimate the relative displacement and rotation angle between adjacent frames based on the main view frame in the image sequence to form motion observation; a fusion module, used to construct an estimation model with position and orientation as state variables, fuse the position observation and motion observation obtained by the geometric mapping relationship, and output the target two-dimensional coordinates and orientation of the player in a unified coordinate system; and a generation module, used to use the target two-dimensional coordinates and orientation to generate a two-dimensional map layer containing the current position and orientation in an external rendering module for display and call.

[0008] A third aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0009] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects: This method involves capturing an image sequence from the target game window under operating system authorization; locating a minimap region from the image sequence and identifying directional markers and feature points; matching these markers and feature points with corresponding structures in a pre-built or online-constructed global 2D map to select anchor points and establish a geometric mapping relationship from minimap pixels to a unified coordinate system; estimating the relative displacement and rotation between adjacent frames based on the main view frames in the image sequence to form motion observations; constructing an estimation model with position and orientation as state variables; fusing the position observations obtained from the geometric mapping relationship with the motion observations; and outputting the player's target 2D coordinates and orientation in a unified coordinate system; using the target 2D coordinates and orientation to generate a 2D map layer containing the current position and orientation in an external rendering module for display. This application achieves zero-intrusion acquisition, a unified coordinate system, and stable, continuous, real-time positioning of the player's position and orientation to generate a 2D map. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart illustrating the real-time two-dimensional map positioning method for games based on screen observation provided in this application embodiment; Figure 2 This is a schematic diagram of the structure of the game real-time two-dimensional map positioning device based on screen observation provided in this application embodiment; Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0012] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0013] Currently, there are four main methods for obtaining real-time game location data: First, relying on manual reading and screenshot annotation of the game's built-in minimap, which lacks unified coordinates and robust recognition; Second, directly obtaining coordinates through reverse engineering, memory reading, or module injection, which poses compliance and compatibility risks; Third, exporting location data using official plugins or replay logs, but with limited coverage and timeliness; Fourth, using pure visual odometry or template matching for external positioning, which is prone to scale drift, lacks anchor point alignment and loop constraints, and has insufficient coupling between external display and positioning results.

[0014] However, the existing technical solutions mentioned above still have the following shortcomings: how to stably and continuously obtain the player's position and orientation without modifying the game program; how to establish a unified coordinate mapping between the minimap and the global map to reduce drift; and how to maintain robust positioning and achieve real-time linkage with the external two-dimensional map under interface changes and occlusion.

[0015] In view of the problems existing in the prior art, this application provides a real-time 2D map positioning method for games based on screen observation. The method involves acquiring image sequences through screen capture authorized by the operating system; locating a small map region within the image sequence, identifying directional markers and feature points, and matching them with a pre-set or online-constructed global 2D map to select anchor points; solving the geometric mapping relationship from small map pixels to a unified coordinate system; estimating the relative displacement and rotation angle of adjacent frames for the main view frame to form motion observations; constructing an estimation model with position and orientation as state variables; fusing the position observations obtained from geometric mapping with the motion observations; and outputting the target's 2D coordinates and orientation under anchor point constraints; and generating a 2D map layer containing the current position and orientation in an external rendering module for display.

[0016] Through the above-described technical solution of this application, this application achieves zero-intrusion data acquisition and cross-game universal positioning, continuously outputting the player's position and orientation under a unified coordinate system; suppressing scale drift through anchor point matching and geometric mapping, and improving continuity through motion observation fusion; linking external rendering with positioning results, enabling the real-time generation of two-dimensional map layers and supporting stable display.

[0017] The technical solution of this application will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0018] Figure 1 This is a flowchart illustrating the real-time two-dimensional map positioning method for games based on screen observation provided in this application embodiment. Figure 1 As shown, this real-time 2D map positioning method for games based on screen observation can specifically include: S101, under the authorization of the operating system, performs screen capture on the target game window to obtain an image sequence; S102, Locate the small map region from the image sequence and identify the orientation markers and feature points, match the orientation markers and feature points with the corresponding structures in the preset or online constructed global two-dimensional map to select anchor points, and establish the geometric mapping relationship from small map pixels to a unified coordinate system. S103, estimate the relative displacement and rotation angle between adjacent frames based on the main view frame in the image sequence to form motion observation; S104, construct an estimation model with position and orientation as state variables, fuse position observations and motion observations obtained from geometric mapping relationships, and output the player's target two-dimensional coordinates and orientation in a unified coordinate system; S105, use the target's two-dimensional coordinates and orientation to generate a two-dimensional map layer containing the current position and orientation in the external rendering module and make it available for display.

[0019] In some embodiments, screen capture of the target game window is performed under operating system authorization to obtain an image sequence, including: A read-only pixel acquisition channel is established for the target game window without loading code into the process to which the target game window belongs or accessing the memory space of the process to which the target game window belongs. Copy the frame buffer of the target game window and assign time markers according to a uniform time base; Based on the interface layout, the copied results are divided into the main view area and the small map candidate sub-area, and the sampling frequency and resolution are set respectively to form a multi-channel image sequence.

[0020] Specifically, the screen capture module of this application operates under the authorization of the operating system and constructs a complete process around three key technical means: "read-only pixel acquisition channel, unified time base labeling, interface layout division and multi-channel sampling" to provide stable input for subsequent small map positioning and main view motion estimation.

[0021] First, without loading code into the process of the target game window or accessing its memory space, window identification and permission verification are completed, and a read-only pixel acquisition channel is established for the target game window. Specifically, the operating system's graphics compositing channel is called to copy the target window's frame buffer on the video memory side. The copying process does not change the frame buffer content and does not inject any instructions. To reduce jitter, a double-buffering strategy is used to asynchronously transfer pixel blocks between video memory and system memory. For high-resolution and multi-monitor scenarios, the copied area is corrected according to the window's scaling factor and display coordinate system parameters to ensure a one-to-one correspondence between the copied area and the visible area. When a foreground overlay or minimized state is detected, the channel enters a waiting queue and outputs a status flag to prevent invalid frames from entering subsequent processing.

[0022] Secondly, each copied frame is assigned a time identifier and sequence number identifier based on a unified time base. The unified time base is constructed based on a monotonic clock to avoid the influence of system time rollback. When a sudden change in time interval is detected due to the blocking of the acquisition thread, an interval anomaly mark is added to the frame and the most recent frames are retained in the circular buffer so that subsequent modules can use interpolation or discarding strategies to process it. At the same time, the source display number, scaling factor and visible area rectangle of the frame are recorded to form additional attributes for geometric consistency correction.

[0023] Next, based on the interface layout, the copied results are divided into the main view area and small map candidate sub-regions, and sampling strategies are configured for each. Specifically: a color histogram and edge density map are calculated for the entire frame image to generate an interface layout fingerprint; guided by the layout fingerprint, combined with prior knowledge of circular or rectangular containers and corner positions, several small map candidate windows are located; polar coordinate expansion is performed on the candidate windows, and direction scale period matching and indicator voting are conducted to obtain an orientation consistency score, and the one with the highest score is selected as the small map candidate sub-region; the remaining unoccupied main area is designated as the main view area. To avoid frequent area jumps caused by interface switching, a relocation threshold is set that is triggered by window size, position, and fingerprint; the area division is only refreshed when the cumulative changes of the three factors exceed the threshold.

[0024] Subsequently, based on the rate of change of regional content and the complexity of texture, the sampling frequency and resolution are set for the main view channel and the small map channel respectively, forming a multi-channel image sequence. The rate of change of content is measured by a combination of the energy of the difference between adjacent frames and the proportion of moving edges; the complexity of texture is measured by a combination of local entropy and corner density. Under the constraint of limited total computing power, a unified sampling budget allocator allocates the frame rate and scaling factor according to weights. For example, under the default condition, the main view channel is configured with 24 frames per second and a resolution scaling ratio of 0.075, and the small map channel is configured with 12 frames per second and a resolution scaling ratio of 0.5. When the movement of the small map indicator is accelerated or the main view changes rapidly, the sampling weight of the corresponding channel is increased. To avoid jitter introduced by frequent switching, a dual threshold stabilization strategy is adopted: the sampling parameters are increased only when the measurement continuously exceeds the upper threshold, and the sampling parameters are decreased when it continuously falls below the lower threshold. All sampled frames undergo color space correction and geometric consistency correction uniformly before being written to the circular buffer to ensure that the input for subsequent recognition and matching is consistent.

[0025] In a specific example, the user runs an open-world game in a borderless window mode. After establishing a read-only pixel acquisition channel, the screen capture module replicates the window's frame buffer at thirty frames per second, recording a monotonic timestamp and sequence number for each frame. The system uses the interface layout fingerprint to locate a circular minimap candidate sub-region in the upper right corner of the screen, and the candidate with the highest orientation consistency score is locked as the minimap channel input; the main screen is designated as the primary viewpoint area.

[0026] The sampling budget allocator sets the main view channel to 24 frames per second and the minimap channel to 12 frames per second, temporarily increasing the main view channel to 28 frames per second when the player turns quickly. When the task panel briefly covers the minimap, the system maintains area locking, triggering relocation only after the coverage lasts for a preset duration; after the coverage is removed, the sampling parameters gradually decrease according to a dual-threshold stabilization strategy. Finally, the circular buffer outputs two time-aligned multi-channel image sequences, providing synchronous input for minimap localization, orientation recognition, and main view motion estimation.

[0027] In some embodiments, the copied result is divided into a main view area and a small map candidate sub-region according to the interface layout, and the sampling frequency and resolution are set respectively to form a multi-channel image sequence, including: Generate an interface layout fingerprint from the copying results to obtain a set of candidate regions; Perform polar coordinate unfolding and orientation marker voting detection on the candidate regions in the candidate region set to determine the candidate sub-regions of the small map, and use the remaining regions as the main view regions; The unified sampling budget allocator adaptively determines the sampling frequency and resolution of the main view channel and the small map channel based on the rate of change of regional content and texture complexity, and adopts a dual threshold stabilization strategy to control parameter switching. Output a multi-channel image sequence consisting of the main view channel and the minimap channel.

[0028] Specifically, this embodiment divides the copying results into regions based on the interface layout and forms a multi-channel image sequence through adaptive sampling. Specifically, it includes four steps: interface layout fingerprint generation, candidate region screening and orientation voting determination, sampling budget allocation and dual threshold stabilization control, and time-aligned output. These steps are interconnected and provide synchronous and stable input for subsequent small map matching and main view motion estimation.

[0029] First, an interface layout fingerprint is generated from the copied results to obtain a set of candidate regions. The interface layout fingerprint is composed of a color histogram, edge density distribution, and container contour prior: the block color histogram and edge density are calculated across the entire frame, and the saliency peaks of the four corners and edge bands are extracted; combined with the contour priors of circular or rectangular containers and the contrast features of the borders, a fingerprint vector describing the interface structure is formed. Based on the similarity of this fingerprint between the current frame and historical frames, a sliding window update strategy is used to maintain fingerprint stability, and candidate region reconstruction is triggered when the similarity drops sharply. A set of candidate regions is generated based on the highly saliency regions of the fingerprint. The initial positions of the candidates are preferentially located in corners and edge bands, with shape constraints of near-circular or near-rectangular shapes and size constraints within a preset ratio range.

[0030] Furthermore, polar coordinate unfolding and azimuth marker voting detection are performed on candidate regions in the candidate region set to determine candidate sub-regions of the small map. For each candidate region, polar coordinate unfolding is first performed, the periodicity of angular energy is calculated, and possible azimuth scales are searched. Voting is then conducted in the angular direction with a fixed angular step size, and an azimuth consistency score is obtained by combining scale period consistency, indicator shape matching, and boundary sharpness. The scores of several adjacent frames are weighted by temporal consistency, and the candidate with the highest and most stable score is selected as the candidate sub-region of the small map, with its initial rotation value and scale parameters recorded. The remaining unoccupied main areas are designated as the main viewpoint areas. When a change in interface theme, resolution change, or prolonged presence of an overlay panel is detected, a re-evaluation of the candidate set and re-locking of the small map sub-regions are triggered.

[0031] Furthermore, a unified sampling budget allocator adaptively determines the sampling frequency and resolution of the main view channel and the small map channel based on the regional content change rate and texture complexity, and employs a dual-threshold stabilization strategy to control parameter switching. The content change rate is comprehensively measured by the differential energy between adjacent frames and the proportion of moving edges, while texture complexity is comprehensively measured by local entropy and corner density. Under the constraints of total computing power and bandwidth, the sampling budget allocator allocates frames per second and scaling factor to the two channels based on the weighted results of the two types of metrics. When the metric of a certain channel continuously exceeds the upper threshold, the sampling parameters of that channel are increased; when it continuously falls below the lower threshold, the sampling parameters are decreased. A hysteresis zone is set between the two thresholds to suppress frequent jitter. If a small map sub-region is detected to be temporarily occluded, its sampling parameters are kept unchanged and the cache validity period is extended; after the occlusion timeout, it is downgraded according to the metric results.

[0032] Furthermore, the output consists of a multi-channel image sequence composed of a main view channel and a small map channel. The two channels share a unified time base, and frames that have been corrected for color space uniformity and geometric consistency are sequentially written into their respective circular buffers. The output is then aligned according to the timestamp to ensure that the main view frame and the small map frame at the same moment can be synchronously used by subsequent modules.

[0033] For example, in some examples, the target game runs at a resolution of 2560×1440. The system generates several candidate regions in the upper right corner and completes polar coordinate unfolding and orientation voting. The orientation consistency score of the candidate sub-regions on the minimap consistently exceeds the threshold and is stably locked; the main view area covers the rest of the main screen. The default allocation is 24 frames per second for the main view with a scaling factor of 0.75, and 12 frames per second for the minimap with a scaling factor of 0.5. When the player turns around quickly, causing the change rate of the main view content to continuously exceed the upper threshold, the main view channel automatically increases to 28 frames per second; when the task panel briefly covers the minimap, since the occlusion timeout has not been exceeded, the minimap sampling parameters remain unchanged, and output can continue without relocation after the coverage is removed. Throughout the process, both frames are aligned with a unified time base for parallel consumption by minimap matching and motion estimation.

[0034] This embodiment uses layout fingerprinting and orientation voting to stably lock the sub-region of the small map. It maintains the temporal and spatial consistency of the two channels under the condition of limited computing power through adaptive sampling and dual threshold stabilization control. It outputs multi-channel image sequences with a unified time base, providing continuous, synchronous and controllable input for subsequent anchor point matching, geometric mapping solution and motion observation fusion.

[0035] In some embodiments, locating a small map region from an image sequence and identifying location markers and feature points includes: Candidate windows for small maps are generated in image sequences based on interface layout fingerprints and color texture priors. The candidate windows are expanded in polar coordinates and the orientation markers are determined by direction scale period matching and indicator voting. The target small map area is selected by cross-frame consistency scoring. Within the target minimap area, the road texture is skeletonized to extract branch points, and the set of feature points is determined by combining the semantic anchor points obtained from boundary corner detection and place name text recognition. Based on geometric and temporal consistency, the location identifiers and feature point sets are filtered and output.

[0036] Specifically, this embodiment locates the small map region from the image sequence and identifies the directional markers and feature points. It unfolds around four steps: candidate window generation, directional marker determination, road texture skeletonization and semantic anchor point acquisition, and cross-frame consistency screening, so as to form a stable input for subsequent anchor point matching and geometric mapping solution.

[0037] First, candidate windows for the small map are generated in the image sequence based on the interface layout fingerprint and color texture prior. The interface layout fingerprint is characterized by the color histogram of the entire frame, the edge density distribution, and the container outline contrast. Several candidate positions are obtained by searching for near-circular or near-rectangular containers with prominent contrast at the four corners and edges. The color texture prior is used to distinguish the common background colors of the small map from the background colors of the interface panel, such as the light-colored circular border and the medium-brightness terrain texture inside the small map. Candidates that do not meet the prior are eliminated. For the retained candidates, their center, outer boundary, and estimated radius are recorded to form a set of candidate windows.

[0038] Furthermore, the candidate windows are expanded in polar coordinates, and directional scale period matching and indicator voting are used to determine the azimuth markers. For each candidate window, polar coordinates are expanded with the window center as the pole. The energy sequence is calculated along the angular direction, and period peaks are detected to identify possible scale sequences. Voting is conducted in the angular direction with a fixed angular step size, and a azimuth consistency score is given by combining scale period consistency, indicator shape matching, and boundary sharpness. A time-weighted average is performed on several consecutive frames. If the score of the same candidate consistently exceeds a threshold within the sliding window, the candidate is confirmed as the target small map region, and the azimuth marker is recorded with the highest-scoring angular direction corresponding to north. Simultaneously, the initial rotation value and scale parameters of this region are recorded. If multiple high-scoring candidates exist within a frame, the one closest to the historical target small map region is selected; if no effective candidate is found in a short time, the previous result is retained and marked as pending confirmation.

[0039] Furthermore, within the target minimap area, road textures are skeletonized to extract branch points, and the feature point set is determined by combining boundary corner detection and semantic anchor points obtained from place name text recognition. Specifically: based on common grayscale and color distributions within the minimap, adaptive threshold segmentation is performed on the region to obtain binary images of roads and non-roads; the binary images are refined to obtain road skeletons, and skeleton nodes with a degree greater than two are used as branch points; polyline corner detection is performed at the circular or rectangular boundaries of the minimap to obtain stable boundary corner points as geometric auxiliary anchors; candidate connected components of text within the region are filtered by size and aspect ratio, and then the place name text is recognized using a local text recognition model, retaining the center of text boxes with confidence exceeding the threshold as semantic anchor points, while recording the text content and directional relationship. Finally, a feature point set is formed, which includes three types of elements: branch points, boundary corner points, and semantic anchor points, each element carrying pixel coordinates and category identifiers.

[0040] Furthermore, the location identifiers and feature point sets are filtered and output based on geometric and temporal consistency. For geometric consistency, local geometric constraints are constructed using the initial rotation value and scale parameters of the target minimap region, eliminating abnormal feature points that deviate beyond the constraint limits; local topology is formed for skeleton bifurcation points and boundary corner points according to their adjacency relationships, removing broken or isolated nodes. For temporal consistency, cross-frame survival counts and position jitter metrics are maintained for each feature point, with points having excessively low survival counts or excessive jitter being downweighted or eliminated; the location identifiers are output stably using a combination of angular mode and confidence average within a sliding window. When a change in interface theme or resolution is detected, a one-time tolerance window is triggered to relax the filtering threshold, and the normal threshold is restored after stabilization.

[0041] For example, in some examples, the target game runs at a resolution of 2560×1440, and the system obtains three candidate windows in the upper right quadrant of the screen. After polar coordinate expansion, one of the candidates has an orientation consistency score higher than the threshold for twenty consecutive frames and is identified as the target minimap area, with the north angle determined to be near zero degrees. Within this area, binary segmentation and refinement yield the skeleton of the circular main road and four branch points, while four corner points are identified at the boundaries; text recognition yields two place name boxes, "Main Square" and "North Gate," which are added to the feature point set as semantic anchors. After geometric and temporal consistency filtering, the retained branch points and place name anchors stably reproduce in subsequent frames, and the orientation identifiers remain within a small range of fluctuation.

[0042] This embodiment uses layout fingerprints and color texture priors to quickly locate candidate windows, utilizes polar coordinate expansion and voting mechanisms to stably determine directional markers, and uses a multi-source feature point set composed of skeletonized bifurcations, boundary corners and place name text as output. After geometric and temporal consistency filtering, stable small map regions, directional markers and feature points are obtained, providing reliable input for subsequent anchor point matching and geometric mapping from the small map to a unified coordinate system.

[0043] In some embodiments, the orientation markers and feature points are matched with corresponding structures in a pre-built or online-constructed global two-dimensional map to select anchor points and establish a geometric mapping relationship from small map pixels to a unified coordinate system, including: Perform tile indexing on the global 2D map and generate a topology map of road skeletons and branch points; The initial value of the minimap's rotation relative to north is determined based on the orientation markers, and the topological fingerprint is calculated for the feature points according to the adjacent angle sequence and connectivity. On the topology map, a corresponding candidate set is established based on the consistency between topological fingerprint and orientation, and compatibility is filtered using semantic anchor points; Multiple non-collinear anchor points are selected from the candidate set. Based on the pixel coordinates of the non-collinear anchor points and the global map coordinates, a robust estimation is used to solve the homography transformation matrix to obtain the geometric mapping relationship from the small map pixels to the unified coordinate system.

[0044] Specifically, the global 2D map is first indexed using tiles to generate a topological map of road skeletons and branch points. The map is then divided into tiles of fixed side lengths, and a quadtree index is established to facilitate rapid retrieval of structural elements within local areas. The map texture is binarized and thinned to obtain the road skeleton, and nodes with a degree of not less than two are extracted as branch points. An undirected graph is generated based on connectivity. At the same time, the geometric center and boundary vertices of place name labels are retained as auxiliary elements, and attributes such as the direction angle sequence and side length ratio of adjacent edges are recorded for each topological node.

[0045] Then, based on the orientation markers on the minimap, an initial rotation value relative to north is determined. Topological fingerprints are calculated for feature points within the minimap using adjacent angle sequences and connectivity. For branch points obtained from road skeleton refinement, the directional angles of adjacent skeleton branches are statistically analyzed and arranged in ascending order of polar angle to form an angle sequence. The relative length ratio and included angle distribution of adjacent branches are recorded as the topological fingerprint of the branch point. For boundary corner points and semantic anchor points, the incident angle of the boundary polyline, the center position of the text box, and the text content are recorded as supplementary geometric and semantic fingerprints. The initial rotation value given by the orientation markers is applied to the feature points on the minimap to align their angle dimensions with the global topology. Figure 1 To.

[0046] Furthermore, a corresponding candidate set is established on the global topology map based on topological fingerprint and orientation consistency, and compatibility screening is performed using semantic anchors. Starting from the tile corresponding to the center of the small map, topological nodes with similar feature point fingerprints are searched within several adjacent tile layers. Candidates with consistent angular sequence similarity and connectivity are prioritized, while candidates with orientation differences exceeding a threshold are directly eliminated. When place name semantic anchors exist within the small map, only nodes that match the place name dictionary and whose spatial position relationship is consistent with the text direction are retained in the candidate set. When only boundary corners are involved, candidate nodes are constrained to be located within the convex hull neighborhood of the outer edge of the road skeleton. The corresponding candidate set obtained through the above screening covers three types of elements: branch points, boundary corners, and semantic anchors, providing a pool of candidates for subsequent anchor selection.

[0047] Furthermore, multiple non-collinear anchor points are selected from the candidate set. A robust estimation method is used to solve the homography transformation matrix based on the pixel coordinates of the non-collinear anchor points and the global map coordinates, thus obtaining the geometric mapping relationship from the small map pixels to the unified coordinate system. Specifically: at least four non-collinear anchor points are selected first to form an initial solution set based on spatial distribution uniformity. A robust estimation method combining weighted least squares and random sampling consistency is used to gradually eliminate outliers with reprojection errors exceeding a threshold. The weights are jointly given by topological matching similarity and semantic consistency, with higher weights assigned to text anchor points and highly connected branch points. When anchor points are detected to be nearly collinear or concentrated in a local area, a degradation judgment is triggered, and the tile retrieval range is expanded to introduce new candidates. After the solution is completed, the homography matrix is ​​scaled and oriented to ensure that the north direction aligns with the vertical axis of the unified coordinate system. The confidence level and residual statistics of the matrix are recorded for runtime monitoring.

[0048] For mapping maintenance, the obtained homography transformation matrix and its confidence level are written to the mapping cache. Incremental reestimation is performed with a small number of newly added anchor points when the small map theme changes or the zoom level changes. When a continuous increase in reprojection residuals or a decrease in the number of available anchor points is detected, the system automatically reverts to the most recent high-confidence solution and initiates extended retrieval to restore a stable mapping. To improve operational efficiency, a short-term hot cache is maintained for areas with frequent tile index hits, and the survival count and historical residuals of each anchor point are recorded for rapid reuse.

[0049] For example, in some cases, the player is located near the city's main square. The minimap extracts four branch points, two place name semantic anchors, and three boundary corner points. After providing initial rotation values ​​based on the orientation identifiers, the system calculates an angle sequence consisting of three branches for each branch point and matches it to the corresponding node in the global topology map. The place names "Main Square" and "North Gate" match the map dictionary and have correct spatial relative positions, thus retaining compatible nodes in the candidate set. Finally, five non-collinear anchor points, including two branch points and two semantic anchors, are selected for robust estimation. The homography matrix is ​​obtained, and residual statistics confirm that the reprojection median is less than a threshold. This matrix is ​​then written to the mapping cache, and incremental corrections are performed using newly added branch points when the player moves to adjacent blocks, continuously maintaining a stable mapping of the unified coordinate system.

[0050] This embodiment achieves rapid retrieval through tile indexing and topology graph construction. It obtains a set of highly reliable anchor points by combining orientation alignment and topology fingerprint matching with semantic compatibility filtering. Robust estimation is used to solve homography transformation and degradation processing to ensure mapping reliability. Thus, the geometric mapping relationship between small map pixels and a unified coordinate system is continuously established under different interface themes and scaling conditions, providing a stable and reusable geometric foundation for subsequent location observation and fusion estimation.

[0051] In some embodiments, estimating the relative displacement and rotation angle between adjacent frames based on the main view frame in an image sequence to form motion observations includes: A multi-scale pyramid is constructed for adjacent main view frames to extract corner and edge features and calculate sparse matching, while also calculating dense optical flow; A dynamic object suppression mask is generated based on the optical flow consistency residual and appearance difference. Based on the horizon estimation, the main view frame is divided into ground candidate area and non-ground area, and affine fitting is performed on the matching interior points and optical flow interior points in the ground candidate area to obtain the planar displacement vector. Phase correlation is performed on adjacent frames to obtain initial rotation values, and the rotation angle is obtained by affine fitting constrained by the initial rotation values. The time consistency gating of the sliding window is used to filter the planar displacement vector and rotation angle, and the motion observation is output.

[0052] Specifically, this embodiment estimates the relative displacement and rotation angle between adjacent frames based on the main view frame in the image sequence, constructs a processing link consisting of multi-scale pyramid, joint measurement of features and optical flow, dynamic object suppression, affine fitting under horizon constraint, initial value of phase-related rotation and sliding window time consistency gating, and outputs motion observations for fusion positioning.

[0053] First, a multi-scale pyramid is constructed for adjacent main-view frames, and corner and edge features are extracted at each scale to obtain sparse matching, while dense optical flow is calculated simultaneously. Corner points are used to provide stable texture constraints, and edge features are used to enhance matching availability in low-texture regions. Sparse matching is verified by cross-scale back projection, and the set of interior points is retained. Dense optical flow is estimated in both forward and backward directions for subsequent consistency checks. To ensure temporal alignment, both frames involved in the calculation come from a circular buffer with a unified temporal base annotation, and color and geometric consistency corrections are performed before entering this stage.

[0054] Furthermore, a dynamic object suppression mask is generated based on the optical flow consistency residual and appearance difference. The optical flow consistency residual is measured by the composite deviation of the forward and backward optical flow, while the appearance difference is given by the normalized difference in brightness between adjacent frames and the local texture similarity. These two are normalized and then fused in a weighted manner to obtain a dynamic probability map. Connected component analysis and morphological refinement are used to remove noisy patches, forming the dynamic object suppression mask, which is used to mask pixels and matching points corresponding to non-rigid or independently moving regions such as pedestrians, vehicles, and flags in subsequent fitting.

[0055] Furthermore, based on horizon estimation, the main view frame is divided into ground candidate regions and non-ground regions. Within the ground candidate region, affine fitting is performed on matched interior points and optical flow interior points to obtain the planar displacement vector. Horizon estimation can be jointly completed based on vanishing point inference and ground texture direction consistency: first, the main vanishing point is inferred using line segment clustering, and then the horizon position is determined by combining the geometric relationship between the camera's view direction and the ground normal. Taking the area below the horizon as the ground candidate region, the aforementioned sparse matched interior points and dense optical flow interior points are aggregated under the action of dynamic object suppression masking, and a robust weighted affine model is used to fit the planar motion to obtain the planar displacement vector; the weights are inversely proportional to the local consistency of the points, texture confidence, and dynamic probability. When the area of ​​the ground candidate region is insufficient, the candidate region is temporarily expanded to a strip-shaped area near the horizon, and a regularization term is added to the fitting parameters to suppress ill-posedness.

[0056] Furthermore, phase correlation is performed on adjacent frames to obtain initial rotation values, which are then used to constrain affine fitting to obtain the rotation angle. Phase correlation calculates the global pose offset between two frames in the frequency domain, using its rotation component as the initial rotation value. This initial rotation value is injected into the angle parameters of the affine model as a priori, and the angle search range is limited to avoid angle drift caused by repeated textures. After completing the constrained quadratic fitting, the rotation angle estimate for the current frame pair is obtained. If the phase correlation peak significance is insufficient, a small-angle linear approximation is used, and the angle regularization weight of the affine model is increased to ensure result stability.

[0057] Furthermore, time-consistent gating using a sliding window is employed to filter the planar displacement vector and rotation angle, outputting motion observations. The sliding window maintains the displacement and rotation sequence of the most recent few frames, detecting outliers based on first-order motion assumptions and an innovation threshold. When an observation exceeds the threshold, its weight is replaced or reduced by the predicted value within the window. Gradually varying constraints are applied to displacement, and smooth angular velocity constraints are applied to rotation, with statistics updated synchronously as the window slides. Finally, the gated and filtered planar displacement vector and rotation angle are output based on the window's center time, along with the observation confidence, and provided to the subsequent state estimator.

[0058] For example, in some examples, the main view resolution is 2560×1440, and the player runs at a moderate speed and briefly looks around. The system obtains stable sparse matching and calculates dense optical flow on a three-layer pyramid. Dynamic object suppression masks are generated through forward and backward consistency and brightness difference to obscure the mounted guards and waving flags rushing on the left side of the screen. The horizon is inferred from the intersection of distant street edge segments and building facades. Intrinsic points within the ground candidate area are affine-fitted to output planar displacement vectors. Phase correlation provides initial values ​​for small-angle rotations, and constrained fitting yields the rotation angle for the current frame. A sliding window gates abnormally large displacements when the player makes a brief stop, outputting a smooth and continuous motion observation sequence.

[0059] This embodiment robustly obtains planar displacement and rotation between adjacent frames through the synergistic effect of joint measurement of features and optical flow, dynamic object suppression, plane affine fitting under horizon constraints, and phase-related rotation prior. It also suppresses instantaneous anomalies through sliding window time consistency gating, ultimately providing low-latency and continuous motion observation input for fusion positioning.

[0060] In some embodiments, an estimation model is constructed using position and orientation as state variables, including: Two-dimensional position, orientation and geometric mapping scale offset are used as state variables, and motion observation is used as process input to establish state recursion. The position obtained from the geometric mapping relationship is used as the observation to establish the observation equation. The observation confidence level is calculated based on the anchor point topological residual, azimuth identifier consistency and time consistency, and the observation noise and process noise are adaptively recalibrated based on the observation confidence level. When the observation confidence level is below the threshold, observation gating is implemented and the process priority mode is switched. After the observation confidence level recovers, the fusion mode is switched back. Anchor point holding constraints are applied within a sliding window, and the geometric mapping scale bias is jointly estimated to output the target's two-dimensional coordinates and orientation.

[0061] Specifically, the state variables are first defined as three parts: two-dimensional position, orientation, and geometric mapping scale bias, denoted as position component, orientation component, and scale bias component, respectively. Using the planar displacement vector and rotation angle from motion observations as process inputs, the state variables are recursively updated according to a unified time base and frame intervals: the position component is superimposed based on the position and planar displacement vector of the previous moment; the orientation component is superimposed based on the orientation and rotation angle of the previous moment; and the scale bias component is modeled using a zero-mean random walk based on a slowly varying assumption. The process noise intensity is adaptively adjusted according to the innovation magnitude of motion observations and the fitting residuals of ground candidate regions to maintain recursive stability in scenarios with sharp turns, accelerations, and sparse textures.

[0062] Furthermore, a location observation model derived from geometric mapping relationships is established. Utilizing the aforementioned geometric mapping relationship from small map pixels to a unified coordinate system, the coordinates of the current location indicator pixels within the small map are transformed to obtain the location observation. This location observation is then used to establish an observation equation with the location component in the state variables. Considering that interface zooming and small map theme switching may introduce scale perturbations, a scale bias component is introduced into the observation equation, allowing the observation residual to simultaneously constrain both the location and scale bias components.

[0063] Furthermore, observation confidence is calculated and noise adaptive recalibration is performed. Observation confidence consists of three parts: first, anchor point topological residuals, which are the geometric residuals and angular sequence differences when reprojecting nearby anchor points onto the global topological map using the current geometric mapping; second, azimuth label consistency, which is the angular difference between the current orientation component and the azimuth label on the minimap, and its consistency score within the sliding window; and third, temporal consistency, which is the sequence stability of location observation innovations and cross-frame survival counts. These three metrics are normalized and weighted to form a confidence metric with dimensions between zero and one. Based on this confidence metric, observation noise and process noise are adaptively recalibrated: when confidence increases, observation noise is reduced and process noise is moderately increased to suppress over-memory; when confidence decreases, observation noise is increased and process noise is reduced to mitigate the impact of unreliable observations.

[0064] Furthermore, observation gating and mode switching are implemented. Hysteresis-based upper and lower thresholds are set. When the observation confidence falls below the lower threshold, observations exceeding the threshold are rejected or downweighted, and the system switches to a process-priority mode, relying solely on motion observations for recursion while maintaining a gradual change in the scale bias component. When the observation confidence recovers and continuously exceeds the upper threshold, the system switches back to the fusion mode, resuming regular fusion of position observations. Mode switching status is recorded over time to avoid frequent jitter during short-term fluctuations.

[0065] Furthermore, anchor point preservation constraints are applied within the sliding window, and the geometric mapping scale bias is jointly estimated. The sliding window maintains the state variables, position observations, and corresponding anchor point sets for the most recent time periods. Preservation constraints are introduced for historically existing anchor points to maintain their relative topological relationships and reprojection stability. Simultaneously, the scale bias component is used as a window optimization variable in the solution, so that scale perturbations caused by interface scaling or theme changes are resolved within the window. The window optimization adopts an incremental update strategy, prioritizing the retention of constraints for high-confidence anchor points and reducing or eliminating low-confidence constraints, outputting the updated target 2D coordinates and orientation, as well as the current scale bias estimate.

[0066] For example, in some scenarios, players run from the city gate along the main road into the main square. Motion observations, constrained by the horizon and dynamic suppression, provide planar displacement and small-angle turning. The geometric mapping obtained from the minimap through anchor point matching maintains high consistency near the square, and the observation confidence consistently exceeds the upper threshold, indicating the system is in fusion mode. Upon entering a side path, the interface scaling changes, and the scale bias component is gradually corrected within the sliding window, resulting in a short-term decrease in confidence without triggering gating. Subsequently, the task panel briefly covers the minimap, causing the confidence to fall below the lower threshold. The system automatically switches to process-first mode and relies on motion observations to maintain recursion. After the cover is removed, the anchor points are regained, the confidence recovers, and the system switches back to fusion mode, with the position and scale bias components in the state variables updated synchronously.

[0067] This embodiment achieves online adjustment of uncertainty through state variable expansion and adaptive noise recalibration, ensures continuous recursion when observation is interrupted or inaccurate through observation gating and mode switching with hysteresis, and suppresses scale disturbances introduced by interface scaling through joint estimation of anchor point holding constraints and scale bias of sliding window, and finally outputs stable two-dimensional coordinates and orientation of target, providing a consistent unified coordinate system input for external rendering and path planning.

[0068] In some embodiments, position observations obtained from geometric mapping relationships are fused with motion observations to output the player's target two-dimensional coordinates and orientation in a unified coordinate system, including: A fusion cost function containing position residuals, orientation residuals, and integral residuals is constructed using position observations and motion observations; Anisotropic weights are determined based on anchor point topological consistency, the angle between the road skeleton direction and the direction of movement, and observation confidence, and a block diagonal weight matrix is ​​formed. The target's two-dimensional coordinates, orientation, and geometric mapping scale bias are incrementally solved within a sliding window, and anchor point preservation constraints are introduced to complete joint optimization, resulting in updated target two-dimensional coordinates and orientation.

[0069] Specifically, a fusion cost function is first constructed. This cost function consists of three types of residuals: first, position residuals, which measure the difference between the two-dimensional position in the state and the position observation obtained from geometric mapping; second, orientation residuals, which measure the difference between the orientation in the state and the reference orientation derived from orientation markers combined with motion observations; and third, integral residuals, which measure the inconsistency between the pose increment accumulated from motion observations within the sliding window and the transition relationship between adjacent states. These three types of residuals are associated with the state nodes at each time point within the window using a unified time base and are aggregated in a weighted manner to form the total cost.

[0070] Furthermore, anisotropic weights are determined and a block-diagonal weight matrix is ​​formed. The weights of the location residual blocks are adaptively set based on anchor point topological consistency, the angle between the road skeleton direction and the movement direction, and the observation confidence: when the angle between the skeleton direction and the movement direction is small, the weight along the skeleton tangent is greater than the normal weight, thus exhibiting anisotropy in position constraints; the weight of the orientation residual blocks changes with the consistency of the orientation markers, increasing the weight when the orientation on the minimap is stable and decreasing the weight when the orientation is unstable; the weight of the integral residual blocks is related to the temporal stability of motion observations, and is moderately increased when speed and angle fluctuations are large, in order to maintain the consistency of state transitions within the window. The three types of weights are normalized and assembled in a block-diagonal manner, so that different types of residuals do not interfere with each other but can jointly constrain the same state.

[0071] Furthermore, the target's 2D coordinates, orientation, and geometric mapping scale bias are incrementally solved within a sliding window, and anchor point preservation constraints are introduced to complete joint optimization. The sliding window covers the state variables and corresponding observations at several recent time points, sharing a slowly varying geometric mapping scale bias variable within the window; preservation constraints are applied to historically existing high-confidence anchor points to maintain their relative relationships and reprojection biases in the global topology map; outlier observations are weighted or eliminated using innovative thresholds and robust losses; the solution process converges according to the dual criteria of residual descent and parameter step size, and the termination condition is that the total residual descent is lower than the threshold or the number of iterations reaches the upper limit; after optimization, the state at the center of the window is used as the output to obtain the updated target 2D coordinates and orientation, and the current scale bias estimate and anchor point survival count are updated simultaneously.

[0072] For example, in some cases, players enter a side road from the main road and then turn back. The anchor points on the minimap and global topology are highly consistent on the main road section, and the angle between the road skeleton direction and the direction of movement remains within a small range for a long time. The position residual has a greater weight along the tangential direction than the normal direction. After entering the side road, the orientation marker fluctuates briefly, the weight of the orientation residual automatically decreases, and the weight of the integral residual is increased to maintain the stability of the state transition within the window. During the turnback, two new branch points are added and included in the set of high-confidence anchor points. The anchor point retention constraint causes the reprojection deviation of several nodes within the window to fall back to within the threshold. The final output target 2D coordinates and orientation remain temporally continuous in the three stages of turning in, turning back, and resuming straight travel, and the scale offset is gradually corrected as the interface is scaled.

[0073] This embodiment achieves adaptive fusion of position observation and motion observation by using a fusion cost that includes position residual, orientation residual and integral residual, anisotropic weights based on the consistency of road skeleton and anchor points, and joint incremental solution under sliding window and anchor point preservation constraints. It can still output continuous and consistent target two-dimensional coordinates and orientation in scenarios with interface changes and short-term unstable observations.

[0074] In some embodiments, the target 2D coordinates and orientation are used to generate a 2D map layer containing the current position and orientation in an external rendering module for display, including: The map viewport center, rotation angle, and scaling factor are determined based on the target's two-dimensional coordinates and orientation, and corrected by combining the geometric mapping scale offset. Based on the speed and road skeleton direction obtained from motion observation, the anisotropic field of view and tile level are determined, and tile pre-fetching and non-field of view tile elimination are performed along the predicted path direction. Based on a unified time base, the base map tile layer, road skeleton layer, anchor point and place name label layer, current location and orientation compass layer, and historical trajectory and target path layer are drawn sequentially using a double buffering method. Based on the observation confidence level, the visibility and refresh rate of the layer are switched between fusion mode and process priority mode, and a two-dimensional map layer is output for display.

[0075] Specifically, firstly, the map viewport center, rotation angle, and scaling factor are determined based on the target's 2D coordinates and orientation, and then corrected using the geometric mapping scale offset. The rendering module uses the target's 2D coordinates as the viewport center and the orientation as the reference for the viewport rotation angle, calculating the initial scaling factor according to the aspect ratio of the display window. The geometric mapping scale offset is then superimposed on the scaling factor as a scale correction to obtain the corrected viewport parameters. To suppress scaling jitter, a first-order low-pass constraint is applied to the scale offset, and the viewport rotation angle is updated gradually when the orientation change is small, ensuring geometric consistency of the viewport matrix across consecutive frames.

[0076] Subsequently, based on the velocity and road skeleton orientation obtained from motion observations, the anisotropic field of view and tile level are determined, and tile prefetching and non-field-of-view tile elimination are performed along the predicted path direction. The rendering module derives the short-term predicted path using the target's 2D coordinates and orientation, extending the field of view in the forward direction and moderately contracting it in the lateral direction to form a fan-shaped field of view mask that is wider in the front and narrower in the back. The tile level is adaptively selected based on the velocity magnitude; when the velocity increases, the level of distant tiles is reduced first, while maintaining the sharpness of near-field tiles. In tile management, prefetching is performed with the predicted path as the main axis. For tiles outside the mask boundary, the elimination order is determined by a combination of the least recently used strategy and distance penalty. A short-term hot cache is maintained for high-frequency hit areas, and near-field tiles are refreshed first when the scale offset changes to avoid sharpness mismatch.

[0077] Next, using a unified time base and a double-buffered approach, the base map tile layer, road skeleton layer, anchor point and place name label layer, current position and orientation compass layer, and historical trajectory and target path layer are drawn sequentially. The double-buffered back buffer completes the compositing of all layers within one frame before switching to the foreground display, avoiding flickering caused by partial layer updates. The layer drawing order is as follows: first, load the base map tiles matching the viewport and complete the stitching; then draw the road skeleton and branch points, with the skeleton line width adaptively adjusted according to the scaling factor; subsequently, overlay the anchor points and place name labels, with the label font size and occlusion priority adaptively adjusted according to viewport scaling and density; then draw the current position arrow and orientation compass on top, with the arrow direction consistent with the orientation and the size proportionally varying with the scaling factor; finally, draw the historical trajectory and target path, with the historical trajectory using time-decayed transparency, and the target path maintaining continuity in the predicted path direction. All layers are data retrieved and plotted using a unified time base, ensuring that the position, orientation, skeleton, and labels within the same frame are at the same time section.

[0078] Furthermore, based on the observation confidence level, the visibility and refresh frequency of the switching layers between fusion mode and process-first mode are adjusted, and a 2D map layer for display is output. When the observation confidence level is higher than the upper threshold, fusion mode is entered, the refresh frequency of anchor points and place name labels is increased, and loopback indicator markers are enabled; when the observation confidence level is lower than the lower threshold, process-first mode is entered, the refresh frequency of label layers and historical trajectory layers is reduced, and only the base map, road skeleton, and current location arrow are refreshed at high frequency, while weak observation indicators are displayed at the viewport edge; within the threshold range, the current mode is maintained to avoid frequent jitter. The mode state and threshold adopt a switching strategy with hysteresis and are recorded with a unified time base.

[0079] For example, in some examples, the display window resolution is 2560×1440, and the player is traveling north along the main road at a medium speed. The system uses the target's 2D coordinates as the viewport center and the orientation as the rotation angle, and corrects the scaling factor to 0.75 by combining the scale offset; it forms a forward-extended fan-shaped field of view based on the speed and road skeleton direction, pre-captures three columns of tiles along the predicted path direction, and discards low-hit tiles in the back; the background buffer completes the synthesis in the order of base map tile layer, road skeleton layer, anchor point and place name label layer, current position and orientation compass layer, and historical trajectory and target path layer, and switches the whole to the foreground display.

[0080] When the task panel briefly occludes the small map, causing the observation confidence to drop below the lower threshold, the rendering module automatically enters the process priority mode, reduces the refresh frequency of the annotation layer, and retains the high-frequency refresh of the main layer; after the occlusion is removed, the confidence recovers to above the upper threshold, and the system returns to the fusion mode and prioritizes refreshing tiles and annotations in the near area.

[0081] This embodiment enhances near-field clarity and forward readability while ensuring low latency and stable display through viewport correction involving scale offset, anisotropic field of view and tile management combining velocity and skeleton direction, double-buffered layered rendering with unified time base, and mode switching based on observation confidence. It also maintains usable navigation presentation in scenarios with unstable observation.

[0082] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0083] Figure 2 This is a schematic diagram of the structure of the real-time two-dimensional map positioning device for games based on screen observation provided in this application embodiment. Figure 2 As shown, the game real-time 2D map positioning device based on screen observation includes: The acquisition module 201 is used to capture the screen of the target game window under the authorization of the operating system to obtain an image sequence; The matching module 202 is used to locate the small map region from the image sequence and identify the orientation markers and feature points. It matches the orientation markers and feature points with the corresponding structures in the preset or online constructed global two-dimensional map to select anchor points and establish the geometric mapping relationship from the small map pixels to the unified coordinate system. Estimation module 203 is used to estimate the relative displacement and rotation angle between adjacent frames based on the main view frame in the image sequence to form motion observation; The fusion module 204 is used to construct an estimation model with position and orientation as state variables. It fuses position observations and motion observations obtained from geometric mapping relationships and outputs the player's target two-dimensional coordinates and orientation in a unified coordinate system. The generation module 205 is used to generate a two-dimensional map layer containing the current position and orientation in the external rendering module by using the target's two-dimensional coordinates and orientation, and then display it.

[0084] In some embodiments, Figure 2 The acquisition module 201 establishes a read-only pixel acquisition channel for the target game window without loading code into the process to which the target game window belongs or accessing the memory space of the process to which the target game window belongs; copies the frame buffer of the target game window and assigns a time identifier according to a unified time base; divides the copied result into the main view area and the small map candidate sub-area according to the interface layout, and sets the sampling frequency and resolution respectively to form a multi-channel image sequence.

[0085] In some embodiments, Figure 2 The acquisition module 201 generates an interface layout fingerprint from the copied results to obtain a candidate region set; polar coordinate unfolding and orientation marker voting detection are performed on the candidate regions in the candidate region set to determine the candidate sub-regions of the small map, and the remaining regions are taken as the main view regions; the unified sampling budget allocator adaptively determines the sampling frequency and resolution of the main view channel and the small map channel according to the change rate of the region content and the texture complexity, and adopts a dual threshold stabilization strategy to control parameter switching; the output is a multi-channel image sequence composed of the main view channel and the small map channel.

[0086] In some embodiments, Figure 2 The matching module 202 generates candidate windows for the small map in the image sequence based on the interface layout fingerprint and color texture prior; the candidate windows are expanded in polar coordinates and the orientation markers are determined by direction scale period matching and indicator voting; the target small map region is selected by cross-frame consistency scoring; the road texture is skeletonized in the target small map region to extract branch points, and the semantic anchor points obtained by boundary corner detection and place name text recognition are combined to determine the feature point set; the orientation markers and feature point set are filtered and output according to geometric consistency and temporal consistency.

[0087] In some embodiments, Figure 2 The matching module 202 performs tile indexing on the global two-dimensional map and generates a topological map of road skeletons and branch points; it determines the initial value of the rotation of the small map relative to the north based on the orientation identifier, and calculates the topological fingerprint of the feature points according to the adjacent angle sequence and connectivity; it establishes a corresponding candidate set on the topological map according to the consistency of the topological fingerprint and orientation, and performs compatibility screening using semantic anchors; it selects multiple non-collinear anchors from the candidate set, and uses robust estimation to solve the homography transformation matrix based on the pixel coordinates of the non-collinear anchors and the global map coordinates to obtain the geometric mapping relationship from the small map pixels to the unified coordinate system.

[0088] In some embodiments, Figure 2The estimation module 203 constructs a multi-scale pyramid for adjacent main view frames, extracts corner and edge features and calculates sparse matching, and simultaneously calculates dense optical flow; it generates a dynamic object suppression mask based on the optical flow consistency residual and appearance difference; it divides the main view frames into ground candidate areas and non-ground areas based on horizon estimation, and performs affine fitting on the matching interior points and optical flow interior points in the ground candidate areas to obtain the planar displacement vector; it performs phase correlation on adjacent frames to obtain the initial value of rotation, and uses the initial value of rotation to constrain the affine fitting to obtain the rotation angle; it uses the time consistency gating of the sliding window to filter the planar displacement vector and rotation angle, and outputs motion observations.

[0089] In some embodiments, Figure 2 The fusion module 204 uses two-dimensional position, orientation, and geometric mapping scale bias as state variables, establishes state recursion with motion observation as process input, and establishes observation equations with the position obtained from the geometric mapping relationship as the observation. It calculates the observation confidence based on anchor point topological residuals, orientation identifier consistency, and time consistency, and performs adaptive recalibration of observation noise and process noise based on the observation confidence. When the observation confidence is lower than the threshold, it implements observation gating and switches to process priority mode. After the observation confidence recovers, it switches back to fusion mode. It applies anchor point preservation constraints within the sliding window and jointly estimates the geometric mapping scale bias, outputting the target's two-dimensional coordinates and orientation.

[0090] In some embodiments, Figure 2 The fusion module 204 constructs a fusion cost function containing position residuals, orientation residuals, and integral residuals based on position observations and motion observations; it determines anisotropic weights and forms a block diagonal weight matrix based on anchor point topology consistency, the angle between the road skeleton direction and the motion direction, and the observation confidence; it incrementally solves the target's two-dimensional coordinates, orientation, and geometric mapping scale bias within a sliding window, and introduces anchor point preservation constraints to complete joint optimization, thereby obtaining the updated target's two-dimensional coordinates and orientation.

[0091] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0092] Figure 3 This is a schematic diagram of the structure of the electronic device 3 provided in an embodiment of this application. For example... Figure 3 As shown, the electronic device 3 of this embodiment includes a processor 301, a memory 302, and a computer program 303 stored in the memory 302 and executable on the processor 301. When the processor 301 executes the computer program 303, it implements the steps in the various method embodiments described above. Alternatively, when the processor 301 executes the computer program 303, it implements the functions of each module / unit in the various device embodiments described above.

[0093] For example, computer program 303 may be divided into one or more modules / units, which are stored in memory 302 and executed by processor 301 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of computer program 303 in electronic device 3.

[0094] Electronic device 3 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 3 may include, but is not limited to, processor 301 and memory 302. Those skilled in the art will understand that... Figure 3 This is merely an example of electronic device 3 and does not constitute a limitation on electronic device 3. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device may also include input / output devices, network access devices, buses, etc.

[0095] Processor 301 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0096] The memory 302 can be an internal storage unit of the electronic device 3, such as a hard disk or RAM. The memory 302 can also be an external storage device of the electronic device 3, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 302 can include both internal and external storage units of the electronic device 3. The memory 302 is used to store computer programs and other programs and data required by the electronic device. The memory 302 can also be used to temporarily store data that has been output or will be output.

[0097] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0098] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0099] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0100] In the embodiments provided in this application, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. Multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of apparatus or units may be electrical, mechanical, or other forms.

[0101] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0102] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0103] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium may include: any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0104] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although the technical solutions of this application are described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A real-time two-dimensional map positioning method for games based on screen observation, characterized in that, include: Under operating system authorization, screen capture is performed on the target game window to obtain an image sequence; Locate the small map region from the image sequence and identify directional markers and feature points. Match the directional markers and feature points with the corresponding structures in a preset or online constructed global two-dimensional map to select anchor points and establish a geometric mapping relationship from small map pixels to a unified coordinate system. Based on the main view frame in the image sequence, the relative displacement and rotation angle between adjacent frames are estimated to form motion observation; An estimation model with position and orientation as state variables is constructed, and the position observations obtained from the geometric mapping relationship are fused with the motion observations to output the player's target two-dimensional coordinates and orientation in the unified coordinate system; The target's two-dimensional coordinates and orientation are used to generate a two-dimensional map layer containing the current position and orientation in the external rendering module for display and use; The step of locating the minimap region from the image sequence and identifying location markers and feature points includes: Candidate windows for small maps are generated in the image sequence based on interface layout fingerprints and color texture priors. The candidate window is expanded in polar coordinates and the orientation marker is determined by direction scale period matching and indicator voting. The target small map area is selected by cross-frame consistency scoring. Within the target minimap area, the road texture is skeletonized to extract branch points, and the set of feature points is determined by combining the anchor points obtained from boundary corner detection and place name text recognition. The orientation markers and the set of feature points are filtered and output based on geometric consistency and temporal consistency. The step of matching the directional markers and feature points with corresponding structures in a pre-set or online-constructed global two-dimensional map to select anchor points and establish a geometric mapping relationship from small map pixels to a unified coordinate system includes: Perform tile indexing on the global 2D map and generate a topology map of road skeletons and branch points; The initial value of the minimap's rotation relative to north is determined based on the directional markers, and the topological fingerprint is calculated for the feature points according to the adjacent angle sequence and connectivity. On the topology map, a corresponding candidate set is established based on the consistency of the topology fingerprint and orientation, and compatibility screening is performed using anchor points; Multiple non-collinear anchor points are selected from the candidate set. Based on the pixel coordinates of the non-collinear anchor points and the global map coordinates, a robust estimation is used to solve the homography transformation matrix to obtain the geometric mapping relationship from the small map pixels to the unified coordinate system. The process of using the target's two-dimensional coordinates and orientation to generate a two-dimensional map layer containing the current position and orientation in an external rendering module for display includes: The map viewport center, rotation angle, and scaling factor are determined based on the target's two-dimensional coordinates and orientation, and corrected by combining the geometric mapping scale offset. Based on the speed and road skeleton direction obtained from motion observation, the anisotropic field of view and tile level are determined, and tile pre-fetching and non-field of view tile elimination are performed along the predicted path direction. Based on a unified time base, the base map tile layer, road skeleton layer, anchor point and place name label layer, current location and orientation compass layer, and historical trajectory and target path layer are drawn sequentially using a double buffering method. Based on the observation confidence level, the visibility and refresh rate of the layer are switched between fusion mode and process priority mode, and a two-dimensional map layer is output for display.

2. The method according to claim 1, characterized in that, The step of capturing the target game window's screen to obtain an image sequence under operating system authorization includes: A read-only pixel acquisition channel is established for the target game window without loading code into the process to which the target game window belongs or accessing the memory space of the process to which the target game window belongs. Copy the frame buffer of the target game window and assign time markers according to a uniform time base; Based on the interface layout, the copied results are divided into the main view area and the small map candidate sub-area, and the sampling frequency and resolution are set respectively to form a multi-channel image sequence.

3. The method according to claim 2, characterized in that, The step of dividing the copied result into a main view area and a small map candidate sub-region according to the interface layout, and setting the sampling frequency and resolution of each sub-region to form a multi-channel image sequence, includes: Generate an interface layout fingerprint from the copying results to obtain a set of candidate regions; Polar coordinate unfolding and orientation marker voting detection are performed on the candidate regions in the candidate region set to determine the candidate sub-regions of the small map, and the remaining regions are used as the main view regions. The unified sampling budget allocator adaptively determines the sampling frequency and resolution of the main view channel and the small map channel based on the rate of change of regional content and texture complexity, and adopts a dual threshold stabilization strategy to control parameter switching. Output a multi-channel image sequence consisting of the main view channel and the minimap channel.

4. The method according to claim 1, characterized in that, The step of estimating the relative displacement and rotation angle between adjacent frames based on the main view frame in the image sequence to form motion observation includes: A multi-scale pyramid is constructed for adjacent main view frames to extract corner and edge features and calculate sparse matching, while also calculating dense optical flow; A dynamic object suppression mask is generated based on the optical flow consistency residual and appearance difference. Based on the horizon estimation, the main view frame is divided into ground candidate area and non-ground area, and affine fitting is performed on the matching interior points and optical flow interior points in the ground candidate area to obtain the planar displacement vector. Phase correlation is performed on adjacent frames to obtain initial rotation values, and the affine fitting is constrained by the initial rotation values ​​to obtain the rotation angle. The planar displacement vector and the rotation angle are filtered using time-consistency gating with a sliding window to output motion observations.

5. The method according to claim 1, characterized in that, The construction of the estimation model with position and orientation as state variables includes: Two-dimensional position, orientation and geometric mapping scale offset are used as state variables, and the motion observation is used as process input to establish state recursion. The position obtained from the geometric mapping relationship is used as the observation to establish the observation equation. The observation confidence level is calculated based on the anchor point topological residual, azimuth identifier consistency and time consistency, and the observation noise and process noise are adaptively recalibrated based on the observation confidence level. When the observation confidence level is lower than the threshold, observation gating is implemented and the process priority mode is switched. After the observation confidence level is restored, the fusion mode is switched back. Anchor point holding constraints are applied within a sliding window, and the geometric mapping scale bias is jointly estimated to output the target's two-dimensional coordinates and orientation.

6. The method according to claim 5, characterized in that, The process of fusing the position observations obtained from the geometric mapping relationship with the motion observations to output the player's target two-dimensional coordinates and orientation in the unified coordinate system includes: A fusion cost function comprising position residual, orientation residual, and integral residual is constructed using the position observation and the motion observation; Anisotropic weights are determined and a block diagonal weight matrix is ​​formed based on anchor point topological consistency, the angle between the road skeleton direction and the direction of movement, and the observation confidence. Within a sliding window, the target's two-dimensional coordinates, orientation, and geometric mapping scale bias are incrementally solved, and anchor point retention constraints are introduced to complete joint optimization, resulting in updated target two-dimensional coordinates and orientation.

7. A real-time two-dimensional map positioning device for games based on screen observation, characterized in that, include: The acquisition module is used to capture the screen of the target game window to obtain an image sequence, provided that the operating system is authorized. The matching module is used to locate the small map area from the image sequence and identify the directional markers and feature points, match the directional markers and feature points with the corresponding structures in the preset or online constructed global two-dimensional map to select anchor points and establish the geometric mapping relationship from the small map pixels to the unified coordinate system. The estimation module is used to estimate the relative displacement and rotation angle between adjacent frames based on the main view frame in the image sequence to form motion observation; The fusion module is used to construct an estimation model with position and orientation as state variables, fuse the position observations obtained from the geometric mapping relationship with the motion observations, and output the player's target two-dimensional coordinates and orientation in the unified coordinate system; The generation module is used to use the target's two-dimensional coordinates and orientation to generate a two-dimensional map layer containing the current position and orientation in the external rendering module for display and call; The matching module is used to generate candidate windows for a small map in the image sequence based on the interface layout fingerprint and color texture prior; to expand the candidate windows in polar coordinates and determine the directional markers by using direction scale period matching and indicator voting; to select the target small map region by cross-frame consistency scoring; to skeletonize the road texture in the target small map region to extract branch points, and to determine the feature point set by combining the anchor points obtained by boundary corner detection and place name text recognition; and to filter and output the directional markers and the feature point set according to geometric consistency and temporal consistency. The matching module is also used to perform tile indexing on the global two-dimensional map and generate a topological map of road skeletons and branch points; determine the initial value of the rotation of the small map relative to north based on the azimuth identifier, calculate the topological fingerprint of the feature points according to the adjacent angle sequence and connectivity; establish a corresponding candidate set on the topological map according to the consistency of the topological fingerprint and azimuth, and perform compatibility screening using anchor points; select multiple non-collinear anchor points from the candidate set, and use robust estimation to solve the homography transformation matrix based on the pixel coordinates of the non-collinear anchor points and the global map coordinates to obtain the geometric mapping relationship from the small map pixels to the unified coordinate system; The generation module is used to determine the map viewport center, rotation angle, and scaling factor based on the target's 2D coordinates and orientation, and to perform corrections by combining geometric mapping scale bias; it determines the anisotropic field of view and tile level based on the speed and road skeleton direction obtained from motion observations, and performs tile prefetching and non-field-of-view tile elimination along the predicted path direction; it sequentially draws the base map tile layer, road skeleton layer, anchor point and place name label layer, current location and orientation compass layer, and historical trajectory and target path layer in a double-buffered manner according to a unified time base; it switches the visibility and refresh frequency of the layers between fusion mode and process priority mode according to the observation confidence, and outputs the 2D map layers for display.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images