Route guidance machine interaction system and method based on AI digital person

Abstract

This invention discloses a wayfinding interaction system and method based on an AI digital human, belonging to the field of artificial intelligence human-computer interaction technology. The system includes: collecting on-screen input events and normalizing them into an event stream; establishing reference lines; dynamically determining the user's physical location and gaze area under environmental perception; dividing the area from the starting point to the target location into semantic zones based on the reference lines, the user's physical location, and the gaze area, arranging them into an ordered semantic zone sequence; binding on-screen indicators and several candidate target areas to each semantic zone; dynamically sorting the candidate target areas; and establishing an interaction log. The AI ​​digital human broadcasts information according to the ordered semantic zone sequence. When a semantic zone name is broadcast, the on-screen indicator and candidate target area are highlighted, and the system determines whether the user's gaze remains on the first candidate target area within a fixed confirmation period. This maintains the continuity and logical consistency of path expression in complex spaces, improving the accuracy of human-computer interaction.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence human-computer interaction technology, and in particular to a wayfinding interaction system and method based on AI digital humans. Background Technology

[0002] With the rapid development of artificial intelligence and human-computer interaction technologies, interactive devices based on voice recognition, gesture recognition, and visual perception have been widely used in public navigation, exhibition hall guidance, and commercial guidance scenarios. Traditional wayfinding devices mostly adopt touch input or fixed voice broadcast modes, providing users with directional prompts through preset routes and map interfaces. In recent years, wayfinding interaction methods combining computer vision and depth perception have begun to emerge. These methods can collect users' spatial position and line of sight information through cameras, achieving a more immersive interactive experience. AI-driven AI digital human technology has further promoted this development, enabling the wayfinding process to have semantic understanding and contextual response capabilities. AI digital humans can adjust the content of the speech, visual direction, and feedback actions according to user input and status, completing route broadcasting, semantic prompts, and multi-round confirmation interactions in complex environments, significantly improving the naturalness and continuity of human-computer communication.

[0003] However, conventional wayfinding interaction methods still have limitations in maintaining path consistency and utilizing user behavior feedback in dynamic environments. Although traditional methods can record interaction events, they mostly remain at the input response level and lack the ability to reconstruct the real semantic path from behavior logs. They cannot determine the deviation between the actual guidance path and the preset path. After semantic zone misalignment, misselection of candidate target area, or user rollback operation, existing methods are unable to achieve path self-correction and instruction adaptive adjustment, resulting in AI digital human guidance lacking self-learning ability and self-consistency update mechanism. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a wayfinding interaction method based on AI digital humans to solve the problems of semantic path deviation not being recognized and interactive feedback not being self-correcting during the wayfinding process.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides a wayfinding interaction method based on an AI digital human, comprising,

[0008] Collect on-screen input events and normalize them into an event stream, establish reference lines, and dynamically determine the user's physical location and line of sight under environmental awareness;

[0009] Based on reference lines, user physical location, and line of sight area, the starting point to the target location is divided into semantic zones and arranged into an ordered semantic zone sequence. Each semantic zone is bound to an on-screen indicator and several candidate target areas on the screen. The candidate target areas are dynamically sorted and an interaction log is established.

[0010] The AI ​​digital human broadcasts according to an ordered sequence of semantic zones. When the name of a semantic zone is broadcast, the indicator and candidate target area on the screen are highlighted. The AI ​​digital human also determines whether the user’s gaze is fixed on the candidate target area at the top of the list within a fixed confirmation period.

[0011] The observed semantic zone sequence is reconstructed based on the interactive logs and then checked for consistency with the ordered semantic zone sequence.

[0012] As a preferred embodiment of the AI-based digital human-based wayfinding interaction method of the present invention, the steps of collecting and normalizing on-screen input events into an event stream and establishing reference lines are as follows:

[0013] The navigation device uniformly records the atomic inputs generated in the screen side area and the screen front side area. Each atomic input is written as an event record, and the event records are strictly ordered by time to form an event stream.

[0014] By using the pair of points formed by the calibration points laid on the ground and the pixels on the guide camera screen, the geometric mapping relationship between the screen plane and the ground reference plane is obtained. The upper and lower pixels of the vertical center line of the screen are taken and converted into actual points on the ground reference plane. The line connecting the actual points is used as the reference line, and the direction of the line connecting the actual points is used as the direction of the reference line.

[0015] As a preferred embodiment of the AI-based digital human-based wayfinding interaction method of the present invention, the specific steps of dynamically determining the user's physical location and line of sight area under environmental perception are as follows:

[0016] The system obtains the user's physical location using the depth imaging camera of the wayfinding device, acquires the user's head orientation based on the user's head posture, and converts the user's head orientation to a ground reference coordinate system consistent with the reference line.

[0017] Compare the angle between the user's head orientation and the reference line direction. When the angle does not exceed the forward angle threshold, the user's gaze is considered to be facing forward of the reference line. When the angle falls into the left allowable range, the user's gaze is considered to be pointing to the left. When it falls into the right allowable range, the user's gaze is considered to be pointing to the right. When the angle exceeds both the left and right allowable ranges and no interaction occurs within a fixed time, the interaction is considered to have ended.

[0018] The system measures the nearest lateral distance from the user's physical location to the reference line. When the nearest lateral distance does not exceed the forward bandwidth threshold, the user is considered to be within the effective interaction area of ​​the navigation system. When the nearest lateral distance exceeds the forward bandwidth threshold and no interaction is performed within a fixed time, the user is considered to have deviated from the effective interaction area of ​​the navigation system.

[0019] When the included angle does not exceed the forward angle threshold and the lateral nearest distance does not exceed the forward bandwidth threshold, the current line of sight area is classified as the reference line's forward dominant area. When the included angle is within the left allowable range and the lateral nearest distance does not exceed the forward bandwidth threshold, the current line of sight area is classified as the left adjacent area. When the included angle is within the right allowable range and the lateral nearest distance does not exceed the forward bandwidth threshold, the current line of sight area is classified as the right adjacent area.

[0020] As a preferred embodiment of the AI-based digital human-based wayfinding interaction method of the present invention, the specific steps of dividing the starting point to the target location into semantic zones and arranging them into an ordered semantic zone sequence are as follows:

[0021] Based on the user's physical location, the user's current location is determined as the starting point. The preset path is read from the site diagram built into the navigation device. The reference line direction is used as the benchmark. The topology nodes are identified sequentially along the preset path, and the continuous road segments between adjacent topology nodes are used as semantic zones.

[0022] Starting from the semantic zone of the starting point and ending at the semantic zone of the target location, an ordered sequence of semantic zones is obtained.

[0023] As a preferred embodiment of the AI-based digital human-based wayfinding interaction method of the present invention, the steps of binding on-screen indicators and several candidate target regions on the screen to each semantic zone, and dynamically sorting the candidate target regions, are as follows:

[0024] Record directional descriptions in each semantic zone and align arrows, annotation lines, and AI digital human speech anchors with the directional descriptions of the semantic zones.

[0025] The target regions located within the geometric range of the current semantic zone, as well as the target regions in the next semantic zone that intersect or are tangent to the boundary of the current semantic zone, are merged into a set of candidate target regions for the semantic zone.

[0026] Whenever a valid record is added to the event stream, the set of candidate target regions for the current semantic zone and the next adjacent semantic zone is rearranged in the following order: line-of-sight consistency priority, zone consistency priority, event proximity priority, proximity along the reference line priority, and turning complexity priority.

[0027] As a preferred embodiment of the AI-based digital human-based wayfinding interaction method of the present invention, wherein: when the semantic zone name is broadcast, the indicator and candidate target area on the screen are highlighted, the specific steps are as follows:

[0028] When the broadcast enters the semantic zone name field, the arrow describing the direction of the current semantic zone is immediately highlighted on the screen, and the first candidate target area icon in the sorting is also highlighted.

[0029] As a preferred embodiment of the AI-based digital human-based wayfinding interaction method of the present invention, the specific steps of determining whether the user's gaze remains on the top-ranked candidate target area within a fixed confirmation period are as follows:

[0030] During the fixed confirmation period, the user's gaze points are sampled at equal intervals and assigned to the candidate target areas marked on the screen. The cumulative dwell time and continuity of each candidate target area covered by the gaze points are continuously counted. If the first-ranked candidate target area maintains the first cumulative dwell time and is not interrupted during the fixed confirmation period, it is considered that the gaze confirmation is passed; otherwise, it is considered that the gaze confirmation is not passed.

[0031] If the visual confirmation fails, a confirmation control will appear around the first candidate target area in the sorting, and a back control will appear on the side of the screen.

[0032] If the line of sight is confirmed to be clear, the current semantic zone is marked as confirmed. The AI ​​digital human continues to announce the name and action statements of the next semantic zone, repeating until the target location is reached.

[0033] As a preferred embodiment of the AI-based digital human wayfinding interaction method of the present invention, the specific steps of reconstructing the observed semantic zone sequence based on the interaction log are as follows:

[0034] By reading the operation category records in the interaction log, semantic zone identifiers are extracted and concatenated into an observation semantic zone sequence in chronological order.

[0035] As a preferred embodiment of the AI-based digital human-based wayfinding interaction method of the present invention, the specific steps of performing consistency verification with the ordered semantic zone sequence are as follows:

[0036] By aligning the observed semantic zone sequence with the ordered semantic zone sequence using a table with the minimum number of steps in dynamic programming, the system calculates the minimum number of difference steps and locates the earliest misaligned semantic zone, and provides one-click correction and re-guidance on the site map.

[0037] Secondly, this invention provides a wayfinding interaction system based on an AI digital human, comprising:

[0038] The data acquisition and judgment module collects input events in front of the screen and normalizes them into an event stream, establishes reference lines, and dynamically judges the user's physical position and line of sight area under environmental awareness.

[0039] The semantic zone module divides the area from the starting point to the target location into semantic zones based on reference lines, user physical location, and line of sight. These semantic zones are arranged into an ordered sequence. Each semantic zone is bound to an on-screen indicator and several candidate target areas on the screen. The candidate target areas are dynamically sorted, and an interaction log is established.

[0040] The AI ​​digital human broadcasting module broadcasts information according to an ordered semantic zone sequence. When the semantic zone name is broadcast, the indicator on the screen and the candidate target area on the screen are highlighted. Within a fixed confirmation period, it is determined whether the user's gaze is fixed on the candidate target area ranked first.

[0041] The reconstruction and verification module reconstructs the observed semantic zone sequence based on the interaction log and performs a consistency check with the ordered semantic zone sequence.

[0042] The beneficial effects of this invention are as follows: by dividing semantic zones and arranging them into an ordered semantic zone sequence, semantic modeling and structured expression of the path are realized, enabling the AI ​​digital human to perform accurate broadcasting and interactive control, thereby maintaining the continuity and logical consistency of the path expression in complex spaces. Through consistency verification, real-time misalignment identification and automatic correction are realized in the navigation process. The AI ​​digital human performs self-correction and self-learning, ensuring the stability and intelligent adaptability of the wayfinding process, and improving the accuracy and continuous guidance capability of human-computer interaction. Attached Figure Description

[0043] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a flowchart of a wayfinding interaction method based on AI digital humans.

[0045] Figure 2 This is a schematic diagram of a wayfinding interaction system based on AI digital humans.

[0046] Figure 3 Create a flowchart for the reference lines.

[0047] Figure 4 Flowchart for dynamically determining the user's line of sight area. Detailed Implementation

[0048] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0049] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0050] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0051] Reference Figures 1-4 As one embodiment of the present invention, this embodiment provides a wayfinding interaction method based on an AI digital human, comprising the following steps:

[0052] S1. Collect on-screen input events and normalize them into an event stream, establish reference lines, and dynamically determine the user's physical location and line of sight area under environmental awareness.

[0053] The navigation machine's data acquisition starts immediately upon the beginning of the interaction, uniformly recording atomic inputs generated in the side area and the front area of ​​the screen.

[0054] Atomic input refers to the smallest and indivisible input event in the human-computer interaction of a wayfinding device, including touch press, touch release, selection confirmation, back trigger, etc.

[0055] Each atomic input is written as an event record. The event record includes the event time, event type (screen side or front side), screen coordinates or spatial location associated with the atomic input, and a measure of strength or importance (such as touch pressure or dwell time). All event records are strictly ordered by time to form an event stream, which is continuously added throughout the interaction.

[0056] When the guide camera is deployed for the first time, a one-time calibration is performed. By using the four corner pixels of the guide camera screen corresponding to the four calibration points on the ground, the pixels of the guide camera screen and the ground calibration points are paired to obtain the geometric mapping relationship from the screen plane to the ground reference plane. The upper and lower two pixels of the vertical center line of the screen are taken and converted into two actual points on the ground reference plane. The line connecting the two actual points is the reference line, and the direction of the line connecting the two actual points is the direction of the reference line.

[0057] The depth imaging camera above the navigation device will collect data in real time from the area in front of the screen, locate the user in the area in front of the screen, obtain the three-dimensional position of the user's head center and torso center, and orthogonally project the head center onto the ground reference plane along the vertical direction to obtain the user's physical position.

[0058] After obtaining the user's physical location, the user's head orientation is obtained based on the user's head posture, and the user's head orientation is converted to a ground reference coordinate system consistent with the reference line.

[0059] Compare the angle between the user's head orientation and the direction of the reference line. If the angle does not exceed the forward angle threshold, the user's gaze is considered to be roughly facing the front of the reference line. If the angle falls into the left allowable range, the user's gaze is considered to be pointing to the left. If it falls into the right allowable range, the user's gaze is considered to be pointing to the right. If the angle exceeds the left and right allowable ranges and no interaction is performed within a fixed time (e.g., touch, select, enter the left and right allowable ranges, etc.), the interaction is considered to have ended.

[0060] The forward angle threshold is preset to 20°. Under typical interactive geometry, the half visual angle from the screen centerline to the screen edge is about 14° to 27° (determined by the ratio of screen width to interactive distance). 20° is located in the upper-middle part of the range, covering the central gaze band of common screen viewing, venue viewing, and screen return, and also leaving a clear boundary for the left and right allowable intervals. Moving the boundary to 19° will be too narrow in the forward direction and trigger the left and right allowable intervals too early. Moving it to 21° will be too wide in the forward direction and delay the triggering of the left and right allowable intervals. Head orientation estimation in public scenes is usually on the order of 3° to 5° error. 20° provides a margin of several times the error for forward gaze, reducing the probability of misjudging forward gaze as lateral gaze. 19° reduces the margin, and edge samples are more likely to be misjudged as lateral gaze. 21° retains some obviously lateral samples in the forward direction, reducing the direction discrimination.

[0061] Both the left and right allowable intervals are greater than 20° and no more than 45°. With central gaze and slight head movement, stable reading is generally possible within 30°. However, beyond 40°, users usually need to turn their torso, causing the screen to fall into the far peripheral field of vision. It is difficult to see and understand the same words and gestures at the same time. Setting the upper boundary of the allowable interval to 45° covers the actual side view while avoiding treating large side turns or back views as valid interactions. If the upper limit of the allowable interval exceeds 45°, it will treat postures that are no longer suitable for in-screen guidance as valid side views, which is prone to orientation jitter and misclassification.

[0062] The system measures the nearest lateral distance from the user's physical location to the reference line. If the nearest lateral distance does not exceed the forward bandwidth threshold, the user is considered to be within the effective interaction area of ​​the navigation device. If the nearest lateral distance exceeds the forward bandwidth threshold and no interaction occurs within a fixed time, the user is considered to have deviated from the effective interaction area of ​​the navigation device.

[0063] The forward bandwidth threshold is preset to 0.8 meters. A typical adult's shoulder width is approximately 0.45 meters. There is a natural left-right sway of about 0.1 to 0.15 meters when stationary, plus a safety margin of about 0.2 meters, totaling approximately 0.8 meters. This covers most normal standing positions and slight sideways movements, preventing misinterpretation of small natural displacements as leaving the effective area. Common interaction distances are between 0.8 and 1.5 meters. Even if the user shifts laterally to 0.8 meters, as long as their head orientation remains within a 20° forward angle, the on-screen arrows, highlights, and speech anchors remain within a stable reading range. However, when the lateral shift increases further, the user often needs to turn or move significantly, making forward judgment and confirmation prone to instability. The effective field of view of the depth imaging camera above the screen in public scenes typically covers... Coverage bandwidth: Within 0.8 meters, key points of the head and torso are less obstructed by screen edges and bystanders. Further widening the bandwidth, such as to 0.9 meters or more, makes it easier to include passersby on the side in the same bandwidth, increasing false associations and sorting jitter. 0.8 meters, as the upper limit of effective bandwidth for a single person, can form a distinguishable zone with the common pedestrian spacing in corridors or lobbies, reducing the probability of misjudging adjacent users into the same effective zone and ensuring that the sorting based on line-of-sight consistency is not disturbed by brief stops on the side. 0.7 meters is too strict for users with wide shoulders or those carrying items, and normal slight movements are easily judged as deviations from the bandwidth, leading to unnecessary backtracking and reconfirmation. 0.9 meters still includes obvious lateral movement scenarios in the effective bandwidth, making it easier to treat passersby on the side or those standing across displacement positions as effective users, increasing the risk of false advances.

[0064] When the included angle does not exceed the forward angle threshold and the lateral nearest distance does not exceed the forward bandwidth threshold, the current line of sight area is classified as the reference line's forward dominant area. When the included angle is within the left allowable range and the lateral nearest distance does not exceed the forward bandwidth threshold, the current line of sight area is classified as the left adjacent area. When the included angle is within the right allowable range and the lateral nearest distance does not exceed the forward bandwidth threshold, the current line of sight area is classified as the right adjacent area.

[0065] S2. Based on the reference line, user's physical location, and line of sight area, the starting point to the target location is divided into semantic zones and arranged into an ordered semantic zone sequence. Each semantic zone is bound to an on-screen indicator and several candidate target areas on the screen. The candidate target areas are dynamically sorted, and an interaction log is established.

[0066] Based on the user's physical location, the system determines the user's current location as the starting point, reads the preset path from the built-in site diagram of the navigation device, uses the reference line direction as a benchmark, and sequentially identifies topological nodes (such as entrances, passage corners, stairwells, doorways, escalator landings, and other structural locations that can cause changes in the direction of travel or floor level; in open spaces without obvious geometric transitions, functional points are used as temporary nodes). The continuous road segments between adjacent topological nodes are defined as a semantic zone, starting from the semantic zone where the starting point is located and ending at the semantic zone where the target location is located, to obtain an ordered sequence of semantic zones.

[0067] The preset path refers to a fixed path from the starting point to the target location that is pre-set in the site diagram embedded in the navigation device.

[0068] Record a direction description in each semantic zone. The direction description is derived from the turning relationship of the preset path at the starting position of the semantic zone. If the route is in the same direction as the reference line, it is marked as "keeping forward". If the route is turning left relative to the reference line, it is marked as "turning left". If the route is turning right relative to the reference line, it is marked as "turning right".

[0069] Align the arrows, annotation lines, and AI digital human speech anchors with the directional descriptions of the semantic zone. If the directional description of the semantic zone is to keep forward, the arrows will point outward along the reference line. If the directional description of the semantic zone is to turn left or right, a turning or entering prompt will be given in advance at the node where the current semantic zone enters the next semantic zone.

[0070] Arrows are directional symbols on the screen, pointing in the same direction as the semantic zone. When they encounter a turning semantic zone, they turn in the next direction at the turning point.

[0071] Annotation lines refer to thin lines or path lines on the screen that connect the current location to the candidate target area icon.

[0072] AI digital human speech anchor points refer to the on-screen points or object markers that indicate where the AI ​​digital human looks, points, and speaks.

[0073] The target regions located within the geometric range of the semantic zone and the target regions in the next semantic zone that intersect or are tangent to the boundary of the semantic zone are merged into a candidate target region set for the semantic zone. To avoid the candidate target region set being too wide, distant target regions that are not in the current semantic zone and are not immediately adjacent to the next semantic zone are not included. Each candidate target region in the candidate target region set is supplemented with a lateral attribute. Based on the reference line, the representative point of the candidate target region on the site diagram is taken (by default, the geometric center of the candidate target region is taken, and irregular shapes are replaced by the nearest point of the passable path). If the representative point falls on the left half-plane relative to the reference line, it is marked as left. If the representative point falls on the right half-plane relative to the reference line, it is marked as right. If the representative point falls on the reference line or points to the target position in the same direction as the reference line, it is marked as forward. If the candidate target region spans both the left and right half-planes, the half-plane where the representative point is located is taken as the reference.

[0074] Whenever a valid record is added to the event stream or the current line of sight changes, the set of candidate target regions for the current semantic zone and the next adjacent semantic zone is rearranged in the following order: line of sight consistency priority, zone consistency priority, event proximity priority, proximity along the reference line priority, and turning complexity priority.

[0075] Prioritizing line-of-sight consistency means that if the current line-of-sight area is the dominant area in front of the reference line, then candidate target areas with a forward lateral attribute are ranked before all candidate target areas. If the current line-of-sight area is the adjacent area on the left, then candidate target areas with a left lateral attribute are ranked first. If it is the adjacent area on the right, then candidate target areas with a right lateral attribute are ranked first.

[0076] Zone consistency priority means that candidate target regions located in the current semantic zone are ranked before candidate target regions located in the next semantic zone.

[0077] The "most recently associated event priority" means that if the most recent valid touch or selection in the event stream is associated with any on-screen indication of the candidate target area (icon hit, voice anchor hit, annotation line hit, highlight coverage hit, and target surface hit), then the current candidate target area will be moved up within the candidate target area set.

[0078] "Prioritizing proximity along the reference line" means that, when viewed from the direction of the reference line, the closer the user's physical location is to the representative point of the candidate target area on the site diagram along the reference line, the higher the priority will be.

[0079] Turning complexity priority means prioritizing the number of turns required to get from the current semantic region to the candidate target region. If they are still tied, the one with fewer consecutive turns is prioritized.

[0080] After dynamic sorting, if the items are still completely parallel, the original relative order of the parallel items is maintained to reduce screen jitter. When there are no candidate target areas in the field of view, the candidate target area in the next semantic zone that is consistent with the direction description of the current semantic zone is given priority. If there are still no candidate target areas, the previous sorting and on-screen indication remain unchanged, and the sorting is rearranged after the event stream produces new valid records or a stable field of view area is obtained again.

[0081] Construct an interaction log, which is an append-only structure. Write the event time and event type, the reason for triggering the write, the current semantic zone identifier and direction description, the current line of sight area, the candidate target area identifier and the rearrangement order, and the projection of the user's physical location on the site diagram. The write timing includes the appending of valid records to the event stream, changes in the current line of sight area, and switching of semantic zones.

[0082] S3, the AI ​​digital human broadcasts according to the ordered semantic zone sequence. When the semantic zone name is broadcast, the indicator on the screen and the candidate target area on the screen are highlighted, and the user's gaze is judged to stay on the candidate target area ranked first within a fixed confirmation period.

[0083] Extract the current semantic zone and candidate target region set from the ordered semantic zone sequence, read the directional description, arrows, annotation lines and AI digital human speech anchors of the current semantic zone, and read the dynamic sorting of the candidate target regions.

[0084] The AI ​​digital human begins to announce the name of the current semantic zone and the action statement (e.g., walk straight ahead to the corner of the corridor, then turn left to enter the service area). When the name of the semantic zone is announced, the arrow describing the direction of the current semantic zone is immediately highlighted on the screen, and a prompt to enter is marked at the end of the current semantic zone. At the same time, the first candidate target area icon is highlighted, and a line is drawn from the current position to the candidate target area icon, placing the AI ​​digital human's speech anchor point on or next to the candidate target area icon, forming a visual and voice synchronization effect of speaking and pointing simultaneously. Only the first candidate target area is highlighted on the screen, while the other candidate target areas are presented in a secondary style that is always lit and does not flicker, to avoid screen shaking.

[0085] After entering the fixed confirmation period, the user's gaze points are sampled at equal intervals and assigned to the candidate target areas marked on the screen. Throughout the fixed confirmation period, the cumulative dwell time and continuity of each candidate target area covered by the gaze points are continuously counted. If the first-ranked candidate target area maintains the first cumulative dwell time throughout the fixed confirmation period and is not interrupted by the continuous dwell time of other candidate target areas, it is considered that the gaze confirmation has passed; otherwise, it is considered that the gaze confirmation has failed.

[0086] With a fixed confirmation period of 2 seconds, the mainstream in-screen gaze collection and display refresh rate is 30 to 60 times per second. In a 2-second period, 60 to 120 sample points can be obtained, resulting in smaller statistical fluctuations and more stable results. Even at lower frequencies, sufficient samples can be maintained.

[0087] If the gaze confirmation fails, the on-screen indicator and highlight status remain unchanged, and the semantic zone is not advanced. A selection confirmation control is displayed around the highlighted first candidate target area in the sorting, and a back control is displayed on one side of the screen for the user to return to the previous semantic zone or re-trigger the display. Once any candidate target area is selected, the selected candidate target area is immediately moved to the first position in the sorting within the current semantic zone and is considered as explicitly confirmed. If the back control is triggered, the back operation is recorded and the user returns to the previous semantic zone according to the ordered semantic zone sequence, and the broadcast and gaze dwell determination are performed again.

[0088] If the visual confirmation or explicit confirmation is successful, the current semantic zone is marked as confirmed. The AI ​​digital human continues to announce the name and action statement of the next semantic zone. The on-screen indicator simultaneously advances the arrow to the starting position of the next semantic zone and presents the next entry prompt in advance. Based on the dynamic sorting of candidate target areas, the set of candidate target areas in the next semantic zone is reordered, and the first candidate target area in the sorting is highlighted on the screen. The fixed confirmation period is entered again, and the cycle continues until the semantic zone where the target location is located is reached.

[0089] Throughout the broadcast process, the following information is recorded in the interaction log: time stamp and operation category (e.g., start broadcast, enter confirmation observation, explicit selection confirmation and rollback), current semantic zone identifier and direction description, on-screen indicator element identifier associated with the current semantic zone (e.g., arrow, annotation line, AI digital human speech anchor point), candidate target area identifier, pre-rearrangement order and post-rearrangement order, rearrangement trigger reason (adding valid records to the event stream or change in the current gaze area), cumulative dwell time and continuity of the first candidate target area in the confirmation process, whether selection confirmation and rollback operations occur, user physical location point and current gaze area values ​​at the time of writing, fixed confirmation period start and end time and whether it is prematurely terminated due to explicit operation.

[0090] S4. Reconstruct the observed semantic zone sequence based on the interactive logs and perform consistency verification with the ordered semantic zone sequence.

[0091] During the session holding period indicated on the screen, records marked as confirmed, advanced, and rolled back in the interaction log are read in chronological order. Semantic zone identifiers are extracted one by one, concatenated into an observation semantic zone sequence, and the candidate target region actually used is saved for each semantic zone in the observation semantic zone sequence.

[0092] "Confirmed" refers to the moment when the fixed confirmation period ends and the line of sight is confirmed, or the moment when the user selects and confirms any candidate target area.

[0093] "Advancement" refers to the moment when the AI ​​digital human begins to announce the name of the next semantic zone, the arrow on the screen advances to the starting position of the next semantic zone, and the first candidate target area in the new sorting is highlighted.

[0094] Back means that the current semantic zone is paused and the user returns to the previous semantic zone to resume the broadcast and confirmation the moment the user clicks the back control.

[0095] The observed semantic zone sequence refers to the path that was actually confirmed in this on-screen indication session.

[0096] To align the ordered semantic zone sequence with the observed semantic zone sequence using the minimum number of steps, construct a table with one more row and one more column than the ordered semantic zone sequence. Fill in the table cell by cell from the top left to the bottom right. Each cell represents the minimum number of steps required to align the first few semantic zones of the ordered semantic zone sequence with the first few semantic zones of the observed semantic zone sequence. Start from zero in the first row and the first column and increment by one for each cell. For any cell in the table, take the cumulative steps of the three arrival methods and select the minimum one to write. Align the current prefix of the observed semantic zone sequence with the previous semantic zone of the ordered semantic zone sequence, and increment the cumulative steps by one from the cell above. Align the previous semantic zone of the observed semantic zone sequence with the current prefix of the ordered semantic zone sequence, and increment the cumulative steps by one from the cell to the left. If the two semantic zone identifiers are the same, carry the cumulative steps down from the original value in the top left cell. If the two semantic zone identifiers are different, increment the cumulative steps by one from the top left cell.

[0097] The current prefix refers to a subsequence that extends from the beginning of the semantic zone sequence to the position represented by the current cell.

[0098] The cell value in the lower right corner is the minimum number of difference steps between the ordered semantic zone sequence and the observed semantic zone sequence. By tracing back from the lower right corner to the upper left corner according to the table, the first occurrence of insertion, deletion, or substitution on the tracing path is the earliest misaligned semantic zone. If the lower right corner is zero, it means that the ordered semantic zone sequence and the observed semantic zone sequence are completely consistent.

[0099] On the site diagram, two different line types are used to depict the earliest misaligned semantic zone of the ordered semantic zone sequence and the earliest misaligned semantic zone of the observed semantic zone sequence, respectively. A one-click correction button is provided at the starting point of the earliest misaligned semantic zone of the ordered semantic zone sequence. After clicking, the on-screen indicator and candidate target area are re-bound. A re-guidance button is provided at the starting point of the earliest misaligned semantic zone of the observed semantic zone sequence, which is taken as the current semantic zone and the process is restarted.

[0100] Within the semantic zone identifier and direction description related to the earliest misaligned semantic zone, a constraint is recorded: the candidate target region actually used in this round should be ranked before the first candidate target region in this round of sorting.

[0101] The next time a candidate target region with the same semantic zone identifier and direction description enters the dynamic sorting, the system first checks whether there is a corresponding constraint. If there is, the recorded candidate target region is placed at the front of the candidate target region set, and the remaining candidate target regions remain in their original order before dynamic sorting is performed.

[0102] If multiple constraints conflict, the most recent coverage takes effect. If there are still cases where they cannot be satisfied simultaneously, the constraint with the most recent time will be satisfied first, and the remaining candidate target regions that are not satisfied will maintain their original relative order.

[0103] When the ordered semantic zone sequence and the observed semantic zone sequence are consistent, or are consistent again after one-click correction and reguidance, the semantic zone that has been passed will be displayed on the screen with a faded trajectory, the next semantic zone will be highlighted, the arrow will advance to the starting point of the next semantic zone, the annotation line and the AI ​​digital human speech anchor point will be updated synchronously to the first candidate target area of ​​the new sorting, and the verification completion will be written in the interaction log. If the next semantic zone exists, the loop will continue. If the current semantic zone is the semantic zone where the target location is located, an arrival prompt will be displayed, and the current on-screen instruction session will end.

[0104] This embodiment also provides a wayfinding interaction system based on AI digital human, including:

[0105] The data acquisition and judgment module collects input events in front of the screen and normalizes them into an event stream, establishes reference lines, and dynamically judges the user's physical position and line of sight area under environmental awareness.

[0106] The semantic zone module divides the area from the starting point to the target location into semantic zones based on reference lines, user physical location, and line of sight. These semantic zones are arranged into an ordered sequence. Each semantic zone is bound to an on-screen indicator and several candidate target areas on the screen. The candidate target areas are dynamically sorted, and an interaction log is established.

[0107] The AI ​​digital human broadcasting module broadcasts information according to an ordered semantic zone sequence. When the semantic zone name is broadcast, the indicator on the screen and the candidate target area on the screen are highlighted. Within a fixed confirmation period, it is determined whether the user's gaze is fixed on the candidate target area ranked first.

[0108] The reconstruction and verification module reconstructs the observed semantic zone sequence based on the interaction log and performs a consistency check with the ordered semantic zone sequence.

[0109] In summary, this invention achieves semantic modeling and structured expression of paths by dividing semantic zones and arranging them into ordered semantic zone sequences. This enables AI digital humans to perform accurate broadcasting and interactive control, thereby maintaining the continuity and logical consistency of path expression in complex spaces. Through consistency verification, real-time misalignment identification and automatic correction are achieved during navigation. The AI ​​digital human performs self-correction and self-learning, ensuring the stability and intelligent adaptability of the wayfinding process, and improving the accuracy and continuous guidance capability of human-computer interaction.

[0110] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A wayfinding interaction method based on AI digital human, characterized in that: include, Collect on-screen input events and normalize them into an event stream, establish reference lines, and dynamically determine the user's physical location and line of sight under environmental awareness; Based on reference lines, user physical location, and line of sight area, the starting point to the target location is divided into semantic zones and arranged into an ordered semantic zone sequence. Each semantic zone is bound to an on-screen indicator and several candidate target areas on the screen. The candidate target areas are dynamically sorted and an interaction log is established. The AI ​​digital human broadcasts according to an ordered sequence of semantic zones. When the name of a semantic zone is broadcast, the indicator and candidate target area on the screen are highlighted. The AI ​​digital human also determines whether the user’s gaze is fixed on the candidate target area at the top of the list within a fixed confirmation period. The observed semantic zone sequence is reconstructed based on the interactive logs and its consistency is checked with the ordered semantic zone sequence. The specific steps for dividing the starting point to the target location into semantic zones and arranging them into an ordered semantic zone sequence are as follows: based on the user's physical location, determine the user's current location as the starting point, read the preset path in the site diagram built into the navigation device, take the reference line direction as the benchmark, identify the topological nodes along the preset path in sequence, and take the continuous road segments between adjacent topological nodes as semantic zones. Starting from the semantic zone of the starting point and ending at the semantic zone of the target location, an ordered sequence of semantic zones is obtained; The specific steps for reconstructing the observation semantic zone sequence based on the interaction log are as follows: by reading the operation category records in the interaction log, extracting the semantic zone identifiers, and splicing them into an observation semantic zone sequence in chronological order.

2. The wayfinding interaction method based on AI digital human as described in claim 1, characterized in that: The specific steps for collecting and normalizing input events from the screen into an event stream and establishing reference lines are as follows: The navigation device uniformly records the atomic inputs generated in the screen side area and the screen front side area. Each atomic input is written as an event record, and the event records are strictly ordered by time to form an event stream. By using the pair of points formed by the calibration points laid on the ground and the pixels on the guide camera screen, the geometric mapping relationship between the screen plane and the ground reference plane is obtained. The upper and lower pixels of the vertical center line of the screen are taken and converted into actual points on the ground reference plane. The line connecting the actual points is used as the reference line, and the direction of the line connecting the actual points is used as the direction of the reference line.

3. The wayfinding interaction method based on AI digital human as described in claim 2, characterized in that: The specific steps for dynamically determining the user's physical location and line-of-sight area under environmental perception are as follows: The system obtains the user's physical location using the depth imaging camera of the wayfinding device, acquires the user's head orientation based on the user's head posture, and converts the user's head orientation to a ground reference coordinate system consistent with the reference line. Compare the angle between the user's head orientation and the reference line direction. When the angle does not exceed the forward angle threshold, the user's gaze is considered to be facing forward of the reference line. When the angle falls into the left allowable range, the user's gaze is considered to be pointing to the left. When it falls into the right allowable range, the user's gaze is considered to be pointing to the right. When the angle exceeds both the left and right allowable ranges and no interaction occurs within a fixed time, the interaction is considered to have ended. The system measures the nearest lateral distance from the user's physical location to the reference line. When the nearest lateral distance does not exceed the forward bandwidth threshold, the user is considered to be within the effective interaction area of ​​the navigation system. When the nearest lateral distance exceeds the forward bandwidth threshold and no interaction is performed within a fixed time, the user is considered to have deviated from the effective interaction area of ​​the navigation system. When the included angle does not exceed the forward angle threshold and the lateral nearest distance does not exceed the forward bandwidth threshold, the current line of sight area is classified as the reference line's forward dominant area. When the included angle is within the left allowable range and the lateral nearest distance does not exceed the forward bandwidth threshold, the current line of sight area is classified as the left adjacent area. When the included angle is within the right allowable range and the lateral nearest distance does not exceed the forward bandwidth threshold, the current line of sight area is classified as the right adjacent area.

4. The wayfinding interaction method based on AI digital human as described in claim 3, characterized in that: The step involves binding an on-screen indicator and several candidate target regions to each semantic zone, and dynamically sorting the candidate target regions. The specific steps are as follows: Record directional descriptions in each semantic zone and align arrows, annotation lines, and AI digital human speech anchors with the directional descriptions of the semantic zones. The target regions located within the geometric range of the current semantic zone, as well as the target regions in the next semantic zone that intersect or are tangent to the boundary of the current semantic zone, are merged into a set of candidate target regions for the semantic zone. Whenever a valid record is added to the event stream, the set of candidate target regions for the current semantic zone and the next adjacent semantic zone is rearranged in the following order: line-of-sight consistency priority, zone consistency priority, event proximity priority, proximity along the reference line priority, and turning complexity priority.

5. The wayfinding interaction method based on AI digital human as described in claim 4, characterized in that: When the semantic zone name is broadcast, the indicator and candidate target areas on the screen are highlighted. The specific steps are as follows: When the broadcast enters the semantic zone name field, the arrow describing the direction of the current semantic zone is immediately highlighted on the screen, and the first candidate target area icon in the sorting is also highlighted.

6. The wayfinding interaction method based on AI digital human as described in claim 5, characterized in that: The specific steps for determining whether the user's gaze remains on the top-ranked candidate target area within a fixed confirmation period are as follows: During the fixed confirmation period, the user's gaze points are sampled at equal intervals and assigned to the candidate target areas marked on the screen. The cumulative dwell time and continuity of each candidate target area covered by the gaze points are continuously counted. If the first-ranked candidate target area maintains the first cumulative dwell time and is not interrupted during the fixed confirmation period, it is considered that the gaze confirmation is passed; otherwise, it is considered that the gaze confirmation is not passed. If the visual confirmation fails, a confirmation control will appear around the first candidate target area in the sorting, and a back control will appear on the side of the screen. If the line of sight is confirmed to be clear, the current semantic zone is marked as confirmed. The AI ​​digital human continues to announce the name and action statements of the next semantic zone, repeating until the target location is reached.

7. The wayfinding interaction method based on AI digital human as described in claim 6, characterized in that: The specific steps for performing consistency verification with the ordered semantic zone sequence are as follows: By aligning the observed semantic zone sequence with the ordered semantic zone sequence using a table with the minimum number of steps in dynamic programming, the system calculates the minimum number of difference steps and locates the earliest misaligned semantic zone, and provides one-click correction and re-guidance on the site map.

8. A wayfinding interaction system based on AI digital human, based on the wayfinding interaction method based on AI digital human as described in any one of claims 1 to 7, characterized in that: include, The data acquisition and judgment module collects input events in front of the screen and normalizes them into an event stream, establishes reference lines, and dynamically judges the user's physical position and line of sight area under environmental awareness. The semantic zone module divides the area from the starting point to the target location into semantic zones based on reference lines, user physical location, and line of sight. These semantic zones are arranged into an ordered sequence. Each semantic zone is bound to an on-screen indicator and several candidate target areas on the screen. The candidate target areas are dynamically sorted, and an interaction log is established. The AI ​​digital human broadcasting module broadcasts information according to an ordered semantic zone sequence. When the semantic zone name is broadcast, the indicator on the screen and the candidate target area on the screen are highlighted. Within a fixed confirmation period, it is determined whether the user's gaze is fixed on the candidate target area ranked first. The reconstruction and verification module reconstructs the observed semantic zone sequence based on the interaction log and performs a consistency check with the ordered semantic zone sequence.

Images