A Cultural Tourism Guidance Method Based on the Collaboration of AR and Digital Landscape Signage
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-01
- Publication Date
- 2026-08-14
AI Technical Summary
[0006]有鉴于此,本发明的目的在于提供一种基于AR与数字景观标识协同的文旅导览方法,以缓解依赖游客主动操作导致游客游览连贯性差与沉浸感低的技术问题
[0016]本发明的其他特征和优点将在随后的说明书中阐述,并且,部分地从说明书中变得显而易见,或者通过实施本发明而了解。本发明的目的和其他优点在说明书、权利要求书以及附图中所特别指出的结构来实现和获得。
Smart Images

Figure CN122579056A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of smart cultural tourism technology, and in particular to a cultural tourism guidance method based on the collaboration of AR and digital landscape signage. Background Technology
[0002] Traditional physical signs in scenic areas offer limited information, and the integration of two-dimensional electronic maps with the actual environment is weak. Visitors must manually match map information with the layout of attractions and intersections, making them prone to getting lost or missing important sights in areas with obstructed visibility or complex building complexes. This creates a cognitive gap where visitors "can understand the map but can't find their way or see the whole picture." While existing digital landscape installations can display maps, they lack dynamic interaction with the visitor's real-time location and line of sight.
[0003] The existing system lacks an effective on-site feedback channel for tourists. When tourists encounter unexpected situations such as facility malfunctions, crowd congestion, or discover unexpected scenic spots / special activities, they cannot promptly inform scenic area managers and other tourists. Managers struggle to keep abreast of on-site dynamics, leading to delayed emergency response, inefficient resource allocation, and excessively long information transmission chains.
[0004] Traditional text-based, icon-based, and static map-based guides lack emotional connection and cultural depth. Tourists face cold signs or mechanical voice announcements, resulting in a monotonous experience lacking companionship, interactive fun, and cultural immersion, failing to meet the current demand for "deep cultural experiences."
[0005] However, existing technologies have the following problems: relying on tourists' active operation leads to poor continuity and low immersion in the tourist experience. Summary of the Invention
[0006] In view of this, the purpose of this invention is to provide a cultural tourism guide method based on the collaboration of AR and digital landscape signage, so as to alleviate the technical problems of poor continuity and low immersion of tourists due to reliance on tourists' active operation.
[0007] In a first aspect, embodiments of this application provide a cultural tourism guidance method based on the collaboration of AR and digital landscape signage, the method comprising: Step S1: The service device determines whether the tourist has entered the fence around the digital landscape signage device based on the location data reported by the tourist's mobile terminal, and marks the condition of the tourist entering the fence as a trigger event. In step S2, the service device sends AR content and digital human interaction data to the tourist's mobile terminal based on the triggering event, and at the same time sends a screen content update instruction to the digital landscape sign device so that the digital landscape sign device can be updated to be consistent with the tourist's mobile terminal. Step S3: In response to the tourist's operation command, the digital landscape signage device is updated synchronously with the tourist's mobile terminal.
[0008] In one implementation of the first aspect, a process of dynamic optimization based on the response result of the operation instruction is further included, the process comprising: The service equipment continuously collects visitor behavior data, digital landscape signage device operation data, and visitor mobile terminal operation data. The preprocessed data is input into a Long Short-Term Memory (LSTM) network to predict future pedestrian flow heatmaps, and the fence radius is adjusted based on the prediction results. Represented as: , in, The radius of the fence. This is the ratio of the current pedestrian density to the predicted pedestrian density. This represents the historical average population density.
[0009] One implementation of the first aspect is based on distance. smaller than the fence radius And absolute difference When a triggering event occurs, distance The calculation formula is as follows:
[0010] in, For the Earth's radius, and These are the latitude and longitude of the tourist's location, respectively. and These are the latitude and longitude of the location of the digital landscape signage device, respectively. The absolute difference The calculation formula is as follows: Calculate the azimuth angle from which a visitor points to the digital landscape signage device, the azimuth angle The calculation formula is as follows: , in, It is the arctangent function in the four quadrants; Calculate the direction of tourist movement absolute difference with azimuth .
[0011] In one implementation of the first aspect, step S3 further includes identifying the authenticity of the operation command, the identification process including: Operation commands under preset conditions are grouped into the same cluster, and a fusion confidence score is calculated for event clusters within the same cluster. Represented as: , in, For the first Confidence level of each reported event It is a natural constant. This is the current timestamp. For the first The timestamp of each event report.
[0012] In one implementation of the first aspect, the service equipment is deployed in a data center or public cloud platform to receive and process real-time data from tourists' mobile devices and manage the status of digital landscape signage devices.
[0013] Secondly, embodiments of this application provide a cultural tourism guide system based on the collaboration of AR and digital landscape signage, the system comprising: The trigger event recognition module is used to determine whether a tourist has entered the fence around the digital landscape signage device based on the location data reported by the tourist's mobile terminal, and to mark the condition of the tourist entering the fence as a trigger event. The visual synchronization module is used to send AR content and digital human interaction data to the tourist's mobile terminal based on the triggering event, and at the same time send a screen content update instruction to the digital landscape sign device so that the digital landscape sign device can be updated to be consistent with the tourist's mobile terminal. The collaborative interactive control module is used to respond to the operation commands of tourists, and the digital landscape signage device is updated synchronously with the tourist's mobile terminal.
[0014] Thirdly, embodiments of this application provide an electronic device, including: a processor, a memory, and a communication bus, wherein the processor and the memory communicate with each other through the communication bus; the memory stores program instructions executable by the processor, and the processor can execute the method provided in the first aspect or any possible implementation of the first aspect by calling the program instructions.
[0015] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer instructions that cause a computer to perform the method provided in the first aspect or any possible implementation thereof.
[0016] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained in accordance with the structures particularly pointed out in the description, claims and drawings.
[0017] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description
[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0019] Figure 1 This is a flowchart illustrating the cultural tourism guidance method based on the collaboration of AR and digital landscape signage provided in an embodiment of the present invention. Figure 2 This is a diagram illustrating the overall architecture of a cultural tourism guide system based on the collaboration of AR and digital landscape signage, as provided in an embodiment of the present invention. Figure 3 A schematic diagram of the module connection of a cultural tourism guide system based on the collaboration of AR and digital landscape signage provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0021] It should be noted that the data in this embodiment are all derived from a comprehensive analysis and evaluation of historical test data and corresponding historical test results from the three months prior to this test. Those skilled in the art will understand that the determination method for any of the above parameters can be based on selecting the value with the highest percentage in the data distribution as the preset standard parameter, using weighted summation to obtain the value as the preset standard parameter, substituting each historical data point into a specific formula and using the value obtained from that formula as the preset standard parameter, or other selection methods, as long as the present invention can clearly define different specific situations in the single-item judgment process through the obtained values.
[0022] Please see Figure 1 As shown, it is a flowchart of a cultural tourism guide system based on the collaboration of AR and digital landscape signage provided in an embodiment of the present invention.
[0023] To facilitate understanding of this embodiment, a detailed description of a cultural tourism guidance method based on the collaboration of AR and digital landscape signage disclosed in this embodiment of the invention will be provided first, including: Step S1: The service device determines whether the tourist has entered the fence around the digital landscape signage device based on the location data reported by the tourist's mobile terminal, and marks the condition of the tourist entering the fence as a trigger event. In step S2, the service device sends AR content and digital human interaction data to the tourist's mobile terminal based on the triggering event, and at the same time sends a screen content update instruction to the digital landscape sign device so that the digital landscape sign device can be updated to be consistent with the tourist's mobile terminal. Step S3: In response to the tourist's operation command, the digital landscape signage device is updated synchronously with the tourist's mobile terminal.
[0024] Specifically, this invention automatically captures tourists through positioning fences, pushes personalized AR content and digital human data to mobile devices, synchronously drives the screen updates of landscape installations, and performs path planning, event warnings and cultural explanations in conjunction with a unified control model. This eliminates information silos through multi-screen synchronization, and ensures controllable end-to-end response latency through a unified model, thereby improving tourist interaction rate, tour efficiency and cultural dissemination depth, and alleviating the problem of poor tour continuity and low immersion caused by reliance on tourists' active operation.
[0025] Specifically, in step S1, the tourist's mobile device reports location data, including latitude and longitude, once per second. , direction of travel ,speed After receiving the data, the location service module of the service equipment queries the surrounding digital landscape signage device fence with the tourist's coordinates as the center and a radius of 500 meters.
[0026] Specifically, in this embodiment of the invention, the distance between the visitor and the digital landscape signage device is calculated for each digital landscape signage device fence. and the azimuth angle from which tourists point to digital landscape signage devices The formula is as follows: , , in, The radius of the Earth is taken as 6371 km. and The latitude and longitude of the tourist's location (in radians). and The latitude and longitude (in radians) of the location of the digital landscape signage device. The arctangent function is defined in four quadrants, and its return value range is ( The π,π radians need to be converted to [0,360) degrees according to the convention.
[0027] Specifically, in the embodiments of the present invention, when the distance is obtained... and azimuth Under the condition that, when smaller than the fence radius Then, further determine the directional constraints. For fences with directional constraints, calculate the tourist's direction of travel. Orientation of the device absolute difference .when When the time is right, it is determined that the tourist is walking towards the device, thus meeting the triggering condition.
[0028] Specifically, in this embodiment of the invention, the triggering event is published to the message queue for consumption by the AR content engine module and the digital human interaction engine module. Once the same visitor triggers the same fence, it will not be triggered again within a five-minute cooldown period.
[0029] Specifically, this invention uses the location data reported by the tourist's mobile terminal every second. The service equipment calculates the spherical distance and azimuth angle with the surrounding devices in real time. Combined with the fence radius and travel direction constraints, it accurately determines the valid entry event, which improves the accuracy of fence triggering. The cooling mechanism reduces redundant pushes and saves terminal and network resources, thereby improving the timeliness and smoothness of the tourist experience and alleviating the problem of poor continuity and low immersion of the tourist tour due to reliance on the tourist's active operation.
[0030] Specifically, in step S2, the AR content engine module of the service device queries related content resources based on the device identification code, and after weighting and scoring according to timeliness, visitor matching degree, and content popularity, selects the content with the highest score for distribution. The calculation formula is as follows: , in, The timeliness score is calculated based on how close the current time is to the content's expiration date. The score is calculated based on the interest tags in the visitor profile to determine the match between the content and the visitor's interests. Rate the popularity of the content.
[0031] Specifically, in this embodiment of the invention, after obtaining a weighted score, the tourist's mobile device acquires the device's unique identifier through visual recognition. It then retrieves the device's three-dimensional coordinates in the scenic area's world coordinate system and a predefined set of anchor point feature point coordinates from local cache or the cloud. The tourist's mobile device runs a simultaneous visual localization and mapping algorithm to extract device image feature points and match them with predefined anchor point feature points, obtaining a 2D-3D corresponding point set. An efficient perspective n-point algorithm combined with a random sampling consensus algorithm is used to solve for the pose transformation matrix from the device's local coordinate system {D} to the camera coordinate system {C}. , in, Let be a rotation matrix. It is a translation vector.
[0032] Specifically, the digital landscape signage device predefines multiple anchor point coordinates, all defined in the device's local coordinate system {D}. For example, the navigation arrow anchor point... Information floating window anchor point The rendering location of AR content The coordinates in the camera coordinate system {C} are calculated using the following formula: , in, Let be a rotation matrix. It is a translation vector. This is the anchor point for the navigation arrow.
[0033] Specifically, in this embodiment of the invention, the three-dimensional points in the camera coordinate system are projected onto the two-dimensional screen coordinates through the camera intrinsic parameter matrix KK, thereby achieving accurate overlay rendering of AR content.
[0034] Specifically, in step S2, when a tourist triggers the fence, the service device sends a screen content update command to the digital landscape signage device. The device screen is visually synchronized with the AR content on the tourist's mobile device, and the device screen synchronously displays: a highlighted mark of the tourist's current location, the name and brief introduction of the current attraction, a recommended tour route and estimated time, and a digital human image to be activated.
[0035] Specifically, in this embodiment of the invention, the device screen content is uniformly managed by the cloud map linkage module. When tourists perform route planning or attraction search on their mobile devices, the device screen updates the displayed content synchronously, achieving "one-stop operation, multiple synchronizations".
[0036] Specifically, in step S2, the triggering condition for the digital human to actively serve is that the tourist is less than 10 meters away from the digital landscape signage device and stays for more than 3 seconds, or the tourist is walking towards the digital landscape signage device and the distance is less than 15 meters. After the triggering, the digital human on the device screen will automatically wake up, turn its head towards the tourist, show a smiling expression, make a greeting gesture that matches the culture of the scenic spot, and issue a voice greeting.
[0037] Specifically, in this embodiment of the invention, the greeting voice is played through a directional speaker with a beam angle of 30 to 60 degrees. If the visitor does not respond within five seconds, the digital human automatically enters standby mode and the screen displays interactive prompts.
[0038] Specifically, this invention intelligently selects AR content by weighting scores based on timeliness, tourist matching degree, and content popularity. It then uses visual SLAM and anchor point matching to calculate device pose, achieving precise overlay rendering of elements such as navigation arrows and information floating windows. Simultaneously, it drives the device screen to synchronously highlight the tourist's location, recommended routes, and the digital human to be woken up. The digital human proactively greets the user based on distance and dwell time, and plays voice messages through directional speakers. This enhances the personalization and real-time nature of content delivery, achieves seamless multi-screen collaboration between mobile devices and fixed devices, optimizes the tour guide experience, and alleviates the problem of poor tour continuity and low immersion caused by reliance on active tourist operation.
[0039] Specifically, in step S3, for any linked scene The unified control model is implemented, and the unified control model is expressed as follows: , in, Update actions for the mobile AR interface for tourists. Update the screen of the digital landscape signage device. This is an action to update the dynamic map database in the cloud. Update the digital human's state.
[0040] Specifically, the four update actions of the unified control model are executed synchronously, with an end-to-end delay of no more than two hundred milliseconds.
[0041] Specifically, in this embodiment of the invention, after a tourist initiates a route planning request, the digital human interaction engine module performs intent recognition, extracts destination and preference parameters, and the map linkage module calls the route planning algorithm. The cost function comprehensively considers distance, congestion, altitude changes, and tourist preferences, and the cost function is expressed as follows: , in, The maximum length of all path segments (taken as 500 meters). For the first The congestion coefficient of the path segment, ranging from zero to one, is obtained by normalizing real-time pedestrian flow data. For the first The absolute value of the elevation change of the route segment. The maximum elevation change is taken as 50 meters. For the first The tourist preference matching degree of the route segment is calculated based on the tourist's historical behavior. The higher the preference, the lower the coefficient of the route segment. The route planning result includes the route coordinate sequence, total length, estimated time, and list of attractions along the way.
[0042] Specifically, in this embodiment of the invention, when a route planning request is initiated, the digital human generates voice broadcast content and directional gestures, and simultaneously pushes the route information to the tourist's mobile device, the current device screen, and all digital landscape signage devices on the subsequent route to achieve relay-style guidance. When the tourist deviates from the route by more than ten meters, the route is automatically replanned.
[0043] Specifically, in this embodiment of the invention, after a tourist reports an event, the crowdsourced data processing module performs spatiotemporal clustering under preset conditions. The preset conditions are that events with a location distance of less than 20 meters and a time difference of less than 5 minutes are grouped into the same cluster. A fusion confidence score is calculated for each event cluster. Represented as: , in, For the first Confidence level of each reported event It is a natural constant. This is the current timestamp. For the first The timestamp of each event report.
[0044] Specifically, in this embodiment of the invention, given the obtained fusion confidence level, when the number of events within a cluster... and When the event takes effect, the system will write the event to the dynamic map, push the warning command to devices within 500 meters, display the prompt information on the device screen, generate broadcast content in the digital human, and overlay the event icon on the mobile AR interface. The event is valid for two hours.
[0045] Specifically, in this embodiment of the invention, after tourists click on cultural attractions or ask questions to the digital human through the AR interface, the target attraction is identified, and the explanation content is retrieved from the knowledge base. For attractions that support AR restoration, the historical scene restoration model is downloaded and superimposed on the real scene through visual anchoring. The digital human provides explanations with voice and posture that match the culture of the scenic spot, and gesture guidance is provided. After the explanation, the tourist's mobile AR interface displays the entry point of related content to guide the tourist to continue exploring.
[0046] Specifically, the following data will be continuously collected: tour trajectory data (location sequence, stop points), interactive behavior data (AR click rate, viewing time, completion rate), digital human dialogue data (dialogue rounds, question classification, satisfaction), reported event data (type, location, review results), and check-in and sharing data (check-in points, sharing channels). Sensitive data will be anonymized, and tourist identifiers will be replaced with anonymous identifiers.
[0047] Specifically, this embodiment of the invention uses a long short-term memory network, inputs the past 24 hours of pedestrian flow time series data, and outputs a pedestrian flow prediction heat map for the next 6 hours. The prediction results are used to dynamically adjust the fence radius and the congestion weight of the path planning. The pedestrian flow time series data is tour trajectory data.
[0048] Specifically, the embodiments of the present invention employ collaborative filtering and deep interest networks, input tourist interaction sequences and content features, output click-through rate prediction values, perform incremental training weekly, and analyze interruption rates and satisfaction based on dialogue records, optimize dialogue strategies through reinforcement learning, and update the knowledge base monthly. The tourist interaction sequences include tour trajectory data, interaction behavior data, digital human dialogue data, and check-in sharing data, and the dialogue records are digital human dialogue data.
[0049] Specifically, the embodiments of the present invention dynamically adjust the fence radius, and the adjusted fence radius... Represented as: , in, The radius of the fence. This is the ratio of the current pedestrian density to the predicted pedestrian density. This represents the historical average population density.
[0050] Specifically, under the condition of obtaining the adjusted fence radius, the push priority is adjusted in this embodiment of the invention: high-interaction content is given a higher weight, low-interaction content is given a lower weight or removed from the platform, and the wake-up strategy is optimized: the trigger threshold is adjusted according to the proactive service response rate, and the cooldown time is dynamically adjusted according to visitor feedback.
[0051] Specifically, this invention establishes an evaluation index system: AR content completion rate target ≥60%, digital human dialogue satisfaction target ≥85%, route planning accuracy target ≥95%, human customer service intervention rate target decrease by 50%, tourist complaint rate target decrease by 40%, secondary consumption conversion rate target increase by 15%, average tourist stay time target increase by 30%, and regularly generates evolution effect reports to compare changes in indicators before and after evolution, and rolls back or adjusts optimization measures that do not meet the targets.
[0052] Specifically, this invention triggers visual synchronization between mobile AR and landscape device screens through fence perception, and uses a unified control model to coordinate and update path planning, event linkage, and cultural explanation in four ways. At the same time, it utilizes LSTM crowd prediction, deep recommendation, and dynamic fence radius adjustment to achieve adaptive optimization of content push and wake-up strategies, drive continuous service evolution, improve the completion rate of tourist interaction and route accuracy, and alleviate the problem of poor tourist tour continuity and low immersion caused by relying on tourists' active operation.
[0053] Please see Figure 2 As shown, it is the overall architecture diagram of the cultural tourism guide system based on the collaboration of AR and digital landscape signage in an embodiment of the present invention.
[0054] Specifically, the overall architecture of the cultural tourism guide system based on the collaboration of AR and digital landscape signage includes the following parts: digital landscape signage devices deployed in key locations of the scenic area, service equipment (cloud servers) deployed in data centers or public cloud platforms, and mobile terminals for tourists running on smartphones or tablets.
[0055] Specifically, the digital landscape signage device serves as a spatial anchor point between the physical and digital worlds, providing precise three-dimensional coordinate references and visual recognition targets. All computationally intensive tasks, including digital human rendering, voice interaction, and complex AR content generation, are completed collaboratively by the visitor's mobile terminal and the cloud server. The digital landscape signage device itself only undertakes positioning, communication, identification, and simple prompting functions, achieving the design goals of lightweight, low power consumption, and easy deployment.
[0056] Specifically, the digital landscape signage device in this embodiment of the invention is a lightweight intelligent interactive terminal deployed along scenic routes, important attractions, intersections, service points, and other key locations. The device is in the form of a small sign, with a biomimetic or antique design that blends with the culture and natural landscape of the scenic area, and its protection level reaches the IP65 standard.
[0057] Specifically, the digital landscape signage device described in this embodiment of the invention comprises the following units: 1. Positioning unit: The digital landscape signage device has a built-in high-precision positioning module for obtaining the precise three-dimensional coordinates of the digital landscape signage device itself, which serves as a spatial reference for the overlay of AR content on the tourist's mobile device, ensuring the positioning accuracy of virtual-real fusion. The positioning module uses a multi-mode, multi-frequency, high-precision positioning chip, supporting GPS L1C / A and L2C signals, BeiDou B1I and B2I signals, GLONA SSL1OF and L2OF signals, and Galileo E1 and E5b signals. It has at least 184 channels. The positioning module achieves a single-point positioning accuracy of no more than 1.5m in open areas and no more than 0.6m when receiving differential signals. The cold start time is no more than 26 seconds, the warm start time is no more than 2 seconds, and the data update frequency is 10Hz. It also supports RTCM3.x protocol differential data input and can receive correction data broadcast by differential service providers via cellular networks, further achieving centimeter-level positioning accuracy. When installing the digital landscape signage device, a handheld RTK measuring instrument is used to obtain the precise latitude and longitude of the installation point, with a measurement error of no more than 2cm. The measured coordinates are written into the device's local non-volatile memory, and the coordinates are reported to the cloud server when the digital landscape signage device is first put online. A static mapping relationship is established with the unique identification code of the digital landscape signage device to ensure that the coordinate information is permanently bound and not easily tampered with.
[0058] 2. Communication unit: The digital landscape signage device is equipped with a multi-mode wireless communication module for stable data interaction with the cloud server, receiving display content update instructions and status synchronization instructions to ensure real-time data synchronization between the cloud and the terminal. The cellular communication module uses a 4GCat.1 module, which supports major frequency bands of domestic operators, including B1, B3, B5, B8, B34, B38, B39, B40, and B41. When the signal is normal, it maintains a long connection with the cloud and sends a heartbeat data packet every 30 seconds to report the device's operating status in real time. The multi-mode wireless communication module has a working power consumption of no more than 2W and a standby power consumption of no more than 0.5W, taking into account both communication stability and low power consumption requirements. The digital landscape signage device can be equipped with a Bluetooth Low Energy module that complies with the BLE 5.0 standard. It is mainly used for near-field debugging and parameter configuration, and can also be used as a Bluetooth beacon broadcast device identifier, allowing mobile devices to automatically detect the presence of nearby digital landscape signage devices without scanning the code, thus expanding near-field interaction scenarios. The communication data is encapsulated in JSON format. Typical data structures include fields such as device identifier, timestamp, device status, display content type and text, and LED indicator control parameters. The data format is concise and standardized, facilitating bidirectional parsing and processing between the cloud and the terminal.
[0059] 3. Unique Identifier Code: In this embodiment of the invention, each digital landscape signage device is assigned a globally unique identifier code. This identifier code serves as both the identification ID of the visual anchor point and the device's identity authentication credential, thereby achieving unique identification and spatial positioning binding of the device. The unique identifier adopts a structured encoding format with a total length of 64 binary bits, which is converted into a 16-bit hexadecimal string, printed as a QR code on the surface of the device, and simultaneously written into the NFC tag, thus supporting both QR code scanning and near-field NFC identification. The encoding structure of the unique identifier is divided into the following bit fields: the version number field occupies 4 bits and is used to identify the encoding format version. The initial version is set to 0001. The device type field occupies 4 bits, where 0001 represents a scenic spot node device, which is deployed in front of the core scenic spot; 0010 represents a fork in the road device, which is deployed at the intersection of multiple paths; 0011 represents a service facility device, which is deployed in locations such as toilets, shops, and medical rooms; and 0100 represents a guide point device, which is deployed in the middle of a long path for intermittent guidance. The unique identifier has the following characteristics: the x-coordinate field occupies 20 binary bits, with the southwest corner of the scenic area as the origin and the east direction as the positive X-axis, with a quantization unit of 0.1m; the y-coordinate field occupies 20 binary bits, with the southwest corner of the scenic area as the origin and the north direction as the positive Y-axis, with a quantization unit of 0.1m; the height field occupies 8 binary bits, with the ground at the device installation location as the zero point and upward as the positive direction, with a quantization unit of 0.1m; the same position sequence number field occupies 4 binary bits, used to distinguish when multiple devices are deployed at the same coordinate point; and the cyclic redundancy check field occupies 4 binary bits. The CRC-4 algorithm is used to check the first 60 bits of data, with an error detection capability of not less than 99%, ensuring the accuracy of identifier transmission and parsing. For any digital landscape signage device, let the latitude and longitude of the installation point of the digital landscape signage device be . and The height is The southwest corner of the scenic area has the following coordinates: and Earth's equatorial radius Take 6378137m; Convert the latitude and longitude of the southwest corner into plane coordinates. and The conversion formula is as follows. , , Where λ, φ, λ0, and φ0 are all in radians; The planar coordinates and height are then quantized using the following formula. , , , In the formula, Using 0.1m as the unit of measurement, This is the rounding function. The value range is from 0 to 1048575. The value range is from 0 to 1048575. The value range is from 0 to 255 to adapt to the bit field storage rules; When a mobile device scans the device's QR code or reads an NFC tag, it parses the identification code to obtain... , , The field, calculated in reverse, contains the physical coordinates of the device. , , ; To display the device's location on a map, latitude and longitude can be calculated using the following formula. , , After obtaining the device coordinates on the tourist's mobile device, the device type field is used to retrieve the device's detailed configuration information from the cloud server, including the device name, function description, and associated AR content resource identifiers, providing data support for subsequent interactions.
[0060] 4. Small display screen: The digital landscape signage device is equipped with a small display screen on the front, which is used to provide tourists with simple text prompts, directional indicators and status information without the need for complex interaction, thereby reducing power consumption and cost; The display uses e-ink technology and utilizes bistable properties, consuming power only when refreshing content and having zero power consumption when displaying static content. It is suitable for long-term outdoor standby scenarios. The screen size can be selected between 3.5 inches and 5.5 inches depending on the deployment location, with a resolution of 480×320 pixels or 640×480 pixels. It supports black and white or black and white red display modes and remains clearly readable under direct sunlight without the need for additional backlighting. The display screen refreshes no more than 10 times per day. Content updates are triggered by cloud commands or checked periodically according to preset cycles to avoid frequent refreshes and power consumption. The types of content displayed include directional text (such as "Ancient temple 200 meters ahead"), directional icons (arrows with distance numbers), status icons (markers for public facilities such as restrooms, restaurants, and medical facilities), a QR code with the device's unique identifier (for mobile scanning), and a simple thumbnail map of surrounding attractions to meet basic navigation prompts. The display instructions issued by the cloud server include a content type field, a content text field, and a validity period field. After receiving the instructions, the digital landscape signage device verifies the integrity of the instructions, renders the content into an image format adapted to the screen resolution, and writes it to the electronic ink screen driver circuit through the SPI interface to complete the refresh, ensuring that the displayed content is accurately synchronized.
[0061] 5. Status indicator lights: Full-color RGB LED indicator lights are embedded on the top or frame of the digital landscape signage device to intuitively reflect the device's operating status and interactive status, making it easy for tourists to identify and for maintenance personnel to troubleshoot. The status indicator light uses a three-color LED chip package and achieves different colors and brightness outputs through PWM dimming. The peak brightness of a single LED is not less than 5000mcd, providing strong outdoor visibility. The status indicator light states are defined as follows: solid green indicates that the device is in normal standby mode; flashing green indicates that it is communicating with the cloud server; solid blue indicates that a visitor is interacting with the device by scanning a code or using NFC; solid yellow indicates that the battery level is below 20% and requires timely maintenance; solid red indicates that the device has malfunctioned; and flashing red indicates that the device needs immediate repair or replacement. The status indicator lights are controlled by the main control MCU through the GPIO interface to output PWM signals. The control logic is linked with the operating state machine of the digital landscape signage device, and the state switching is real-time and without delay.
[0062] 6. Power Management System: The digital landscape signage device adopts a multi-source power supply scheme, which flexibly selects the main power supply mode according to the conditions of the deployment location, and is equipped with a backup battery to ensure continuous operation in the event of a short-term power outage, adapting to various deployment scenarios in scenic areas. For deployment locations with access to mains power, a 220V AC to 12V DC power adapter is used for power supply, with a rated output power of not less than 30W. It also has a built-in lithium-ion backup battery pack with a capacity of not less than 5000mAh, which can maintain normal operation of the device for no less than 24 hours when the mains power is interrupted, thus avoiding power outages. For remote areas or areas without mains power, a solar power supply solution is adopted, equipped with monocrystalline silicon solar panels with a rated power of not less than 20W and a peak operating voltage of 18V. A solar charging controller is provided to achieve maximum power point tracking and battery charge and discharge management to prevent overcharging and over-discharging. The energy storage battery uses a lithium iron phosphate battery pack with a nominal voltage of 12.8V and a capacity of not less than 20Ah, which can meet the normal operation of the device for no less than 5 days under continuous cloudy and rainy weather, and achieve long-term stable operation in passive scenarios. The device's main control MCU monitors the battery voltage and charging / discharging current in real time through the ADC interface. When the battery level is lower than a preset threshold, the device will actively switch to energy-saving mode, reduce the communication frequency, and shut down unnecessary peripherals to extend the battery life. The overall power consumption of the digital landscape signage device is controlled within a reasonable range. The communication module consumes no more than 1.5W when working and no more than 0.2W when in standby mode. The positioning module consumes no more than 0.3W when working continuously and no more than 0.05W when in standby mode. The e-ink screen consumes power only when refreshing and consumes zero power when displaying statically. The main control MCU consumes no more than 0.5W when working and no more than 0.1W when in standby mode. The status indicator lights consume no more than 0.3W when working. Based on comprehensive estimates, the digital landscape signage device consumes approximately 11Wh of power per day. Under the condition of 4 hours of effective sunshine per day, the 20W solar panel generates approximately 80Wh of electricity per day. The power generation is far greater than the power consumption, which fully meets the long-term self-sufficiency power supply requirements.
[0063] 7. Main control unit: The digital landscape signage device is equipped with a low-power industrial-grade main control microcontroller, which is responsible for coordinating the operation of various peripheral modules, executing the communication protocol stack, processing cloud instructions and managing local data storage. It is the core control center of the device. The main control chip adopts the Xtensa LX7 architecture or ARM Cortex-M4 architecture, with a main frequency not exceeding 240MHz. It has no less than 512KB of static random access memory and no less than 16MB of flash memory. The peripheral interfaces include no less than three universal asynchronous transceivers, two I²C buses, two serial peripheral interfaces, and no less than 16 universal input / output interfaces, which can meet the connection and control requirements of various peripheral modules. The core functions of the main control chip include: reading NMEA0183 protocol positioning data output by the positioning module and parsing the coordinates through the universal asynchronous transceiver interface; controlling the refresh of the e-ink screen through the serial peripheral interface; controlling the status indicator light through the universal input / output interface; monitoring the battery voltage and charging / discharging status through the analog-to-digital converter interface; interacting with the cellular communication module through the universal asynchronous transceiver interface to achieve data synchronization with the cloud server; and storing key data such as device identification code, coordinate information, and display content in flash memory to ensure that data is not lost after power failure and that the operating status is quickly restored after restart.
[0064] Specifically, the digital landscape signage device interacts with tourists' mobile devices in various ways, covering different scenarios and user operating habits, thereby improving the convenience and adaptability of the interaction.
[0065] Preferably, the surface of the digital landscape signage device is printed with a QR code corresponding to a unique identifier. Tourists can obtain the device's identifier by scanning the QR code with their mobile devices using a camera, and then parse or request the device's precise coordinates, type information, and associated AR content resources from the cloud. This method is the primary interaction method and is suitable for scenarios where tourists actively seek information.
[0066] Preferably, the digital landscape signage device has an embedded NFC tag that stores the device's unique identification code and supports the ISO14443 Type A protocol. When a tourist's mobile device is close to the NFC sensing area of the device, the identification code can be read quickly without turning on the screen or starting the camera, making the operation more convenient and suitable for rapid scene identification.
[0067] Preferably, the digital landscape signage device can be equipped with a Bluetooth Low Energy module to broadcast beacon data packets containing the device identification code at a fixed frequency. When a visitor's mobile device enters the broadcast coverage area, it can automatically detect the presence of nearby devices and trigger corresponding interactions without requiring manual operation by the visitor, thus achieving a seamless triggering experience and optimizing the continuity of the tour.
[0068] Once the mobile device obtains the device identification code, it decodes the device coordinates using the decoding method. At the same time, it requests the AR content resource identifier, digital human interaction scene identifier, and dynamic map data associated with the device from the cloud server. This enables precise overlay of AR content and three-in-one linkage, completing a virtual-real integrated cultural tourism guide experience.
[0069] Specifically, the cloud server described in this embodiment of the invention is deployed in a data center or a public cloud platform to receive and process real-time data from tourists' mobile devices, manage the status of digital landscape signage devices, distribute AR content, drive digital human interaction, maintain dynamic maps, and integrate crowdsourced data. The cloud server adopts a microservice architecture, with each functional module deployed independently and horizontally scaled, supporting high-concurrency access and low-latency response.
[0070] Specifically, the cloud server consists of the following modules: 1. Location Service Module: This module receives real-time location data reported by the tourist's mobile device, calculates the distance and relative orientation between the tourist and each digital landscape sign device, determines whether the tourist has entered the dynamic geofence, and triggers the corresponding interaction process based on the judgment result. The tourist's mobile device reports location data packets at a frequency of no less than once per second. The data packet format adopts a JSON structure and includes the following fields: tourist unique identifier, timestamp, latitude and longitude coordinates, altitude, positioning accuracy, direction angle, movement speed, current battery level, and network status. After receiving the data packet, the location service module first verifies the validity of the timestamp and discards data with a delay of more than five seconds. Then, it filters the coordinates based on the positioning accuracy field. When the positioning accuracy is greater than ten meters, the Kalman filter algorithm is used to smooth the historical trajectory and reduce the impact of positioning jumps on the trigger judgment. Each digital landscape signage device is associated with one or more dynamic geofences. The fence parameters are pre-stored in the database and include the following fields: device identifier, fence type, fence center point coordinates, fence radius, trigger direction constraint, effective time period, and associated interactive process identifier. There are three types of fences: main attraction fence, with a radius of 30 meters, triggered when the visitor is less than the radius of the device and is moving towards the device, used for in-depth guided tours of core attractions; intersection fence, with a radius of 50 meters, triggered when the visitor is less than the radius of the device and has no direction constraint, used for advance guidance of path selection points; and service point fence, with a radius of 15 meters, triggered when the visitor is less than the radius of the device, used for near-field prompts for service facilities such as restrooms and shops. The fence data is stored in Redis, using the device identifier as the key and the fence parameters as the value, and a geospatial index is established to support fast retrieval of surrounding fences by coordinate range. When the visitor's location data arrives, the location service module executes the following trigger judgment process: First, it queries all geofences within a 500-meter radius of the visitor's coordinates; second, it calculates the distance between the visitor and the center point of each fence. The distance The calculation formula is as follows: , in, The radius of the fence is taken as 6371 km. and The latitude and longitude of the tourist's location (in radians). and Latitude and longitude (in radians) for the location of the digital landscape signage device. Next, determine the distance. Is it smaller than the fence radius? If the value is less than the specified value, then the directional constraints are further assessed. For fences with directional constraints, the angle between the tourist's direction of travel and the orientation of the device is calculated. : , in, The direction angle for tourists' movement, ranging from 0 to 360 degrees. The direction angle is the directional angle from the tourist's position to the device's position. Calculate using the following formula: , Among them, when When the angle is less than 90 degrees, it is determined that the tourist is moving towards the device, triggering the fence; Finally, the trigger event is encapsulated into a message and sent to the message queue for consumption by the AR content engine module and the digital human interaction engine module. The trigger event includes fields such as visitor identifier, device identifier code, trigger time, visitor's current location, and visitor's direction of travel. To prevent the same visitor from repeatedly triggering the same fence within a short period of time, the location service module is equipped with a trigger cooling-off mechanism. Once the same visitor triggers the same fence, it will not be triggered again within five minutes. The cooling status is stored in Redis with the key name "trigger_cooldown: visitor identifier: device identifier code" and an expiration time of three hundred seconds.
[0071] 2. AR Content Engine Module: This module stores digital assets such as 3D models, AR navigation data, scenic spot explanation libraries, and historical scene restoration materials bound to the digital landscape signage devices. Based on visitor location and trigger events, it sends customized AR content data packages to visitors' mobile devices. Each AR content resource is defined as a content object in the database, containing the following fields: unique content identifier, associated device identifier, content type, content version number, valid start time, valid end time, applicable visitor age group, applicable weather conditions, content data URL, rendering parameters, and a list of dependent resources. Content types are divided into three categories: navigation indicators (including 3D arrow models, path lines, distance labels, etc.); explanation and display (including 3D models, images, text, videos, etc.); and immersive experience (including historical scene restoration models, AR effects, particle effects, etc.). Content resource files are stored in an object storage service, categorized by content type and device identifier. 3D models use glTF or FBX format, texture images use PNG or JPG format, and videos use H.264 encoded MP4 format. The AR content engine module's delivery strategy is as follows: When the location service module triggers a fence event, the AR content engine module receives the trigger message and executes the content matching and delivery process. First, it queries all AR content resources associated with the device based on the device identifier code, filtering them by valid time, visitor attributes, and weather conditions. Visitor attributes are obtained through a visitor profile service, including age group, interest tags, and historical interaction records. Weather conditions are obtained through a third-party weather service interface to obtain real-time weather data for the visitor's current location. Second, the filtered content is prioritized, and a weighted scoring method is used for priority calculation. The calculation formula is as follows: , in, The timeliness score is calculated based on how close the current time is to the content's expiration date. The score is calculated based on the interest tags in the visitor profile to determine the match between the content and the visitor's interests. Content popularity is scored based on historical click-through rate and interaction completion rate. , , These are the weighting coefficients, taken as 0.4, 0.4, and 0.2 respectively; Next, the content resource with the highest rating is selected to generate a data package for distribution. This data package includes information such as content type, model URL, texture URL, animation parameters, rendering position offset, rendering scaling ratio, and display duration. Finally, the data package is pushed to the visitor's mobile device via a WebSocket persistent connection, with a push latency not exceeding 200 milliseconds. If the visitor's mobile device is currently offline, the data package is temporarily stored in a message queue and resent once the mobile device is online. The AR content engine module supports version updates for its AR content resources. Each content modification generates a new version number. When a tourist requests content on their mobile device, they include the locally cached version number. The server compares the version numbers; if the local version is the latest, it returns an empty response to avoid duplicate transmissions. Content updates use an incremental push method, only distributing changed resource files. Content undergoes an approval process before publication, and an effective time is set after approval. It supports both scheduled and immediate publication modes to meet the needs of scenic area activity previews and temporary adjustments.
[0072] 3. Digital Human Interaction Engine Module: This module manages the digital human image library, voice library, dialogue logic tree, and facial expression / action library. It combines information such as visitor location, historical trajectory, question content, and real-time weather to generate personalized voice responses and action commands, which are presented to users via their mobile devices. The digital human image library stores various styles of digital human models, including virtual ambassadors for scenic spots, historical figures, and cartoon mascots. Each image includes the following attributes: unique image identifier, image name, applicable scenarios, model file URL, texture URL, skeletal binding data, default posture, default expression, and default voice timbre. The voice library stores digital human speech synthesis samples and pre-recorded speech segments. Speech synthesis uses a deep neural network acoustic model, supporting multiple timbres and emotional modes, including happy, gentle, urgent, and solemn. Emotional modes are achieved by modifying the prosodic parameters of speech synthesis, including fundamental frequency range, speech rate, volume, and pause patterns. The dialogue logic tree uses a state machine model to define the flow path of dialogue between the digital human and the visitor. Each dialogue node includes the following attributes: node identifier, trigger condition, digital human voice content, digital human facial expression parameters, digital human motion parameters, next node list, and associated AR content identifier. The dialogue logic tree supports conditional branching, dynamically selecting the next node based on the visitor's input, visitor profile, and scene context. The facial expression and motion library stores the digital human's facial expression animations and body motion parameters. Facial expressions are implemented using BlendShape technology, defining fifty basic facial expression units, which generate rich facial expression variations through weighted combination. Motions are implemented using skeletal animation technology, including common actions such as waving, pointing, nodding, and bowing. Each action lasts two to five seconds and supports loop playback and motion fusion. When a tourist initiates a voice conversation or text input via a mobile device, the digital human interaction engine module receives the request and executes the following processing flow: First, intent recognition is performed. For voice input, the speech recognition service is invoked to convert audio into text; for text input, the input text is used directly, and then word segmentation, part-of-speech tagging, and named entity recognition are performed to extract the tourist's intent and key information. Intent classification adopts a fine-tuned model based on BERT, supporting five types of intents: asking for directions, asking about attractions, asking about facilities, asking about activities, and casual conversation. Second, context association is performed. The history of the tourist's last five rounds of dialogue is retrieved from the conversation cache, and context disambiguation is performed in conjunction with the current intent. For example, if the tourist asks "How big is it?" and "When was it built?" consecutively, the specific attraction referred to by "it" is identified based on the preceding context. Third, dialogue logic tree matching is performed. Based on the intent type and key information, the current dialogue node is located in the dialogue logic tree. If an exact match exists, the content of that node is returned directly; otherwise, a default fallback node is used. Each dialogue node contains the digital human's voice content, facial expression parameters, motion parameters, and information about the next node. Then, a personalized response is generated. Variables in the voice content are dynamically filled in based on the context, such as attraction names, distances, and times. During voice synthesis, an appropriate emotional mode is selected based on the tourist's age and the scene: a lively tone and higher speech rate are used for children's scenes, while a gentle volume and soothing speech rate are used for nighttime scenes. Finally, the response data package is encapsulated and pushed to the tourist's mobile device via WebSocket. The data package contains the voice text, voice synthesis parameters, facial expression-driven parameters, motion-driven parameters, and associated AR content identifiers. Furthermore, the digital human interaction engine module supports a proactive service mode, which can proactively initiate dialogue based on the visitor's location and behavioral characteristics, enhancing the human-like experience of the guided tour. Triggering conditions for proactive service include visitors entering a geofence and staying for more than ten seconds, visitors staring in one direction for an extended period, visitors hesitating at intersections, and visitors frequently turning back when passing devices. These triggering conditions are jointly determined by the location service module and the mobile behavior analysis module. The content of the proactive service is selected based on the scenario type: in main attraction scenarios, the digital human proactively introduces the attraction's background story; in intersection scenarios, the digital human proactively inquires about the destination and recommends routes; in service point scenarios, the digital human proactively informs visitors of facility status and queue conditions. The frequency of proactive service is controlled using a fatigue mechanism, limiting proactive service triggers to no more than three times per hour for the same visitor to avoid harassment. The fatigue counter is stored in Redis and resets every hour.
[0073] 4. Map Linkage Module: The map linkage module is responsible for maintaining the global dynamic map database, which includes scenic area road network data, point of interest data, real-time visitor flow data, facility status data, activity forecast data, visitor check-in point data, etc., and generates the optimal tour route based on visitors' AR operations or digital human interactions, and synchronously updates the screen display content of relevant digital landscape signage devices. The dynamic map database of the map linkage module uses PostgreSQL with PostGIS spatial extension as the storage engine, supporting the storage and querying of geospatial data. Road network data includes geometric and attribute information of scenic area roads. Road geometry is stored as line features, including nodes and edges. Attribute information includes road name, road type, width, slope, number of steps, lighting conditions, wheelchair accessibility, etc. Points of interest (POI) data includes point features such as attractions, service facilities, and entrances / exits. Each POI includes the following attributes: POI identifier, name, type, latitude and longitude coordinates, elevation, opening hours, ticket price, average visit duration, and associated device identification code. Real-time visitor flow data is obtained through [unclear - likely a system or mechanism]. Location data reported by visitors' mobile devices is aggregated and calculated to determine the pedestrian density around each road network node and point of interest in a five-minute time window, categorized into four levels: smooth, normal, crowded, and congested. Facility status data is reported through digital landscape signage devices, including the number of people queuing for restrooms, the business status of shops, the number of parking spaces remaining, and whether the medical room is open. Activity preview data is entered by the scenic area manager through the backend, including the activity name, location, time, participation method, and status. Visitor check-in point data records visitors' check-in behavior in the scenic area, with each check-in point including fields such as visitor ID, check-in time, check-in location, and check-in type, used to generate personalized tour routes and social recommendations. When tourists initiate route planning requests through the AR interface or digital human, the map linkage module executes a path planning algorithm to generate the optimal tour route. The path planning uses the A* algorithm, and the cost function comprehensively considers distance, congestion, altitude changes, and tourist preferences. The calculation formula is as follows: , in, For the first Length of the path segment The maximum length of all path segments (taken as 500 meters). For the first The congestion coefficient of the path segment, ranging from zero to one, is obtained by normalizing real-time pedestrian flow data. For the first The absolute value of the elevation change of the route segment. The maximum elevation change is taken as 50 meters. For the first The tourist preference matching degree of the route segment is calculated based on the tourist's historical behavior; the higher the preference, the lower the coefficient of the route segment. , , , These are weighting coefficients, set to 0.3, 0.3, 0.2, and 0.2 respectively. Visitors can customize the weighting allocation on the mobile device. The algorithm execution flow is as follows: First, based on the tourist's current location and target location, the start and end nodes of the route planning are determined. Second, the A* algorithm is used to search for the optimal path from the start point to the end point, with Euclidean distance as the heuristic function. Third, the searched path is smoothed to remove redundant nodes. Finally, the total path length, estimated tour time, list of points of interest along the way, and other information are calculated. The route planning result includes fields such as the path geometric coordinate sequence, path length, estimated time, list of attractions along the way, and description of key turning points. Once the route planning results are generated, the map linkage module performs multi-terminal synchronous updates to ensure consistency in the content displayed on the visitor's mobile device, the current device screen, and subsequent route markers. The multi-terminal synchronous update mechanism is as follows: First, the route planning results are sent to the visitor's mobile device, where navigation arrows and route lines are overlaid on the AR interface, and the route is highlighted on the electronic map. Second, the route planning results are sent to the current device, where the device screen highlights the route on the dynamic map and indicates the estimated arrival time. Third, all digital landscape markers on the subsequent route are queried, and the route information and estimated arrival time are pushed to these devices. Each device screen displays the direction and distance of the route after this node, realizing relay-style navigation from one station to another. The synchronous update adopts a publish-subscribe model. The map linkage module publishes the route planning results to a Redis message channel, and each subscriber updates its display content after receiving the message. The synchronization latency is controlled within 200 milliseconds.
[0074] 5. Crowdsourced Data Processing Module: This module is responsible for receiving, reviewing, and distributing on-site information reported by tourists, including facility repair requests, crowd congestion, scenic spot discoveries, and special activities. It updates the cloud database and triggers warning displays from identification devices and announcements from digital human-generated content. Tourists report events via mobile devices. The reported data packet includes the following fields: tourist identifier, event type, event location coordinates, event description text, image attachment URL, video attachment URL, reporting timestamp, mobile device location accuracy, and tourist credit score. Event types are divided into four categories: facility malfunctions (including blocked restrooms, damaged signs, and streetlight malfunctions); crowd congestion (including long queues and blocked passageways); scenic spot discoveries (including unexpected scenic views and photo spots); and special activities (including temporary performances and interactive activities). The reported data packet is submitted to the cloud via an HTTP interface. It first undergoes format verification and anti-replay checks. A single tourist can report a maximum of three events per minute; requests exceeding this limit return an error code. After verification, the data packet is stored in a message queue for asynchronous processing by the backend. The crowdsourcing data processing module performs spatiotemporal clustering on the received reported events to determine whether multiple tourists have reported the same event. The clustering conditions are defined as follows: if the location distance between two events is less than 20 meters and the reporting time difference is less than 5 minutes, they are considered to be the same event. The location distance is calculated using the Haversine formula. The calculation is as follows: , in, For the Earth's radius, take 6,371,000 meters; The clustering algorithm uses a sliding window method, sorting the event list by reporting time and traversing it in five-minute windows. Events within a window are compared pairwise, and events within a distance of less than 20 meters are grouped into the same cluster. The clustering result includes the event cluster identifier, the number of events within the cluster, the cluster center location, and the earliest and latest times of the cluster. For each event cluster, a fusion confidence score is calculated. As a criterion for determining the validity of an event, the confidence level is integrated. The calculation formula is as follows: , in, For the first The confidence level of each reported event is calculated based on the reporting tourist's credit score and the location accuracy at the time of reporting. The calculation formula is shown below. , in, This is a tourist credit score, ranging from zero to one. New tourists start with 0.5 points, gaining 0.05 points for each accepted reported incident and decreasing by 0.02 points for each rejected reported incident. For positioning accuracy scoring, a score of 1 is assigned when the positioning error is less than 5 meters, 0.8 is assigned when the error is between 5 and 10 meters, 0.6 is assigned when the error is between 10 and 20 meters, and 0.4 is assigned when the error is greater than 20 meters. and The weighting coefficients are set to 0.6 and 0.4 respectively. Fusion confidence For time Reduce weight, time The weight reduction is calculated using the following formula:
[0075] in, Current timestamp For the first The reporting timestamp of each event The decay factor is set to 0.005 per second, meaning it decays to [a certain value] after five minutes. ; When the number of events m in an event cluster is not less than three and the fusion confidence is... When the value is not less than 0.6, the event is considered effective. After the event is effective, the crowdsourced data processing module executes the following processing flow: First, the event information is written to the dynamic map database. For facility failure events, the state field of the corresponding facility is updated to "failure"; for pedestrian congestion events, the corresponding road segment is marked as "congested" in the road network data; for scenic spot discovery events, a temporary point of interest is added and marked as "recommended check-in point"; for special event events, an event record is added and marked as "in progress". Secondly, query all digital landscape signage devices within a 500-meter radius of the event location, generate warning instructions and push them to these devices. The warning instructions include fields such as event type, event location, event description, and display priority. After receiving the instructions, the devices display prompt information on the screen and the status indicator lights switch to flashing mode. Next, generate digital human broadcast content. Based on the event type and description, call the digital human interaction engine to generate voice broadcast text and facial expression parameters. The broadcast content templates are as follows: Facility failure: "[Facility Name] ahead is currently malfunctioning. We suggest going to the nearby [Alternative Facility]"; Crowd congestion: "There are many people ahead. The estimated queue time is [time] minutes. We suggest taking a detour"; Scenic spot discovery: "A tourist has discovered a stunning view at [Location]. Go check it out!"; Special event: "[Event Name] is underway at [Location]. Don't miss it!" Finally, the broadcast command is pushed to the voice playback queue of the surrounding devices, and the device's digital human broadcasts the command in priority order. At the same time, the event marker is pushed to the tourist's mobile device, and the corresponding icon is displayed on the AR interface and electronic map for other tourists to refer to.
[0076] Specifically, each event undergoes a complete lifecycle management process from reporting to expiration. After an event takes effect, its status is "valid" and it is valid for two hours. After the validity period ends, the event status changes to "expired" and is archived from the dynamic map database. It is no longer pushed to tourists entering the area. For facility failure events, once the facility status returns to normal, the scenic area management personnel can manually close the event through the backend, and the status changes to "resolved". The archived event data is used for subsequent data analysis and model training, including statistical indicators such as event type distribution, event heat map, and event response timeliness.
[0077] Specifically, the mobile terminal for tourists described in this embodiment of the invention runs on smartphones or tablets in the form of a scenic area-specific application, mini-program, or integrated terminal of a cultural tourism platform. The mobile terminal obtains the real-time location and posture of tourists through multimodal positioning technology, and uses digital landscape signage devices as visual anchors to achieve accurate overlay of AR content, providing functions such as remote interaction with digital humans, dynamic map navigation, event reporting, route sharing, and stamp collection.
[0078] Specifically, the mobile terminal for tourists consists of the following units: 1. Multimodal Positioning Unit: This unit integrates data from multiple sensors, including Global Navigation Satellite System (GNSS) positioning, electronic compass, accelerometer, and gyroscope. Combined with simultaneous visual positioning and mapping technology, it uses digital landscape markers as anchor points for precise positioning, addressing the issue of large positioning errors in areas with weak GNSS signals. The tourist mobile terminal acquires the following sensor data through the system interface: the GNSS positioning module provides latitude and longitude coordinates, altitude, positioning accuracy, speed, and orientation angle, updated once per second; the electronic compass provides the orientation angle of the device relative to the geomagnetic north pole, updated ten times per second; the accelerometer provides three-axis acceleration data, updated fifty times per second; and the gyroscope provides three-axis angular velocity data, updated fifty times per second. Sensor data fusion employs an extended Kalman filter algorithm, using GNSS positioning as the observed value and inertial measurement unit (INS) data as the state prediction value, outputting smooth position, velocity, and attitude estimates, and a state vector. Defined as: , in, For position coordinates, For speed, The pose is represented by a quaternion. For accelerometer deviation, This is due to gyroscope bias; In areas with weak global navigation satellite system signals, such as canyons, ancient building complexes, and indoor exhibition halls, the mobile device initiates a simultaneous visual positioning and mapping mode. Digital landscape markers serve as visual anchor points for precise positioning. The simultaneous visual positioning and mapping adopts a feature point-based implementation scheme, and the processing flow is as follows: First, initialize the camera intrinsic parameter matrix. Including focal length , and optical center coordinates , Camera intrinsic parameter matrix Represented as: ; Secondly, the mobile camera captures real-time images and extracts ORB feature points. Each feature point contains key point coordinates and a binary descriptor. When the QR code of a digital landscape signage device is detected in the image or the appearance features of the device are identified, the unique identification code of the device is extracted, and the three-dimensional coordinates Pdevice and the predefined anchor feature point coordinate set {Panchor} of the device are obtained from the local cache or the cloud. Next, a feature point matching algorithm is used to match the detected device feature points in the image with predefined anchor point feature points to obtain a 2D-3D corresponding point set. A perspective n-point algorithm is then used to solve for the pose transformation matrix of the camera relative to the device's local coordinate system. Finally, the camera pose and inertial measurement unit data are fused, and a stable device pose estimate is obtained through a tightly coupled nonlinear optimization method, outputting the device's six-degree-of-freedom pose in the current scene; The multimodal positioning unit evaluates the accuracy of the current positioning mode in real time and automatically switches the positioning mode based on the evaluation results. Evaluation indicators include the Global Navigation Satellite System (GNSS) signal strength, the number of visible satellites, the number of visual feature points, visual reprojection error, and the zero-bias stability of the inertial measurement unit (INS). When the GNSS positioning accuracy is better than five meters and the number of visible satellites is greater than eight, the GNSS-dominant mode is used, with simultaneous visual positioning and mapping (SMR) as an auxiliary correction. When the GNSS signal strength is below a threshold or the number of visible satellites is less than four, the system switches to the SMR-dominant mode, using the most recently identified digital landscape marker as the anchor point for positioning. A smooth transition is used when switching between the two modes to avoid abrupt changes in pose.
[0079] 2. AR Rendering Engine Unit: This unit is responsible for rendering AR content data packages sent from the cloud in real time. It overlays digital content such as 3D models, navigation indicators, and information floating windows onto the camera's real-world view, achieving a "what you see is what you know" augmented reality experience. The AR rendering engine unit adopts a layered rendering pipeline, comprising the following layers: The bottom layer is the camera image acquisition layer, which obtains the real-time image stream from the camera through the mobile terminal system interface. It supports switching between wide-angle, ultra-wide-angle, and telephoto lenses from the rear camera, and the image resolution is adapted to the device's screen resolution. The middle layer is the scene graph management layer, which manages all virtual objects in the AR scene, including 3D model nodes, particle system nodes, text label nodes, navigation indicator nodes, etc. Each node contains transformation matrices, material properties, rendering states, animation controllers, and other attributes. The top layer is the rendering execution layer, which submits rendering commands based on OpenGLES or Metal graphics APIs. The rendering loop executes the following steps each frame: acquiring the current camera image as the background texture; updating the transformation matrices of all virtual objects in the scene graph; performing lighting calculations and material shading; and compositing the rendering results with the camera image and outputting them to the screen. AR content resources are rendered according to unified specifications to ensure visual effects and performance. 3D model rendering uses physically based rendering materials, including diffuse maps, normal maps, metal roughness maps, and ambient occlusion maps. The number of vertices in each model does not exceed 50,000, the texture resolution does not exceed 2048×2048 pixels, and the size of a single model file does not exceed 10 megabytes. Models support skeletal animation and shape-key animation, with an animation frame rate of at least 30 frames per second. Navigation indicators are rendered using semi-transparent materials to maintain visual transparency with the real-world scene. Navigation arrows use 3D models, pointing in the direction of the path. The arrow size adapts to the viewing distance: 0.5 meters at a viewing distance of 5 meters, and 1.5 meters at a viewing distance of 20 meters. Path lines use a striped mesh model, extending along the path direction, with a line width of 0.2 meters, a bright cyan color, and 60% transparency. Information pop-ups use billboard technology, always facing the camera for easy reading by visitors. The pop-up background is a semi-transparent white rounded rectangle, the text is dark gray, the font size adapts to the screen resolution, and the minimum font size is no less than 14 points. The floating window moves with the target object. When the target object moves off the screen, the floating window displays a pointing arrow at the edge of the screen. The AR rendering engine unit supports virtual-real occlusion processing to ensure the correct occlusion relationship between virtual objects and the real scene. When an object in the real scene is in front of a virtual object, the occluded part of the virtual object should be invisible. Occlusion processing uses depth testing technology. First, a sparse depth map of the scene is constructed through simultaneous visual localization and mapping. Then, the depth map is upsampled and filtered to generate a dense depth map. Finally, when rendering the virtual object, the depth value of the virtual object is compared with the real depth value in the depth map. When the depth value of the virtual object is greater than the real depth value, the pixel is discarded to achieve the occlusion effect. For scenes where a depth map cannot be obtained, approximate occlusion processing based on semantic segmentation is used. A lightweight deep learning model performs real-time semantic segmentation on the camera image to identify foreground objects such as people, plants, and buildings. When the virtual object is rendered, collision detection is performed between it and the foreground object area. The area covered by the foreground object has reduced transparency or is hidden during rendering. The AR rendering engine unit employs a two-level caching strategy to reduce redundant downloads and improve loading speed. The local cache stores downloaded AR content resources, organized by device identifier and content version number. The cache capacity is capped at 500 megabytes; exceeding this limit results in the removal of older content using a least recently used algorithm. Cache content includes model files, texture files, animation files, configuration files, etc. The memory cache stores AR content resources needed for rendering the current scene, pre-loaded when entering a geofence, and released upon exiting the geofence. Pre-loading uses a priority queue, prioritizing content associated with the nearest device, followed by content associated with devices along the path planning route. When content is updated, the AR rendering engine unit requests the latest version number from the cloud, carrying the local cache version number. If the version numbers are inconsistent, the changed content is downloaded incrementally.
[0080] 3. Interactive Interface: The interactive interface provides access to functions such as AR navigation, attraction information query, one-click reporting, remote digital human interaction, route sharing, and stamp collection. The interface design follows mobile human-computer interaction standards, supporting one-handed operation and voice commands. The main interface uses a real-world camera view as the background, with AR content overlaid on top. A functional toolbar with a semi-transparent frosted glass effect is located at the bottom of the interface, containing the following function buttons: AR navigation mode switch, map mode switch, digital human dialogue, one-click reporting, and personal center. The status bar at the top of the interface displays the current location mode icon, global navigation satellite system signal strength, network connection status, battery level, and current time. When a visitor enters a geofence, a pop-up notification bubble appears below the status bar, displaying the name and brief information of nearby devices. The bubble disappears automatically after five seconds. A quick operation panel is located on the side of the interface, which can be swiped to access, containing quick access to nearby facilities search, popular attraction recommendations, today's activity previews, and my check-in records. When expanded, the operation panel occupies one-third of the screen width, with the remaining area displaying the real-world camera view. AR navigation mode is the primary interaction mode, overlaying navigation indicators onto the camera's real-world view. When activated, the AR rendering engine starts, rendering navigation arrows, path lines, and information pop-ups from the cloud in real time. During navigation, if the user deviates more than ten meters from the planned path, a prompt box pops up at the top of the interface, and the digital human simultaneously announces in voice, "You have deviated from the route; we have replanned the route for you." The system automatically replans the route from the current location to the target attraction, updating the AR navigation indicators. As the user approaches the target attraction, the navigation arrows gradually disappear, and the information pop-up window for the target attraction enlarges, displaying the attraction's name, description, distance, etc. A "Details" button appears below the information pop-up window; clicking it leads to the attraction's details page. The map mode offers two views: a 2D electronic map and a 3D oblique photogrammetric map. It supports zooming, rotation, and panning. The map displays the following elements: location and status of digital landscape markers, location and popularity of attractions, location and status of service facilities, real-time pedestrian heatmap, current location and orientation of visitors, and planned routes. Device icons display different styles depending on type and status: attraction nodes are displayed as pagoda icons, intersections as road sign icons, and service points as corresponding facility icons. Device icons are green when normal, red when malfunctioning, and flashing blue during interaction. The pedestrian heatmap uses a red-yellow-green gradient to represent pedestrian density: red indicates congestion, yellow indicates crowding, and green indicates smooth flow. Heatmap data is updated every five minutes. Dragging the map automatically requests heatmap data within the current viewport. The map mode supports route planning. Visitors can long-press on an attraction or facility on the map to bring up an operation menu, select "Set as Destination," and the system will automatically plan a route from the current location to the destination, displaying route length, estimated time, and attractions along the way. Tourists can choose "Start Navigation" to switch back to AR navigation mode, or "Share Route" to share the route with fellow tourists; The remote interactive function of the digital human allows tourists to engage in voice and visual conversations with the digital human via mobile devices without being physically near the device. The interactive interface features a full-screen immersive design, with the digital human model displayed as a half-body image in the lower half of the screen. The background uses a virtual environment or a blended real-world scene that matches the current scenic area. The top of the interface displays the current digital human's name and the scenic area's theme. Tourists can start voice input by clicking the microphone button or directly enter text to begin the conversation. During voice input, the interface displays an animated voice waveform, and the digital human adopts a listening posture. During the conversation, the digital human's facial expressions and lip movements are synchronized with the dialogue content, and its body movements are coordinated with the voice content. The digital human supports multi-turn dialogues and contextual understanding. For example, if a tourist asks "What are some good restaurants nearby?" and the digital human answers, and the tourist continues to ask "How far is it from here?", the system can recognize that "here" refers to the restaurant location mentioned in the previous answer. The dialogue history is saved locally, supporting dialogue rewind. The interactive interface provides quick access to ask questions, including buttons for commonly used intents such as "Recommended Routes," "Nearby Facilities," "Attraction Stories," and "Photo Check-in." Clicking these buttons will directly trigger the corresponding dialogue process.
[0081] 4. One-Click Reporting Unit: The one-click reporting unit allows tourists to quickly report on-site information. The interface design simplifies the operation process, and the reporting time is no more than 30 seconds. The one-click reporting unit entrance is located in the bottom toolbar of the main interface. After clicking, a reporting type selection panel will pop up, which includes four options: facility failure, crowd congestion, scenic spot discovery, and special activities. After selecting a type, the tourist's mobile device will automatically obtain the current location coordinates. Tourists can take photos or videos as attachments, or add text descriptions. After the system automatically fills in the reporting information, the tourist confirms and submits. After submission, a reporting record is generated and displayed in the "My Reports" list in the personal center. Tourists can view the reporting status, which includes "Under Review," "Accepted," and "Rejected." After acceptance, tourists will receive points as a reward. Points can be used to redeem cultural and creative products in the scenic area or to offset consumption.
[0082] 5. Route Sharing Unit: This unit allows tourists to share their tour routes with other tourists or on social media platforms. Shared content includes the route track, a list of attractions visited, tour duration, and photos taken at each stop. A sharing card is generated, containing a route thumbnail and a QR code. Other tourists can scan the QR code to load the same route and begin navigation.
[0083] 6. Stamp Collection Unit: This unit is automatically triggered via geofencing. Visitors automatically check in upon arriving at a scenic spot, requiring no manual operation. The check-in record includes the scenic spot name, check-in time, photo, and a description of the spot. Collecting a certain number of check-ins unlocks achievement badges, which are displayed on the user's profile page. The check-in record can be generated into a travel journal, arranging check-ins chronologically, accompanied by photos taken by the visitor and system-generated scenic spot descriptions. This journal can be exported as an image or saved as a PDF.
[0084] Specifically, the cultural tourism guidance method based on AR and digital landscape signage provided by this invention is executed collaboratively by the tourist's mobile terminal, digital landscape signage device, and cloud server. Through clearly defined interfaces and data formats, a complete closed loop of guidance service is formed.
[0085] Please see Figure 3 The diagram shown is a module connection schematic of a cultural tourism guide system based on AR and digital landscape signage collaboration provided in an embodiment of the present invention, including: The trigger event recognition module is used to determine whether a tourist has entered the fence around the digital landscape signage device based on the location data reported by the tourist's mobile terminal, and to mark the condition of the tourist entering the fence as a trigger event. The visual synchronization module is used to send AR content and digital human interaction data to the tourist's mobile terminal based on the triggering event, and at the same time send a screen content update instruction to the digital landscape sign device so that the digital landscape sign device can be updated to be consistent with the tourist's mobile terminal. The collaborative interactive control module is used to respond to the operation commands of tourists, and the digital landscape signage device is updated synchronously with the tourist's mobile terminal.
[0086] Specifically, this invention uses lightweight digital landscape markers as spatial anchors, combined with efficient marker code recognition and pose calculation algorithms, to achieve stable and accurate overlay of AR content with real landscapes. This enhances the realism and immersion of the user experience. Heavy computational tasks such as digital human rendering, voice interaction, and AR content generation are deployed in the cloud, with the terminal device only responsible for positioning, communication, and lightweight display. This reduces the hardware cost and power consumption of individual devices, facilitating large-scale deployment. Simultaneously, unified cloud management ensures consistency in content updates and services. By integrating real-time events and visitor trajectory data reported through crowdsourcing, the system can achieve event propagation, dynamic path planning and fence adjustment based on real-time pedestrian flow, and continuous analysis of attraction heatmaps and visitor preferences. This alleviates the problem of poor visitor continuity and low immersion caused by reliance on active visitor operation.
[0087] Please see Figure 4 As shown, it is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Figure 4 The electronic device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.
[0088] Specifically, the electronic device 400 includes a processor 410, a memory 420, and a communication interface 430, which are interconnected and communicate with each other via a communication bus 440 and / or other forms of connection mechanism (not shown).
[0089] The memory 420 includes one or more (only one is shown in the figure), which may be, but is not limited to, random access memory, read-only memory, programmable read-only memory, erasable programmable read-only memory, electrically erasable programmable read-only memory, etc. The processor 410 and other possible components may access the memory 420 to read and / or write data therein.
[0090] Processor 410 includes one or more (only one is shown in the figure), which can be an integrated circuit chip with signal processing capabilities. The processor 410 described above can be a general-purpose processor, including a central processing unit, microcontroller unit, network processor or other conventional processor; it can also be a special-purpose processor, including a digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component.
[0091] Communication interface 430 includes one or more (only one is shown in the figure) and can be used to communicate directly or indirectly with other devices to exchange data. For example, communication interface 430 can be an Ethernet interface; it can be a mobile communication network interface, such as an interface for 3G, 4G, or 5G networks; or it can be other types of interfaces with data transmission and reception functions.
[0092] One or more computer program instructions may be stored in the memory 420. The processor 410 may read and run these computer program instructions to implement the cultural tourism guide method based on AR and digital landscape signage collaboration provided in the embodiments of this application, as well as other desired functions.
[0093] Understandable. Figure 4 The structure shown is for illustrative purposes only; the electronic device 400 may also include components that are more advanced than those shown. Figure 4 The more or fewer components shown, or having the same Figure 4 The different configurations shown. Figure 4 The components shown can be implemented using hardware, software, or a combination thereof. For example, electronic device 400 can be a single server (or other device with computing power), a combination of multiple servers, a cluster of a large number of servers, etc., and can be either a physical device or a virtual device.
[0094] This application also provides a computer-readable storage medium storing computer program instructions. These instructions are read and executed by a computer's processor to perform the AR-based digital landscape signage-based cultural tourism guidance method provided in this application. For example, the computer-readable storage medium can be implemented as follows: Figure 4 The memory 420 in the electronic device 400.
[0095] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present application should be included within the scope of protection of the present application.
Claims
1. A cultural tourism guidance method based on the collaboration of AR and digital landscape signage, characterized in that, include: Step S1: The service device determines whether the tourist has entered the fence around the digital landscape signage device based on the location data reported by the tourist's mobile terminal, and marks the condition of the tourist entering the fence as a trigger event. In step S2, the service device sends AR content and digital human interaction data to the tourist's mobile terminal based on the triggering event, and at the same time sends a screen content update instruction to the digital landscape sign device so that the digital landscape sign device can be updated to be consistent with the tourist's mobile terminal. Step S3: In response to the tourist's operation command, the digital landscape signage device is updated synchronously with the tourist's mobile terminal.
2. The cultural tourism guidance method based on AR and digital landscape signage collaboration according to claim 1, characterized in that, It also includes a process of dynamic optimization based on the response results of the operation instructions, the process including: The service equipment continuously collects visitor behavior data, digital landscape signage device operation data, and visitor mobile terminal operation data. The preprocessed data is input into a Long Short-Term Memory (LSTM) network to predict future pedestrian flow heatmaps, and the fence radius is adjusted based on the prediction results. Represented as: , in, The radius of the fence. This is the ratio of the current pedestrian density to the predicted pedestrian density. This represents the historical average population density.
3. The cultural tourism guidance method based on AR and digital landscape signage collaboration according to claim 2, characterized in that, In step S1, based on distance smaller than the fence radius And absolute difference When a triggering event is determined to have occurred, the distance... The calculation formula is as follows: in, For the Earth's radius, and These are the latitude and longitude of the tourist's location, respectively. and These are the latitude and longitude of the location of the digital landscape signage device, respectively. The absolute difference The calculation formula is as follows: Calculate the azimuth angle from which a visitor points to the digital landscape signage device, the azimuth angle The calculation formula is as follows: , in, It is the arctangent function in the four quadrants; Calculate the direction of tourist movement absolute difference with azimuth .
4. The cultural tourism guidance method based on AR and digital landscape signage collaboration according to claim 3, characterized in that, Step S3 further includes verifying the authenticity of the operation command, the verification process including: Operation commands under preset conditions are grouped into the same cluster, and a fusion confidence score is calculated for event clusters within the same cluster. Represented as: , in, For the first Confidence level of each reported event It is a natural constant. This is the current timestamp. For the first The timestamp of each event report.
5. The cultural tourism guidance method based on AR and digital landscape signage collaboration according to claim 4, characterized in that, The service equipment is deployed in a data center or public cloud platform to receive and process real-time data from tourists' mobile devices and manage the status of digital landscape signage devices.
6. A cultural tourism guidance system based on the collaboration of AR and digital landscape signage, employing the cultural tourism guidance method based on the collaboration of AR and digital landscape signage as described in any one of claims 1-5, characterized in that, include: The trigger event recognition module is used to determine whether a tourist has entered the fence around the digital landscape signage device based on the location data reported by the tourist's mobile terminal, and to mark the condition of the tourist entering the fence as a trigger event. The visual synchronization module is used to send AR content and digital human interaction data to the tourist's mobile terminal based on the triggering event, and at the same time send a screen content update instruction to the digital landscape sign device so that the digital landscape sign device can be updated to be consistent with the tourist's mobile terminal. The collaborative interactive control module is used to respond to the operation commands of tourists, and the digital landscape signage device is updated synchronously with the tourist's mobile terminal.
7. An electronic device, characterized in that, include: A processor, a memory, and a communication bus, wherein the processor and the memory communicate with each other via the communication bus; The memory stores program instructions that can be executed by the processor, and the processor can execute the cultural tourism guide method based on the collaboration of AR and digital landscape signage as described in any one of claims 1 to 5 by calling the program instructions.
8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause the computer to execute the cultural tourism guidance method based on AR and digital landscape signage collaboration as described in any one of claims 1 to 5.