Vehicle-mounted travel event automatic recording and intelligent travelogue generation method and system

Abstract

The present application belongs to the technical field of smart tourism, and particularly relates to a vehicle-mounted travel event automatic recording and intelligent travel note generation method and system. The method comprises the following steps: obtaining geographical position coordinates in the process of vehicle driving, and triggering the position of a preset interest point based on the geographical position coordinates; verifying whether the current interest point is a travel event object by using a multi-modal data fusion mode; encapsulating the obtained multi-modal data as a structured travel data, and associating the structured travel data to the current interest point, so as to form an event database; at a set time point, reading the travel event object in the event database by using an intelligent agent, generating a travel note draft based on the structured travel data, and presenting the note draft to a user; obtaining the note content finally published by the user, and updating the user portrait. The present application realizes semantic-level understanding of a travel event and automatic and accurate perception of the travel event by using a double triggering mechanism of geographical fence preliminary screening combined with multi-modal verification.

Description

Technical Field

[0001] This invention belongs to the field of smart tourism technology, and in particular relates to a method and system for automatic recording of in-vehicle travel events and intelligent travelogue generation. Background Technology

[0002] With the increasing popularity of in-vehicle smart cockpits, users generate a large amount of location, video, and voice data during road trips. Existing technologies such as GPS-based location-based check-in apps require users to manually click a "check-in" button to record location and time, which is inconvenient and prone to omissions in driving scenarios. Some dashcams can automatically record short videos based on rapid acceleration and deceleration, but their triggering conditions are singular, only recording dynamic vehicle events and failing to correlate with travel semantics. Some mobile photo album apps can automatically categorize photos using image recognition and time and location information, but they process existing static photo libraries and cannot trigger real-time recording when events occur.

[0003] Therefore, the existing recording methods described above result in a lack of unified processing of events throughout the entire travel process. The recorded discrete event data is not effectively organized and does not form a data loop. Summary of the Invention

[0004] To overcome the shortcomings of the prior art, this invention provides a method and system for automatic recording of in-vehicle travel events and intelligent travelogue generation. Through a dual triggering mechanism of "geofencing screening + multimodal verification", the recording of travel events is triggered, notes are automatically generated based on the sequence of travel event objects, and the travel notes are applied in a closed loop to update user preferences. This solves the technical problems of passive travel recording methods, single perception dimension, weak content generation capabilities, and lack of data closed loop in the prior art.

[0005] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: The first aspect of this invention provides a method for automatically recording in-vehicle travel events and generating intelligent travelogues.

[0006] The method for automatically recording in-vehicle travel events and generating intelligent travelogues includes the following steps: Obtain the geographical coordinates of the vehicle during its journey, and trigger the preset points of interest based on the geographical coordinates; When the preset point of interest location is triggered, a multimodal data fusion method is used to verify whether the current point of interest is a travel event object; Once the current point of interest is determined to be a travel event object, the acquired multimodal data is encapsulated into structured travel data, associated with the current point of interest, and thus forms an event database. At a set time, the intelligent agent reads travel event objects from the event database and generates a draft of travel notes based on the structured travel data, presenting the draft notes to the user. Obtain the final published notes from users and update user profiles using these notes.

[0007] As a preferred technical solution, the following are also included: Acquire travel route data, which includes pre-planned point of interest coordinates and corresponding geofence radius parameters; When the distance between the real-time geographic location coordinates of the vehicle and the coordinates of any point of interest in the travel itinerary data is less than the corresponding geofence radius parameter, the preset point of interest location is triggered.

[0008] As a preferred technical solution, when the preset point of interest location is triggered, a multimodal data fusion method is used to verify whether the current point of interest is a travel event object, specifically including: The system acquires images of the current environment, extracts image features through a pre-trained convolutional neural network, and outputs recognition results including scene category labels and confidence scores. The system collects in-vehicle audio signals, extracts acoustic feature parameters of the audio including fundamental frequency, speech rate and energy, and analyzes the acoustic feature parameters through a recurrent neural network to output emotion category and confidence level. At the same time, the audio is converted into text and keywords related to the travel scenario are extracted. The confidence scores of scene category labels, emotion category labels, and keywords are weighted and calculated to determine whether the event is a travel event.

[0009] As a preferred technical solution, the scene category label is a travel-related category, which includes at least one of natural landscapes, historical buildings, animals, and distinctive plants; The emotion categories include at least one of the following: joy, excitement, calmness, and sadness.

[0010] As the preferred technical solution, the overall score = w1 × scene_confidence_max + w2 × emotion_confidence + w3 × keyword_score; Where w1, w2, and w3 are preset weight coefficients; scene_confidence_max is the highest confidence value of the scene label; emotion_confidence is the emotion confidence value; and keyword_score is the keyword semantic relevance score.

[0011] As a preferred technical solution, the acquired multimodal data is encapsulated into structured travel data, linked to the current point of interest, and thus forms an event database, specifically including: The multimodal data within a preset time window before and after the trigger time is encapsulated into structured travel event data containing metadata; The multimodal data includes geographic location coordinates, timestamps, key images, audio clips, and vehicle status data; The structured travel event objects are associated with corresponding points of interest by matching geographic coordinates and timestamps, and then stored in the event database.

[0012] As a preferred technical solution, the structured travel event object is associated with a corresponding point of interest by matching geographic location coordinates and timestamps, specifically as follows: Based on geographic coordinates and timestamps, retrieve the nearest and most recent point of interest nodes in the travel itinerary; Establish the association between structured travel event objects and points of interest nodes, and record the association in the event database.

[0013] As a preferred technical solution, at a set time point, an intelligent agent reads travel event objects from the event database and generates a draft travel note based on structured travel data, specifically including: The large language model is invoked when the trip ends or a rest stop is reached. The large language model reads all structured travel event object sequences of this trip from the event database, and combines them with the event types automatically classified according to scene tags, user history preferences read from the user profile database, and interest point information obtained from the pre-stored interest point database to generate a travel note draft containing timeline, location, and experience description. The travel notes draft is presented on the human-computer interaction interface for users to edit.

[0014] As a preferred technical solution, the process of generating travel note drafts using a large language model includes: Arrange the structured travel event object sequence in chronological order and extract the location, time, scene tag, emotion category and keywords for each travel event object; According to the predefined template format, the location, time, scene description and user emotional state are filled into the corresponding fields of the template in sequence to form structured natural language prompts; Based on the user's preset note template type, select the corresponding text generation style parameters from the template library; The text generation style parameters and natural language prompts are input into a large language model to generate coherent text content.

[0015] As a preferred technical solution, the final published note content of the user is obtained, and the user profile is updated using the final published note content, specifically including: After the user confirms the posting, the final posted note content is obtained and natural language processing is performed. Extract frequently occurring scene categories, interest point types, and emotional keywords, and calculate the weights of each frequently occurring scene category, interest point type, and emotional keyword. The weights of frequently occurring scene categories, interest types, and emotional keywords are used as user preference features to update the user profile model. Based on the updated user profile model, highly matching points of interest are retrieved from the point of interest database, matching scores are calculated and sorted from high to low, and the top-ranked points of interest are recommended to the user's future travel plans.

[0016] As a preferred technical solution, the encapsulation method for multimodal data is as follows: Based on a unified timestamp, geographic coordinates, key images, audio clips, and vehicle status data are synchronized and aligned in time, and then encapsulated into a JSON data object.

[0017] The second aspect of this invention provides a method for automatically recording in-vehicle travel events and generating intelligent travelogues.

[0018] Methods for automatic recording of in-vehicle travel events and intelligent travelogue generation include: The triggering module is configured to: acquire the geographic coordinates of the vehicle during its driving process, and trigger the preset point of interest location based on the geographic coordinates; The verification module is configured to: when the preset point of interest location is triggered, use multimodal data fusion to verify whether the current point of interest is a travel event object; The data encapsulation module is configured to: when the current point of interest is determined to be a travel event object, encapsulate the acquired multimodal data into structured travel data, associate it with the current point of interest, and thus form an event database; The note draft generation module is configured to: at a set time point, use an intelligent agent to read travel event objects from the event database, generate travel note drafts based on structured travel data, and present the note drafts to the user; The user profile update module is configured to: obtain the content of the user's last published notes and update the user profile using the content of the last published notes.

[0019] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of the method for automatic recording of vehicle travel events and intelligent travelogue generation as described in the first aspect of the present invention.

[0020] A fourth aspect of the present invention provides an electronic device including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method for automatic recording of in-vehicle travel events and intelligent travelogue generation as described in the first aspect of the present invention.

[0021] The above one or more technical solutions have the following beneficial effects: This invention provides a method and system for automatic recording of in-vehicle travel events and intelligent travelogue generation. Through a dual-trigger mechanism combining geofencing for initial screening and multimodal verification, it achieves automatic and accurate perception of travel events. Simultaneously, it achieves semantic-level understanding of travel events. Compared to existing technologies that rely solely on GPS location information or vehicle dynamics information, this invention extracts visual scene features through convolutional neural networks, analyzes audio emotional features through recurrent neural networks, and obtains semantic information through keyword extraction. This achieves multi-dimensional fusion perception of location, vision, hearing, and semantics, and the effective fusion and complementarity of multimodal information, significantly improving the accuracy and robustness of travel event judgment. Finally, it achieves automatic conversion from fragmented multimodal data to structured travel notes, extracts preference features from user-posted notes to update user profile models, and optimizes future travel itinerary recommendations, realizing a complete data loop from recording to recommendation.

[0022] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0023] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0024] Figure 1 This is a flowchart of the method in Example 1.

[0025] Figure 2 This is a complete flowchart of the automatic perception and recording of multimodal travel events in Example 1.

[0026] Figure 3 This is a flowchart of the intelligent generation of travel notes based on recorded events, as shown in Example 1. Detailed Implementation

[0027] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0028] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0029] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0030] I. Terminology Explanation To facilitate understanding of the technical solution of this invention, the key terms involved in this application will first be explained and described: Geofencing, in a geographic information system (GIS), refers to a virtual boundary established around a specific geographical location. In this invention, a geofence is a virtual area centered on the coordinates of a point of interest (POC) and encompassing a configurable radius (e.g., 50 meters, 100 meters, 150 meters). When the geographic coordinates output by the vehicle's GPS module enter this area, the system triggers an event detection process. Based on high-precision GPS positioning and geospatial calculation algorithms, geofencing technology can determine in real time whether a vehicle has entered the preset area.

[0031] A travel itinerary is a document pre-planned by the user before a trip, including but not limited to: departure point, destination, coordinates of points of interest along the way, estimated arrival time, duration of stay, and type of activity. The travel itinerary in this invention is stored in a structured data format (such as JSON or XML), and each point of interest node includes attributes such as name, coordinates, category label, and geofence radius. Travel itineraries can be created manually by the user or automatically generated by the system based on user history preferences and collaborative filtering algorithms.

[0032] The travel event object refers to the structured data object used in this invention to encapsulate a single travel event. This object contains the following core fields: event ID, trigger timestamp, geographic coordinates, scene tags (such as "natural landscape" and "historical building"), key image file paths, audio clip file paths, sentiment category and confidence level, keyword list, vehicle status data, and associated route node IDs. The travel event object is stored in JSON format for easy retrieval, analysis, and content generation.

[0033] Multimodal perception refers to the technology of integrating data from multiple sensors for comprehensive analysis and understanding. The multimodal perception in this invention includes: (1) visual perception: acquiring environmental images through an in-vehicle camera and extracting scene features using a convolutional neural network; (2) auditory perception: acquiring audio signals through an in-vehicle microphone, extracting acoustic features, and performing emotion analysis; (3) location perception: obtaining geographical coordinates through a GPS module; and (4) motion perception: acquiring state data such as speed and acceleration through the vehicle's CAN bus. The core of multimodal perception lies in aligning and fusing heterogeneous data under a unified timestamp to achieve semantic-level understanding of travel events.

[0034] User profile: refers to a digital model constructed based on users' historical behavior data, describing users' interests, preferences, and characteristics. The user profile in this invention includes the following dimensions: (1) preference weights for interest category; (2) preference for emotion keywords; (3) travel style tags; (4) preference for note template types; (5) preference for length of stay, etc. The user profile extracts features from users' travel notes using natural language processing technology and is dynamically updated using a weighted calculation method to optimize future interest recommendations and content generation.

[0035] II. Technical Background and Problem Definition With the increasing popularity of in-vehicle smart cockpits and dashcams, users generate a large amount of data during road trips. Existing related technologies mainly include: 1. GPS-based location-based check-in applications, such as some sports or travel mobile apps. Users need to manually click the "check-in" button to record their location and time after arriving at the destination. This method relies on active user input, which is inconvenient and prone to errors, especially when driving.

[0036] 2. Dashcams and Simple Event Recording. Some high-end car models or external devices can automatically record short videos based on the vehicle's rapid acceleration or deceleration. However, their triggering conditions are singular, recording only dynamic vehicle events and cannot be associated with travel-related semantics (such as "arriving at a scenic spot" or "discovering beautiful scenery").

[0037] 3. Smart photo categorization and captioning applications based on photo albums. Some mobile photo album apps can automatically categorize photos and even generate simple captions using image recognition and time and location information. However, they process existing static photo libraries, cannot trigger real-time recording when events occur, and lack deep integration with specific trip planning.

[0038] The aforementioned existing technology has the following key defects: 1. Passive recording and fragmented experience: Existing check-in or recording methods require active user intervention (manual clicking, photo selection afterward), which interrupts the immersive travel experience and is independent of core in-vehicle services such as itinerary planning and navigation guidance.

[0039] 2. Limited perceptual dimensions and lack of semantic understanding: Existing technologies either rely solely on GPS (prone to accidental triggering), rely solely on vehicle dynamics (irrelevant to travel points of interest), or only process static images. They lack the ability to integrate multimodal perception capabilities that combine location, vision, time, audio, and travel context, making it impossible to accurately understand "what travel events are happening."

[0040] 3. Weak content generation capabilities: Existing solutions stop at simple recording or categorization, and cannot automatically synthesize fragmented multimodal data (a photo, a location, a sentence) into structured travel notes or travelogues that are rich in emotion and information, thus failing to lower the creative threshold for users to share.

[0041] 4. Lack of data closure: The recorded discrete event data is not effectively organized and cannot be fed back to the trip planning system for optimizing user profiles and future recommendations, and the data value is not fully explored.

[0042] The logical starting point of this invention is as follows: by constructing a dual triggering mechanism of "geofencing initial screening + multimodal verification", the automatic and accurate perception of travel events is achieved; by encapsulating multimodal data into structured travel event objects and associating them with travel itinerary nodes, the structured storage and organization of travel data is achieved; by calling a large language model intelligent agent and combining it with prompting engineering technology, the automatic conversion from fragmented data to high-quality travel notes is achieved; and by extracting preference features from user-posted notes and updating the user profile model, a complete data closed loop from recording to recommendation is achieved.

[0043] Based on the above logical starting point, this invention aims to solve the problems of passive, singular, and disconnected travel records from the experience in existing technologies, specifically by addressing the following: 1. How to automatically and accurately detect and record meaningful travel events without relying on users' active operation.

[0044] 2. How to integrate data from multiple vehicle-mounted sensors to achieve semantic-level understanding of travel events such as "arrival," "viewing," and "experience."

[0045] 3. How to intelligently generate high-quality, shareable structured travel content from automatically recorded fragmented event data.

[0046] 4. How to link automatically recorded content with the itinerary planning system to form a personal travel memory bank and a closed loop of preference evolution.

[0047] III. Overview of Technical Implementation Approach

[0048] This invention proposes a method for automatically recording in-vehicle travel events and generating intelligent travelogues by integrating geofencing and visual recognition. The overall technical process includes the following five core steps: First, data acquisition of travel itineraries and configuration of geofencing.

[0049] Before or during a trip, the system acquires travel itinerary data containing pre-planned point-of-interest coordinates and corresponding geofence radius parameters. The geofence radius parameters can be manually set by the user or dynamically recommended by the system based on the user's historical travel records and preferences using a collaborative filtering algorithm.

[0050] Second, continuous monitoring of data from multiple onboard sensors.

[0051] During vehicle operation, the onboard controller continuously monitors the data streams from the onboard GPS module, onboard camera, onboard microphone, and vehicle CAN bus to acquire real-time geographic coordinates, environmental images, in-vehicle audio, and vehicle status data.

[0052] Third, geofence triggering and multimodal verification.

[0053] When the distance between the geographic coordinates output by the vehicle GPS module and the coordinates of any point of interest in the travel itinerary data is less than the corresponding geofence radius parameter, the multimodal verification process is triggered.

[0054] The process includes: controlling the vehicle's camera to capture images of the current environment, extracting image features through a pre-trained convolutional neural network, and outputting scene category labels and confidence scores; controlling the vehicle's microphone to capture in-vehicle audio signals, extracting acoustic feature parameters (fundamental frequency, speech rate, energy), analyzing and outputting emotion categories and confidence scores through a recurrent neural network, and simultaneously performing speech recognition to convert to text and extract keywords. A fusion decision engine performs a weighted calculation of scene label confidence scores, emotion category confidence scores, and keywords. When the overall score exceeds a preset threshold, the current event is determined to be a valid travel event.

[0055] Fourth, encapsulation and storage of structured travel event objects.

[0056] When a travel event is determined to be valid, the system automatically encapsulates the multimodal data (geographic coordinates, timestamps, key images, audio clips, and vehicle status data) within a preset time window before and after the triggering time into a structured travel event object containing metadata. After matching the geographic coordinates and timestamps with the corresponding point of interest nodes in the currently executed travel itinerary, the event is stored in the event database.

[0057] Fifth, intelligent travelogue generation and closed-loop learning.

[0058] Upon completion of the trip or arrival at a rest stop, the system invokes a large language model based on the Transformer architecture to read all structured travel event object sequences from the event database for this trip. Combining event type, user history preferences, and points of interest, it generates a draft travel note containing a timeline, location, experience description, and selected media. After the user edits, confirms, and publishes the note, the system performs natural language processing on the note text, extracts user preference features to update the user profile model, and optimizes future travel itinerary recommendations based on the updated profile.

[0059] The core difference between this invention and existing technologies lies in: (1) The dual triggering mechanism of "geofence screening + multimodal verification" significantly improves the accuracy of event judgment compared with the single GPS triggering method; (2) By integrating multi-source sensor data from vehicle GPS, camera, microphone and vehicle bus, semantic-level understanding of travel events was achieved; (3) Through large language models and prompting engineering technology, the automatic conversion from fragmented multimodal data to structured travel notes was realized; (4) By extracting preference features from user-posted notes and updating the user profile model, a complete data loop from recording to recommendation is formed.

[0060] Example 1 The following is combined Figure 1 , Figure 2 and Figure 3 This document describes in detail the specific implementation process of a method for automatically recording in-vehicle travel events and generating intelligent travelogues that integrates geofencing and visual recognition, according to the present invention. The method includes the following steps: Step S101: Obtain travel itinerary data.

[0061] In this step, travel route data is obtained, which includes pre-planned point of interest coordinates and corresponding geofence radius parameters.

[0062] Travel itinerary data refers to structured data created by users before or during their trips, containing itinerary arrangements and points of interest (POIs). Each POI node includes the following fields: POI ID; name; geographic coordinates; category tags such as "natural landscapes," "historical buildings," and "restaurants"; description information; and geofence radius parameter. The geofence radius parameter defines the radius of a virtual circular area centered on the POI coordinates; an event detection process is triggered when a vehicle enters this area.

[0063] The sources of travel itinerary data include, but are not limited to: (1) Travel itineraries created manually by users through the vehicle central control screen or mobile app. Users can mark points of interest on the map and set the geofence radius; (2) Travel itineraries automatically recommended by the system based on users' historical travel records and preferences through collaborative filtering algorithms. The system reads the user's point of interest category preference weights from the user profile database, retrieves points of interest with high matching degree from the point of interest database, calculates the matching degree score and sorts them from high to low according to the score, and combines the top-ranked points of interest into recommended travel itineraries; (3) Travel itineraries imported by users from third-party travel platforms. The system obtains the travel itinerary data through API interface or file import method and converts it into the data format of this system.

[0064] This step, by acquiring travel route data in advance, can determine which points of interest the user plans to visit. This allows the event detection process to be proactively triggered when the vehicle approaches these points of interest, avoiding false or missed triggers caused by relying solely on GPS coordinates.

[0065] By supporting both manual user setup and automatic system recommendations to obtain travel itinerary data, this step not only meets the user's need to travel according to a pre-planned itinerary but also intelligently recommends potentially interesting locations based on the user's historical behavior, improving the system's adaptability and intelligence. By configuring a geofence radius parameter for each point of interest, the system can accurately trigger event detection when a vehicle approaches the point of interest, significantly reducing computational resource consumption and false trigger rate compared to traditional global GPS monitoring methods.

[0066] Step S102: Continuously monitor data from the vehicle's multi-source sensors.

[0067] In this step, while the vehicle is in motion, the data streams from the vehicle's GPS module, camera, microphone, and CAN bus interface are continuously monitored by the vehicle controller.

[0068] Vehicle-mounted multi-source sensor data refers to heterogeneous data streams from various sensors integrated into a vehicle, including location data, image data, audio data, and vehicle status data.

[0069] The vehicle controller continuously monitors data from multiple vehicle sensors in the following ways: (1) Data acquisition; (2) Time synchronization: Since different sensors have different sampling frequencies and data output delays, the vehicle controller uses a time synchronization algorithm to ensure that all sensor data are aligned on a unified time axis; (3) Data preprocessing: Preprocess the collected raw data, including: coordinate transformation and filtering of GPS data; distortion correction and brightness normalization of camera images; noise reduction and gain control of microphone audio; unit conversion and validity verification of CAN bus data; (4) Data caching: The preprocessed data is stored in the corresponding circular buffer.

[0070] This step outputs a preprocessed multi-source sensor data stream stored in the corresponding circular buffers of each sensor. Each data record includes a timestamp, sensor ID, data type identifier, and data content fields. This data is used in subsequent steps for geofencing trigger determination, multimodal verification, and event object encapsulation.

[0071] By continuously monitoring the data streams from the vehicle's GPS module, cameras, microphones, and CAN bus, the system can acquire multi-dimensional information in real time, including vehicle location, surrounding environment, in-vehicle audio, and vehicle status, providing a rich data foundation for accurate perception of travel events. Through the use of high-precision sensors and efficient data acquisition and preprocessing processes, the system can reliably acquire high-quality multi-source sensor data during vehicle operation, providing a reliable guarantee for subsequent multimodal verification and fusion decision-making.

[0072] Step S103: Geofencing Triggering and Multimodal Validation.

[0073] In this step, the multimodal verification process is triggered when the distance between the geographic coordinates output by the vehicle GPS module and the coordinates of any point of interest in the travel itinerary data is less than the corresponding geofence radius parameter.

[0074] Geofence triggering refers to the mechanism that initiates the event detection process when the geographic coordinates of a vehicle enter a pre-defined geofence area. Multimodal verification refers to the process of secondary confirmation of triggered events by integrating information from multiple modalities such as visual recognition and audio analysis, in order to improve the accuracy of event determination.

[0075] The input data for this step includes: (1) the current geographic coordinates of the vehicle, which are from the real-time output of the vehicle GPS module; (2) the coordinates of all points of interest and the geofence radius parameters in the travel route data, which are from step S101; (3) the current environmental image, which is from the real-time acquisition of the vehicle camera; and (4) the current in-vehicle audio, which is from the real-time acquisition of the vehicle microphone.

[0076] This step includes the following sub-processes: (1) Geofencing trigger judgment: The vehicle controller periodically reads the latest geographic coordinates from the GPS data buffer, traverses all points of interest in the travel route data, and calculates the spherical distance between the vehicle coordinates and the coordinates of each point of interest.

[0077] If the calculated spherical distance is less than the geofence radius parameter corresponding to the point of interest, the vehicle is determined to have entered the geofence of the point of interest, triggering the multimodal verification process.

[0078] (2) Visual recognition verification: When the geofence triggering conditions are met, the vehicle controller immediately controls the vehicle camera to collect the current environment image.

[0079] The specific operation is as follows: the latest frame image is read from the camera image buffer (or multiple consecutive frames are averaged), and the image is input into a pre-trained convolutional neural network model for scene recognition. The convolutional neural network model adopts a lightweight architecture, such as MobileNetV3 or EfficientNet-Lite, and performs inference computation on the vehicle's GPU, with a single frame inference time of no more than 100ms.

[0080] The model outputs scene category labels and corresponding confidence scores. Scene category labels include, but are not limited to: natural landscapes (such as mountains, lakes, seas, forests, grasslands), historical buildings (such as ancient pagodas, temples, city walls, bridges), animals (such as cattle, sheep, horses, birds), distinctive plants (such as cherry blossoms, lotus flowers, ginkgo), and urban landscapes (such as high-rise buildings, squares, streets).

[0081] The system filters scene labels with a confidence level greater than a preset threshold (e.g., 0.6 to 0.9) from the output results and determines whether these labels belong to travel-related categories loaded from the pre-stored category library during system initialization. Travel-related categories are defined by the system administrator or user and typically include categories closely related to the travel experience, such as natural landscapes, historical buildings, animals, and distinctive plants, while excluding categories unrelated to the travel experience, such as city roads, traffic signs, and ordinary buildings.

[0082] (3) Audio emotion analysis verification: In parallel with visual recognition, the vehicle controller controls the vehicle microphone to collect in-vehicle audio signals.

[0083] The specific operation involves reading an audio segment from the microphone audio buffer, lasting 5 to 10 seconds before and after the trigger moment, and extracting acoustic features from the audio segment. The acoustic features include: (a) Fundamental frequency: Reflects the pitch of the speaker's voice, extracted by autocorrelation or cepstral method, in Hz; (b) Speech rate: Reflects the speed of the speaker's speech, calculated by syllable detection algorithm, in syllables / second; (c) Energy: Reflects the loudness of the audio signal, obtained by calculating the short-time energy of the audio frame, in dB.

[0084] The extracted acoustic feature sequences are input into a recurrent neural network model (such as LSTM or GRU) for sentiment analysis. The recurrent neural network model performs real-time analysis on an onboard DSP chip, with a processing latency of no more than 500ms. The model outputs the sentiment category and its corresponding confidence score. Sentiment categories include: joy, excitement, calmness, sadness, anger, and fear.

[0085] Speech recognition employs deep neural network acoustic models (such as DeepSpeech and Conformer) and language models (such as n-gram and Transformer), achieving an accuracy rate of no less than 90%.

[0086] The identified text content is used to extract keywords. The keyword extraction uses the TF-IDF algorithm or a BERT-based keyword extraction model to extract keywords related to the travel scenario, such as "lamb", "beautiful scenery", "fun", and "happy".

[0087] The multimodal verification results output in this step include: (1) Visual recognition results: a list of scene category labels and their corresponding confidence scores, for example, [{"label": "grass", "confidence": 0.92}, {"label": "sheep", "confidence": 0.87}]; (2) Audio sentiment analysis results: sentiment category and confidence level, for example {"emotion": "joy", "confidence": 0.8}; (3) Keyword list: Keywords extracted from the speech recognition text, such as ["lamb", "feeding", "happy"]. These results are passed to the fusion decision engine for comprehensive judgment.

[0088] This step addresses the issue of further improving the accuracy of event determination through multimodal information fusion. Relying solely on GPS geofencing carries the risk of false triggers (e.g., a vehicle merely passes near a point of interest without actually entering or stopping). By adding visual recognition and audio emotion analysis verification, the system can confirm that the current environment indeed matches the scene characteristics of the point of interest, and that the occupants of the vehicle genuinely exhibit positive emotional responses, thereby significantly reducing the false trigger rate.

[0089] By performing inference calculations for the visual recognition model on the vehicle's GPU, real-time performance and accuracy are ensured. By setting confidence thresholds and travel-related category restrictions, the accuracy of event determination is ensured, avoiding misclassification of non-travel-related scenarios such as ordinary roads and traffic signs as valid events.

[0090] By performing audio sentiment analysis on an in-vehicle DSP chip, real-time processing of in-vehicle audio signals is achieved. By identifying emotion categories and extracting keywords, the system can capture the user's emotional state and semantic information, providing crucial auxiliary judgment for fusion decision-making.

[0091] Actual testing showed that, compared to a single GPS triggering method, adding multimodal verification improved event capture accuracy by approximately 80% and reduced false triggering rate by approximately 70%.

[0092] Step S104: Fusion decision engine makes a judgment In this step, the scene label confidence, emotion category confidence, and keywords output in step S103 are weighted and calculated by the fusion decision engine. When the comprehensive score is greater than the preset threshold, the current event is determined to be a valid travel event.

[0093] The input data for this step includes: (1) scene labels and confidence scores in the visual recognition results; (2) emotion categories and confidence scores in the audio emotion analysis results; (3) a list of keywords extracted from the speech recognition text; and (4) preset fusion decision rules and threshold parameters.

[0094] The fusion decision engine makes its judgments based on the following rules: (1) Scene label confidence judgment: Filter scene labels with a confidence score greater than a preset threshold (e.g., 0.7) and belonging to the travel-related category from the visual recognition results. If there is at least one scene label that meets the condition, the scene judgment is passed, and it is recorded as scene_pass = True. The highest confidence score value scene_confidence_max is also recorded.

[0095] (2) Emotion category confidence judgment: Determine whether the emotion category is a positive emotion (joy, excitement, happiness, etc.) from the audio emotion analysis results, and whether the confidence is greater than the preset threshold (e.g., 0.6). If the condition is met, the emotion judgment is passed, recorded as emotion_pass = True, and the emotion confidence value emotion_confidence is recorded.

[0096] (3) Keyword semantic relevance judgment: Calculate the semantic relevance between keywords in the keyword list and scene tags. The specific method is as follows: convert keywords and scene tags into word vectors respectively, and calculate the cosine similarity between word vectors. If there is at least one keyword in the keyword list whose cosine similarity with the scene tag is greater than a preset threshold (such as 0.5), then the keyword is determined to be semantically related to the scene tag, and is recorded as keyword_related = True.

[0097] (4) Calculation and Judgment of Overall Score: Calculate the overall score based on the above three judgment results. The formula for calculating the overall score is: Overall score = w1 × scene_confidence_max + w2 × emotion_confidence + w3 × keyword_score Where w1, w2, and w3 are preset weight coefficients (satisfying w1 + w2 + w3 = 1), scene_confidence_max is the highest confidence value of the scene label, emotion_confidence is the emotion confidence value, and keyword_score is the keyword semantic relevance score (if keyword_related = True, then keyword_score = 1.0, otherwise keyword_score = 0).

[0098] Typical weighting values ​​are: w1 = 0.5, w2 = 0.3, w3 = 0.2, indicating that the scene tag confidence has the highest weight, followed by sentiment confidence, and the keyword semantic relevance has the lowest weight. When the overall score is greater than a preset threshold (e.g., 0.7), the current event is determined to be valid, denoted as event_valid = True.

[0099] The output of this step is a boolean value `event_valid` (True indicates a valid event, False indicates an invalid event) and a comprehensive score. If `event_valid = True`, the event object encapsulation process in step S105 is triggered; if `event_valid = False`, the temporary data triggered this time is discarded, and the process returns to step S102 to continue monitoring.

[0100] This step effectively integrates the multimodal verification results and makes an accurate judgment. By integrating information from three dimensions—visual recognition confidence, sentiment analysis confidence, and keyword semantic relevance—and employing a weighted calculation method, the system can cross-validate the validity of events across multiple dimensions, significantly improving the accuracy and robustness of the judgment.

[0101] By employing a fusion decision engine to weight and calculate scene label confidence, emotion category confidence, and keyword semantic relevance, the system effectively integrates and complements multimodal information, significantly improving the accuracy of event determination. Actual testing shows that the event determination accuracy after adopting fusion decision-making reaches over 92%, an improvement of approximately 30% compared to single-modal judgment (such as relying solely on GPS or visual recognition). By setting a comprehensive score threshold (e.g., 0.7) and optimization rules (such as time persistence verification and multi-frame image verification), the system effectively avoids false positives and false negatives, ensuring that only truly meaningful travel events are recorded.

[0102] Step S105: Encapsulation and storage of structured travel event objects In this step, when a valid travel event is determined, the multimodal data within a preset time window before and after the trigger time is automatically encapsulated into a structured travel event object containing metadata. This object is then associated with the corresponding point of interest node in the currently executed travel itinerary by matching geographic coordinates and timestamps and stored in the event database.

[0103] This step includes the following sub-processes: (1) Data extraction within the time window: According to the preset time window parameters, data within the corresponding time range is extracted from the circular buffer of each sensor.

[0104] (2) Multimodal data time synchronization alignment: Since different sensors have different sampling frequencies and data output delays, it is necessary to perform time synchronization alignment on the extracted multimodal data. The specific method is: using a unified timestamp as a benchmark, normalize the timestamps of all data records to ensure the consistency of different modal data in the time dimension.

[0105] (3) Structured data object encapsulation: The time-synchronized and aligned multimodal data is encapsulated according to a predefined data structure to form a structured travel event object. The data object adopts JSON format.

[0106] (4) Link to travel route nodes: Based on the geographical coordinates and timestamps in the event object, retrieve the nearest and most recent point of interest nodes in the travel route data, and establish the association between the event object and the point of interest nodes.

[0107] The specific method is as follows: (a) Calculate the spherical distance between the geographic coordinates of the event object and the coordinates of all points of interest in the travel itinerary (using the Haversine formula); (b) Filter points of interest whose distance is less than the radius of the geofence; (c) If the filtering results contain multiple points of interest, select the point of interest with the closest time based on the timestamp (i.e., the point of interest whose expected arrival time is closest to the trigger time); (d) Fill the selected point of interest ID and name into the poi_id and poi_name fields of the event object to establish the association.

[0108] (5) Store to event database: Store the encapsulated structured travel event object to the event database.

[0109] This step outputs a structured travel event object stored in the event database. This object contains multimodal data and its metadata within a preset time window before and after the triggering time, and is associated with the corresponding point of interest nodes in the travel itinerary. The event object can be used for intelligent travelogue generation in subsequent steps.

[0110] This step involves structuring and storing fragmented multimodal data. Time synchronization based on a unified timestamp ensures consistency across different modalities in terms of time. Encapsulation using JSON format facilitates subsequent storage, retrieval, and processing, improving data scalability and interoperability. Dual matching of geographic coordinates and timestamps ensures accurate association between event objects and travel itinerary nodes, enabling recorded events to accurately reflect the user's travel trajectory and itinerary planning.

[0111] By encapsulating multimodal data within a preset time window before and after the trigger moment into structured travel event objects, the system achieves structured storage and organization of travel data. By setting appropriate time window parameters, the system can capture complete contextual information before and after the event, including the user's anticipation as they approach a point of interest, their excitement upon arrival, and their reflections upon departure, making the encapsulated event objects richer and more complete. By using a quality assessment algorithm based on image sharpness metrics and composition scores to select key images, the system ensures that the encapsulated images have high clarity and good composition, improving the visual effect of subsequently generated notes. By associating event objects with travel itinerary nodes, the system achieves deep integration of event data and itinerary planning, providing a structured data foundation for subsequent note generation and data analysis.

[0112] Step S106: Intelligent Content Generation Using a Large Language Model In this step, at the end of the trip or upon arrival at a rest stop, the large language model based on the Transformer architecture is invoked to read all the structured travel event object sequences of this trip from the event database. Combining event type, user history preferences, and points of interest information, a draft travel note containing a timeline, location, experience description, and selected media is generated.

[0113] This step includes the following sub-processes: (1) Event object sequence reading and sorting: Read all structured travel event objects for this trip (identified by the travel itinerary ID) from the event database, sort them in ascending order according to the trigger_time timestamp, and form an event object sequence.

[0114] (2) Automatic event type classification: Based on the scene labels in each event object, events are automatically classified into different types. Event types include, but are not limited to: natural landscapes (such as mountains, lakes, seas, and forests), historical buildings (such as ancient pagodas, temples, and city walls), animal interactions (such as zoos and farms), food experiences (such as restaurants and snack shops), cultural experiences (such as museums and exhibitions), and leisure and entertainment (such as parks and amusement parks). The classification method is as follows: extract the scene labels from the scene_labels field of the event object, and determine the event type according to the predefined label-type mapping table.

[0115] (3) Reading user history preferences: Read the user's history preference data from the user profile database, including the preference weight of interest category, emotion keyword preference, travel style tag, note template type preference, etc.

[0116] For example, user profile data shows that users have a preference weight of 0.9 for "animal interaction" interests, 0.6 for "historical architecture" interests, and 0.7 for "food experiences" interests; users' preferred emotional keywords include "joy," "excitement," and "happiness"; users' travel style tag is "family travel"; and users' preferred note template type is "family travelogue."

[0117] (4) Obtaining information on points of interest: Read detailed information of the points of interest associated with the event object from the point of interest database, including the name, category, description, opening time, user rating, and reasons for recommendation of the point of interest.

[0118] (5) Prompt word construction: Based on the user's preset note template type, select the corresponding text generation style parameters from the template library, and convert the event object sequence, user preference data and interest point information into structured natural language prompt words.

[0119] (6) Large language model invocation and text generation: Input the constructed prompt words into the large language model to generate text.

[0120] (7) Selected Media Screening and Embedding: Selected media are screened based on key images in the event object sequence using a quality assessment algorithm based on image sharpness index and composition score.

[0121] (8) Formatted output of the travel notes draft: The generated text content and the embedded selected media are formatted and output in HTML5 or Markdown format to form the final travel notes draft.

[0122] This step outputs a draft travel journal containing a timeline, locations, experience descriptions, and selected media. The draft journal should have a text coherence score of at least 0.8 and an information accuracy rate of at least 90%.

[0123] This step automatically transforms fragmented multimodal event data into high-quality, shareable structured travel notes. Through prompting engineering, the structured event data is converted into natural language prompts, guiding a large language model to generate personalized travel note content according to the user's preset style, achieving intelligent conversion from data to content. By automatically selecting premium media based on image quality assessment algorithms, high-resolution images and good composition are ensured in the notes, enhancing their visual appeal and readability.

[0124] By invoking a large language model based on the Transformer architecture, the system can generate coherent, fluent, and emotionally resonant travel note text, significantly lowering the barrier to entry for users. By combining event type, user history preferences, and points of interest, the system can generate personalized note content. For example, for users who prefer "family travel," the generated notes highlight the warm family atmosphere and the child's happy emotions; for users who prefer "photography travel," the generated notes emphasize lighting effects and compositional aesthetics. By automatically selecting and embedding premium media into the notes, the system generates draft notes that are rich in images and text, and ready to share directly. Users only need to make minor edits (such as adding or deleting images and modifying a few words) before publishing, increasing creation efficiency several times over. Actual testing shows that the generated note drafts achieve a text coherence score of no less than 0.8, an information accuracy rate of no less than 90%, and a user satisfaction rate of over 85%.

[0125] Step S107: User Interaction and Closed-Loop Learning In this step, the travel notes draft is presented to the human-computer interaction interface for the user to edit. After the user confirms and publishes, natural language processing is performed on the note text content to extract frequently occurring scene categories, interest point types, and emotional keywords. The weights of each category and keyword are calculated, and the weights are used as user preference features to update the user profile model. Based on the updated user profile model, highly matching interest points are retrieved from the interest point database, the matching score is calculated, and the interest points are sorted from high to low. The top-ranked interest points are recommended to the user's future travel itinerary planning.

[0126] This step includes the following sub-processes: (1) Presentation and editing of travel notes: The system presents the travel notes draft generated in step S106 to the user through the human-computer interaction interface (in-vehicle central control screen or mobile app).

[0127] The interface uses a WYSIWYG editor, supporting text editing, image management, layout adjustment, and template switching.

[0128] (2) Natural language processing of note text content: After the user confirms the posting, the system performs natural language processing on the final note text content to extract user preference features.

[0129] (3) User profile model update: The extracted keyword weights are used as user preference features to update the user profile model. The user profile model is represented by vectors, with each dimension corresponding to a scene category or emotion keyword, and the dimension value representing the user's preference for that category or keyword. The updated profile vectors are stored in the user profile database, overwriting the original profile data.

[0130] (4) Interest point recommendation: Based on the updated user profile model, highly matching interest points are retrieved from the interest point database to optimize future travel itinerary recommendations.

[0131] The output of this step includes: (1) the final travel notes confirmed and published by the user, which are stored in the user's personal center and can be shared to social platforms; (2) the updated user profile model, which is stored in the user profile database; and (3) a list of recommended points of interest, which are presented on the human-computer interaction interface for the user to select.

[0132] This step leverages user-submitted notes to optimize system services. By performing natural language processing on the note text, user preference features are extracted and the user profile model is updated. The system can continuously learn from changes in user interests and preferences, enabling ongoing optimization of personalized services. Based on the updated user profile model, the system recommends points of interest, providing users with more accurate travel itinerary planning suggestions tailored to their preferences.

[0133] By extracting preference features from user-posted notes to update the user profile model, the system forms a reinforced closed loop of "behavior generates data, data optimizes recommendations, and recommendations lead to new behaviors," making the intelligent service increasingly "understand" the user the more it is used. By employing an exponential moving average method to update the profile vector, the system can balance historical and current preferences, maintaining profile stability while responding promptly to changes in user preferences. By calculating a matching score based on cosine similarity and recommending points of interest, the system can provide users with personalized travel itinerary planning suggestions, achieving an accuracy rate of over 75%. Actual testing showed that after three or more travel records, the matching accuracy of the system's recommended points of interest significantly improved, increasing user satisfaction by approximately 40%.

[0134] The following uses "family farm check-in" as a specific example to describe in detail the complete implementation process of the method of the present invention.

[0135] 1. Automatic event detection (corresponding to...) Figure 2 ) 1.1 Trigger: The vehicle enters the "Wonderful Animal Farm" geofence (preset radius 150 meters).

[0136] 1.2 Verification: The system immediately initiates multimodal verification.

[0137] 1) Vision: The camera captures environmental images, and the local model identifies high-confidence labels such as "grassland" (0.92) and "sheep" (0.87).

[0138] 2) Audio: The microphone captured the sound inside the car, and the emotion analysis result was "joy" (0.8), and the keyword "lamb" was identified.

[0139] 1.3 Decision: The fusion engine determines a parent-child interaction event as valid based on the rules (visual confidence > 0.7, containing the "animal" label, and positive emotions).

[0140] 1.4 Records: The system automatically packages "key image + 5-second laughter audio + precise location + timestamp", generates a structured TravelEvent object, associates it with the route book node, and stores it in the local database.

[0141] 2. Smart Note Generation (corresponding) Figure 3 ) 2.1 Trigger: After the trip ends and the vehicle is turned off, the system detects a new event and asks whether to generate a travelogue.

[0142] 2.2 Generation: After user confirmation: 1) The system extracts all TravelEvent objects for the day and sorts them by time.

[0143] 2) The content-generating agent converts event data into prompt words, such as: "At 2:30 pm, at the Wonderful Animal Farm, the child is feeding the lambs and is very happy." 3) Invoke the large language model and combine it with the style instructions of "lively and heartwarming parent-child travelogue" to generate a coherent manuscript.

[0144] 4) Automatically match the best image in the event, embed it into the travelogue template, and generate an HTML5 note draft with pictures and text.

[0145] 2.3 Release and Learning: 1) Users can easily edit and publish to social media platforms.

[0146] 2) The system automatically analyzes the notes, extracts keywords such as "parent-child" and "animals", updates the user preference model, and optimizes future route recommendations.

[0147] Through the above complete example, the present invention realizes a complete closed loop from automatic perception of travel events, multimodal data encapsulation, intelligent travelogue generation to user profile updates and point-of-interest recommendations, significantly improving the user's travel recording and sharing experience.

[0148] Compared with the prior art, the present invention has the following beneficial effects: 1. By employing a dual-trigger mechanism combining geofencing for initial screening with multimodal verification, this invention achieves automatic and accurate perception of travel events. Compared to the high false trigger rate caused by single GPS triggering in existing technologies, this invention improves event capture accuracy by over 80% through the fusion of visual recognition and audio emotion analysis, while requiring no manual user operation. This solves the technical problems of passive and easily overlooked recording methods in existing technologies. The technical principle of the dual-trigger mechanism is as follows: geofencing serves as the first screening layer, quickly locating potential event occurrence areas; multimodal verification serves as the second confirmation layer, accurately determining the semantic attributes of events through cross-verification of visual and auditory information, avoiding the limitations of single-modal judgment.

[0149] 2. By fusing multi-source sensor data from in-vehicle GPS, cameras, microphones, and vehicle bus, this invention achieves semantic-level understanding of travel events. Compared to existing technologies that rely solely on GPS location information or vehicle dynamics information, this invention extracts visual scene features through convolutional neural networks, analyzes audio emotion features through recurrent neural networks, and obtains semantic information through keyword extraction. This achieves multi-dimensional fusion perception of location, vision, hearing, and semantics, enabling accurate understanding of "what travel event is happening." The technical principle of multimodal fusion lies in: using a unified timestamp for time synchronization and alignment to ensure consistency of different modal data in the time dimension; and using a fusion decision engine to weightedly calculate scene label confidence, emotion category confidence, and keyword semantic relevance, achieving effective fusion and complementarity of multimodal information, significantly improving the accuracy and robustness of event determination.

[0150] 3. Through intelligent content generation technology based on a large language model, this invention achieves the automatic conversion from fragmented multimodal data to structured travel notes. Compared to existing technologies that limit themselves to simple recording or categorization, this invention uses prompting engineering technology to convert structured event data into natural language prompts, guiding the large language model to generate personalized travel note content according to the user's preset style. Furthermore, it automatically selects and embeds selected media into the notes using an image quality assessment algorithm, generating high-quality, illustrated note drafts. The technical principle of intelligent content generation lies in: providing structured input to the large language model by filling in location, time, scene description, and user emotional state according to a predefined template format; achieving personalized note style customization by selecting corresponding text generation style parameters from a template library; and realizing intelligent media selection by calculating image sharpness using the Laplacian operator and calculating composition scores based on the rule of thirds. Testing shows that the generated note drafts have a text coherence score of no less than 0.8 and an information accuracy rate of no less than 90%. Users only need minimal editing before publishing and sharing, increasing creation efficiency several times over.

[0151] 4. By extracting preference features from user-posted notes to update the user profile model and optimize future travel itinerary recommendations, this invention achieves a complete data closed loop from recording to recommendation. Compared to the problems in existing technologies where recorded discrete event data is not effectively organized and cannot be fed back to the travel planning system, this invention uses natural language processing on the note text content to extract frequently occurring scene categories, interest point types, and emotional keywords. It calculates the weights of each category and keyword as user preference features, updates the user profile model, and, based on the updated profile, retrieves highly matching interest points from the interest point database and recommends them to future travel itinerary planning. The technical principle of the data closed loop is that user-posted notes, as high-quality user-generated content, contain implicit preference information that is automatically extracted by the system and used to optimize the user profile, forming an enhanced closed loop of "behavior generates data, data optimizes recommendations, and recommendations bring new behaviors." This makes the intelligent service increasingly "understands" the user, significantly improving the system's personalized service capabilities and user experience.

[0152] Example 2 This embodiment discloses a method for automatically recording in-vehicle travel events and generating intelligent travelogues.

[0153] Methods for automatic recording of in-vehicle travel events and intelligent travelogue generation include: The triggering module is configured to: acquire the geographic coordinates of the vehicle during its driving process, and trigger the preset point of interest location based on the geographic coordinates; The verification module is configured to: when the preset point of interest location is triggered, use multimodal data fusion to verify whether the current point of interest is a travel event object; The data encapsulation module is configured to: when the current point of interest is determined to be a travel event object, encapsulate the acquired multimodal data into structured travel data, associate it with the current point of interest, and thus form an event database; The note draft generation module is configured to: at a set time point, use an intelligent agent to read travel event objects from the event database, generate travel note drafts based on structured travel data, and present the note drafts to the user; The user profile update module is configured to: obtain the content of the user's last published notes and update the user profile using the content of the last published notes.

[0154] It should be noted that each module in this embodiment corresponds one-to-one with each step in Embodiment 1, and their specific implementation processes are the same.

[0155] Example 3 The purpose of this embodiment is to provide a computer-readable storage medium.

[0156] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for automatic recording of vehicle travel events and intelligent travelogue generation as described in Embodiment 1 of this disclosure.

[0157] Example 4 The purpose of this embodiment is to provide an electronic device.

[0158] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the method for automatic recording of in-vehicle travel events and intelligent travelogue generation as described in Embodiment 1 of this disclosure.

[0159] The steps and methods involved in the apparatuses of Embodiments 2, 3, and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0160] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0161] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for automatically recording in-vehicle travel events and generating intelligent travelogues, characterized in that, Includes the following steps: Obtain the geographical coordinates of the vehicle during its journey, and trigger the preset points of interest based on the geographical coordinates; When the preset point of interest location is triggered, a multimodal data fusion method is used to verify whether the current point of interest is a travel event object; Once the current point of interest is determined to be a travel event object, the acquired multimodal data is encapsulated into structured travel data, associated with the current point of interest, and thus forms an event database. At a set time, the intelligent agent reads travel event objects from the event database and generates a draft of travel notes based on the structured travel data, presenting the draft notes to the user. Obtain the final published notes from users and update user profiles using these notes.

2. The method for automatic recording of in-vehicle travel events and intelligent travelogue generation as described in claim 1, characterized in that, Also includes: Acquire travel route data, which includes pre-planned point of interest coordinates and corresponding geofence radius parameters; When the distance between the real-time geographic location coordinates of the vehicle and the coordinates of any point of interest in the travel itinerary data is less than the corresponding geofence radius parameter, the preset point of interest location is triggered.

3. The method for automatic recording of in-vehicle travel events and intelligent travelogue generation as described in claim 1, characterized in that, When a preset point of interest location is triggered, a multimodal data fusion approach is used to verify whether the current point of interest is a travel event object, specifically including: The system acquires images of the current environment, extracts image features through a pre-trained convolutional neural network, and outputs recognition results including scene category labels and confidence scores. The system collects in-vehicle audio signals, extracts acoustic feature parameters including fundamental frequency, speech rate, and energy, and analyzes these acoustic feature parameters using a recurrent neural network to output emotion category and confidence level. Simultaneously, the audio is converted into text and keywords related to the travel scenario are extracted. The confidence scores of scene category labels, emotion category labels, and keywords are weighted and calculated to determine whether the event is a travel event. or, The scene category label is a travel-related category, which includes at least one of the following: natural landscapes, historical buildings, animals, and distinctive plants. The emotion categories include at least one of the following: joy, excitement, calmness, and sadness. or, Overall score = w1 × scene_confidence_max + w2 × emotion_confidence + w3 × keyword_score; Where w1, w2, and w3 are preset weight coefficients; scene_confidence_max is the highest confidence value of the scene category label; emotion_confidence is the confidence value of the emotion category; and keyword_score is the semantic relevance score of the keyword.

4. The method for automatic recording of in-vehicle travel events and intelligent travelogue generation as described in claim 2, characterized in that, The acquired multimodal data is encapsulated into structured travel data, linked to the current point of interest, and thus forms an event database, specifically including: The multimodal data within a preset time window before and after the trigger time is encapsulated into structured travel event data containing metadata; The multimodal data includes geographic location coordinates, timestamps, key images, audio clips, and vehicle status data; The structured travel event objects are associated with corresponding points of interest by matching geographic coordinates and timestamps and then stored in the event database; or, The structured travel event object is associated with a corresponding point of interest by matching geographic location coordinates and timestamps, specifically as follows: Based on geographic coordinates and timestamps, retrieve the nearest and most recent point of interest nodes in the travel itinerary; Establish the association between structured travel event objects and points of interest nodes, and record the association in the event database.

5. The method for automatic recording of in-vehicle travel events and intelligent travelogue generation as described in claim 1, characterized in that, At a set time point, the intelligent agent reads travel event objects from the event database and generates a draft travel note based on the structured travel data, specifically including: The large language model is invoked when the trip ends or a rest stop is reached. The large language model reads all structured travel event object sequences of this trip from the event database, and combines them with the event types automatically classified according to scene tags, user history preferences read from the user profile database, and interest point information obtained from the pre-stored interest point database to generate a travel note draft containing timeline, location, and experience description. The travel notes draft is presented on the human-computer interaction interface for users to edit; or, The process of generating travel note drafts using a large language model includes: Arrange the structured travel event object sequence in chronological order and extract the location, time, scene tag, emotion category and keywords for each travel event object; According to the predefined template format, the location, time, scene description and user emotional state are filled into the corresponding fields of the template in sequence to form structured natural language prompts; Based on the user's preset note template type, select the corresponding text generation style parameters from the template library; The text generation style parameters and natural language prompts are input into a large language model to generate coherent text content.

6. The method for automatic recording of in-vehicle travel events and intelligent travelogue generation as described in claim 1, characterized in that, Obtain the final published note content from the user, and update the user profile using the final published note content, specifically including: After the user confirms the posting, the final posted note content is obtained and natural language processing is performed. Extract frequently occurring scene categories, interest point types, and emotional keywords, and calculate the weights of each frequently occurring scene category, interest point type, and emotional keyword. The weights of frequently occurring scene categories, interest types, and emotional keywords are used as user preference features to update the user profile model. Based on the updated user profile model, highly matching points of interest are retrieved from the point of interest database, matching scores are calculated and sorted from high to low, and the top-ranked points of interest are recommended to the user's future travel plans.

7. The method for automatic recording of in-vehicle travel events and intelligent travelogue generation as described in claim 4, characterized in that, The encapsulation method for multimodal data is as follows: Based on a unified timestamp, geographic coordinates, key images, audio clips, and vehicle status data are synchronized and aligned in time, and then encapsulated into a JSON data object.

8. A method for automatically recording in-vehicle travel events and generating intelligent travelogues, characterized in that, include: The triggering module is configured to: acquire the geographic coordinates of the vehicle during its driving process, and trigger the preset point of interest location based on the geographic coordinates; The verification module is configured to: when the preset point of interest location is triggered, use multimodal data fusion to verify whether the current point of interest is a travel event object; The data encapsulation module is configured to: when the current point of interest is determined to be a travel event object, encapsulate the acquired multimodal data into structured travel data, associate it with the current point of interest, and thus form an event database; The note draft generation module is configured to: at a set time point, use an intelligent agent to read travel event objects from the event database, generate travel note drafts based on structured travel data, and present the note drafts to the user; The user profile update module is configured to: obtain the content of the user's last published notes and update the user profile using the content of the last published notes.

9. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the method for automatic recording of in-vehicle travel events and intelligent travelogue generation as described in any one of claims 1-7.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the method for automatic recording of in-vehicle travel events and intelligent travelogue generation as described in any one of claims 1-7.

Images