Intelligent terminal adaptive acquisition and cloud event generation system based on multi-modal fusion

Abstract

The present application relates to the technical field of intelligent hardware, Internet of Things, mobile sensing, cloud computing, cloud storage, computer vision, audio processing, multi-modal artificial intelligence and digital content generation, and discloses an intelligent terminal adaptive acquisition and cloud event generation system based on multi-modal fusion; the present application calculates unified event importance scores based on multi-modal saliency, sensing reliability and historical continuity through an intelligent recording terminal, and cooperatively adjusts acquisition strategies and upload priorities; the cloud performs event boundary detection through spatiotemporal constraint and reliability-corrected fusion similarity, generates a structured event timeline, and generates a digital diary or a short video accordingly, while establishing a blood relationship tracing relationship among original media, event objects and generated content, realizing content consistency verification and version management, and improving recording integrity and verifiability.

Description

Technical Field

[0001] This invention relates to the fields of smart hardware, Internet of Things, mobile sensing, cloud computing, cloud storage, computer vision, audio processing, multimodal artificial intelligence and digital content generation, and more specifically discloses a smart terminal adaptive acquisition and cloud event generation system based on multimodal fusion. Background Technology

[0002] With the rapid development of IoT and AI technologies, mobile smart terminals have been widely penetrated into diverse scenarios such as pet monitoring, outdoor sports, and safety recording. Users are increasingly demanding intelligent recording capabilities that ensure "no key events are missed and the generated content is traceable."

[0003] In existing technologies, pet collar cameras, action cameras, wearable recorders, and other devices generally employ fixed strategies for data collection, primarily using manual triggering, continuous recording, or fixed-interval photo taking as their operating modes. Location, audio, image, and video data are stored and managed independently, making it difficult to form a structured and complete event record that can be automatically understood and retrieved over the long term. This situation reveals the following shortcomings in practical applications: Fixed high-frequency data acquisition generates a large amount of redundant data that is repetitive, ambiguous, or has no event value, significantly increasing terminal power consumption, local storage usage, network transmission traffic, and cloud storage costs. On the other hand, fixed low-frequency data acquisition is prone to missing key scenarios such as location changes, rapid movement, significant acoustic events, and subject interactions, resulting in the loss of important event records. Although some existing solutions adjust upload strategies based on network bandwidth, they do not dynamically adapt to the importance of the events themselves at the source of data acquisition, failing to fundamentally resolve the contradiction between the proliferation of invalid data and the omission of key events.

[0004] Different sensors have inherent differences in sampling frequency, clock accuracy, and availability. Multimodal data fusion mechanisms are too simplistic: existing technologies mostly rely on a single time window for coarse data alignment or on single-modal features for event segmentation. They lack strict spatiotemporal constraint pre-verification and multimodal reliability correction mechanisms, which can easily lead to the incorrect segmentation of the same event into multiple segments or the incorrect merging of records from different times and locations into the same event. This can result in problems such as time misalignment, location mismatch, and content duplication in the generated diaries, videos, and other content.

[0005] The traceability and consistency of evidence for automatically generated content are insufficient: Existing AI-generated digital diaries, itinerary summaries, short videos, and other content usually only establish a coarse-grained association with the original data, making it impossible to accurately trace back from the generated content to the original media data. At the same time, there is a lack of consistency verification mechanism between the generated content and the original collected evidence. When the algorithm model is upgraded, the user deletes the original data, or the permissions are changed, it is difficult to accurately update and verify the compliance of the generated content, and it is also impossible to verify whether the generated content is truly based on the collected data.

[0006] In summary, the industry urgently needs a technical solution that can cover the entire chain of "device-side adaptive data acquisition - cloud-based multimodal fusion - structured content generation - end-to-end traceability" to reduce invalid data overhead while improving the accuracy of event recording and the credibility of generated content. Summary of the Invention

[0007] The main technical problem solved by this invention is to provide an intelligent terminal adaptive acquisition and cloud event generation system based on multimodal fusion, which can solve the problems mentioned in the background art.

[0008] To solve the above-mentioned technical problems, according to one aspect of the present invention, more specifically, a smart terminal adaptive acquisition and cloud event generation system based on multimodal fusion, comprising: a smart recording terminal, a cloud platform, and a user terminal; The intelligent recording terminal includes an image acquisition module, an audio acquisition module, a positioning module, a communication module, a motion sensing module, a local storage module, and an edge processing module. It is used to continuously or periodically acquire positioning data and audio data, and to acquire visual media data according to a preset acquisition strategy, manual triggering commands, or remote control commands. The visual media data includes at least one of static image data and dynamic video data. The dynamic video data includes continuous recording data or video clips. Time information is written to the positioning data, audio data, and visual media data based on a unified time reference, forming a multimodal data unit and uploading it to a cloud platform. The edge processing module is used to calculate a unified event importance score based on the degree of change in motion, position, sound, and visual data, the reliability of corresponding sensor data, cross-modal consistency, and historical continuity with preceding events. Based on the unified event importance score, it collaboratively adjusts the visual media acquisition strategy, positioning sampling frequency, audio retention range, local cache level, and upload priority. The cloud platform includes an access service, a storage system, a multimodal processing system, and a content generation system. The multimodal processing system is used to sequentially filter based on time constraints and spatial continuity constraints, and then determine the event boundary based on a comparison between the reliability-corrected fusion similarity and a preset merging threshold and a segmentation threshold, wherein the merging threshold is higher than the segmentation threshold. It merges multiple multimodal data units into multiple event objects and forms an event timeline based on the event objects. The content generation system is used to generate at least one of digital diaries, itinerary summaries, and short videos based on the event timeline. It establishes an event lineage tracing relationship between the original media object, multimodal data unit, event object, and generated content, and performs consistency verification between the content statement unit in the generated content and the corresponding event object or original media evidence. The user terminal is used to display trajectories, event timelines, or generated content, and to control the intelligent recording terminal to perform static image capture, dynamic video recording, or set acquisition strategies.

[0009] Furthermore, the preset acquisition strategy includes a static image acquisition interval and dynamic video recording parameters. The static image acquisition interval is any value from 10 seconds to 30 minutes, with a default value of 60 seconds. When the intelligent recording terminal is in a low-activity state, it extends the static image acquisition interval and performs at least one of the following operations: reducing the dynamic video frame rate or bit rate, shortening the dynamic video retention time, and stopping recording. When it detects a change in position, speed, posture, significant sound, significant visual target, or remote shooting command, it performs at least one of the following operations: shortening the static image acquisition interval, immediately capturing a static image, starting dynamic video recording, extending the video segment duration, and increasing the dynamic video frame rate or bit rate.

[0010] Furthermore, the audio acquisition module segments continuous audio into a circular buffer according to a preset duration; the edge processing module retains audio segments before and after the acoustic event based on the occurrence time and event score of the acoustic event, and performs downsampling, compression, or deletion processing on audio segments for which no effective acoustic features are detected; the edge processing module is also used to perform quality screening and duplication suppression on multimodal data units. The quality screening includes at least one of performing blur detection, exposure detection, occlusion detection, and video frame drop detection on static images or dynamic videos, signal-to-noise ratio detection on audio, and validity detection on positioning data; the duplication suppression identifies duplicate or nearly duplicate data based on at least two of the following: perceptual hash of static image or video keyframes, visual feature vector, video fingerprint, audio fingerprint, location distance, and time distance, and prioritizes retaining data with higher quality scores.

[0011] Furthermore, the user terminal sends remote shooting commands via at least one of Bluetooth Low Energy communication, Wi-Fi communication, or cellular network. These remote shooting commands include at least one of a photo capture command, a start recording command, a stop recording command, and a preset duration video clip acquisition command. Each remote shooting command has a command identifier, command type, generation time, and validity period. After verifying the device identity and validity period, the smart recording terminal performs the corresponding shooting and returns the shooting result. The smart recording terminal also includes physical buttons or touch input components for triggering manual photo capture, starting recording, or stopping recording. Data to be uploaded is written to a breakpoint resume queue. Queue items include at least data fragments, content verification values, event scores, data types, creation time, and upload status. Upload priority is determined by at least two of the following: event score, data type, network quality, remaining battery power, and local storage capacity. Under weak network or cellular network conditions, at least one of the following is prioritized for upload: location data, event metadata, representative images, video keyframes, low-bitrate video preview clips, and low-bitrate audio summaries. Uploading high-resolution still images or original dynamic videos is delayed.

[0012] Furthermore, the edge processing module calculates motion scores, location scores, sound scores, and visual scores respectively, and determines a reliability coefficient based on the validity of the sensor data corresponding to each score. Based on the normalized weighted result of the available scores and their corresponding reliability coefficients, cross-modal consistency within the same time window (where cross-modal consistency refers to the degree of matching between visual data, audio data, and motion data in terms of event semantics or scene changes within the same time window, the historical continuity of current data and preceding events in terms of time, trajectory, or semantics, user trigger priorities, and data quality defect suppression items), a unified event importance score is obtained. The edge processing module performs time smoothing on the unified event importance score and adopts... A state switching mechanism with hysteresis is used between a first threshold and a second threshold below the first threshold. When the smoothed unified event importance score reaches the first threshold, the system enters an important event state. When the smoothed unified event importance score is below the second threshold and remains below the second threshold for a preset duration, the system enters a low-activity state. The start conditions, stop conditions, segment duration, frame rate, resolution, or bit rate of dynamic video recording are determined based on the smoothed unified event importance score, and secondary constraints are applied based on at least one of the remaining battery power, device temperature, and local storage capacity. The same unified event importance score is used to collaboratively determine the visual media acquisition strategy, location sampling frequency, audio retention range, local cache level, and upload priority.

[0013] Furthermore, the storage system includes object storage and a metadata database. The object storage is used to store still images, original dynamic videos, audio, generated videos, and trajectory rendering files. The metadata database is used to store device identifiers, unified timestamps, trajectory points, media object identifiers, media content verification values, event identifiers, access permissions, and processing versions. Each event object is associated with at least a start time, end time, trajectory segment, representative image or video clip, audio clip, event tag, event score, and media object reference. The content generation system establishes an event lineage graph between the original media objects, multimodal data units, event objects, event timelines, and generated content. Each level of object in the event lineage graph stores a unique object identifier and a content verification value. The generated content includes the parent object identifier, processing version, and processing time. It also stores the event object identifier and media object identifier it references, the generation model or rule version, the generation configuration version, and the generated content verification value. When event processing or content generation is re-executed, a new version is created for the new result while retaining the parent version identifier, enabling the generated content to be back-located to the corresponding event object, multimodal data unit, and original media object. The content generation system also splits the generated content into multiple content statement units, associating each content statement unit with at least one event object identifier or media object identifier, and calculating the evidence coverage rate and content conflict degree. When the evidence coverage rate is lower than a preset threshold or the content conflict degree is higher than a preset threshold, the corresponding content statement unit is deleted, corrected, regenerated, or marked as pending verification.

[0014] A method for generating multimodal events and digital content using a smart recording terminal and a cloud platform includes: The intelligent recording terminal continuously or periodically collects positioning data and audio data, and collects visual media data according to a preset collection strategy, manual triggering command or remote shooting command. The visual media data includes at least one of static image data and dynamic video data, and the dynamic video data includes continuous recording data or video clips. Time information is written to positioning data, audio data, and visual media data based on a unified time reference, and time alignment is performed on the collected data to form multimodal data units; A unified event importance score is calculated based on the degree of change in motion, position, sound, and visual data, the reliability of corresponding sensor data, cross-modal consistency, and historical continuity with preceding events. The visual media acquisition strategy, location sampling frequency, audio retention range, local cache level, and upload priority are then adjusted in a coordinated manner based on the unified event importance score. Upload the multimodal data units to the cloud platform; The cloud platform sequentially filters data based on time constraints and spatial continuity constraints, and then determines the event boundary by comparing the reliability-corrected fusion similarity with preset merging and segmentation thresholds, wherein the merging threshold is higher than the segmentation threshold. Multiple multimodal data units are merged into multiple event objects, and an event timeline is formed based on the event objects. The system generates at least one of digital diaries, itinerary summaries, and short videos based on the event timeline, establishes an event lineage tracing relationship between the original media object, multimodal data unit, event object, and generated content, and performs consistency verification on the content statement unit in the generated content with the corresponding event object or original media evidence.

[0015] An intelligent recording terminal includes a processor, a memory, an image acquisition module, an audio acquisition module, a positioning module, a communication module, and a motion sensing module; The memory stores a computer program. When the computer program is executed by the processor, the intelligent recording terminal continuously or periodically collects positioning data and audio data, and collects visual media data according to a preset collection strategy, manual triggering command or remote shooting command. The visual media data includes at least one of static image data and dynamic video data. The dynamic video data includes continuous recording data or video clips. Time information is written into various types of collected data based on a unified time reference to form multimodal data units; A unified event importance score is calculated based on the degree of change in motion, position, sound, and visual data, the reliability of corresponding sensor data, cross-modal consistency, and historical continuity with preceding events. The visual media acquisition strategy, location sampling frequency, audio retention range, local cache level, and upload priority are then adjusted in a coordinated manner based on the unified event importance score. Multimodal data units are sent to the cloud platform in a data format that includes time information, location information, media object identifier, content verification value, and unified event importance score.

[0016] A cloud server system includes at least one processor and a memory communicatively connected to the at least one processor; The memory stores a computer program. When the computer program is executed by the processor, the cloud server system receives location data, audio data, visual media data, and unified event importance scores uploaded by the smart recording terminal. The visual media data includes at least one of static image data and dynamic video data. The dynamic video data includes continuous recording data or video clips. The event boundaries are determined by filtering according to time constraints and spatial continuity constraints, and then by comparing the reliability-corrected fusion similarity with the preset merging threshold and segmentation threshold, wherein the merging threshold is higher than the segmentation threshold. Multiple multimodal data units are merged into multiple event objects, and an event timeline is formed based on the event objects. Generate at least one of the following based on the event timeline: digital diary, itinerary summary, and short video; Establish an event lineage tracing relationship between the original media object, multimodal data unit, event object, and generated content, and perform consistency verification on the content statement unit in the generated content and the corresponding event object or original media evidence.

[0017] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method of claim 7.

[0018] The beneficial effects of the intelligent terminal adaptive acquisition and cloud event generation system based on multimodal fusion of this invention are as follows: By using a unified event importance scoring mechanism based on multimodal saliency, sensing reliability, and historical continuity at the edge, the acquisition strategy, positioning frequency, audio retention, and upload priority are adjusted in a coordinated and adaptive manner, which helps to reduce the risk of missing key events while reducing invalid data overhead. By using a layered event boundary detection method that combines time constraints, spatial continuity constraints, and reliability corrections for similarity in the cloud, and employing a merging threshold higher than the segmentation threshold, it helps reduce event missegmentation and mismerging caused by asynchronous or single-modal fluctuations of sensors. Furthermore, by establishing a lineage relationship including object identifiers, checksums, and parent versions among the original media, multimodal data units, event objects, and generated content, and performing evidence consistency and coverage checks on content statement units, the generated content can be verified back to its source at each level. This also facilitates partial updates and compliance verification during version upgrades or data changes. This solution improves the completeness and accuracy of event records while enhancing the verifiability of generated content and the maintainability of the system. Attached Figure Description

[0019] The present invention will now be described in further detail with reference to the accompanying drawings and specific implementation methods.

[0020] Figure 1 This is a schematic diagram of the system module architecture; Figure 2 A schematic diagram of the hardware components of an intelligent recording terminal; Figure 3 Flowchart for multimodal data acquisition and photo / video recording triggering; Figure 4This is a schematic diagram illustrating the collaboration between edge processing and cloud processing. Figure 5 This is a schematic diagram of a cloud-based tiered storage system. Figure 6 A schematic diagram of the data structure for a multimodal event object; Figure 7 Generate flowcharts for multimodal events in the cloud; Figure 8 This is a schematic diagram of the automatic video generation process; Figure 9 A schematic diagram of long-term digital memory and relationship graphs; Figure 10 This is a schematic diagram of adaptive sampling and low-power control. Figure 11 A diagram illustrating data security, privacy, and traceability controls; Figure 12 This is a schematic diagram illustrating different application forms of intelligent recording terminals. Detailed Implementation

[0021] The present invention will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the present application can be combined with each other.

[0022] Example 1: System Overall Structure According to one aspect of the invention, such as Figure 1 As shown, a smart terminal adaptive acquisition and cloud event generation system based on multimodal fusion is provided. The system consists of a smart recording terminal, an edge processing module, a cloud platform, a user terminal, and a digital memory bank.

[0023] The intelligent recording terminal can be installed on pet collars, harnesses, helmets, backpacks, vehicles, or other mobile objects or carriers. It is responsible for collecting multi-source sensor data such as vision, audio, positioning, and motion, and completing preliminary processing and adaptive acquisition control locally.

[0024] The edge processing module is deployed locally on the intelligent recording terminal and undertakes the core functions of data time alignment, quality screening, event importance calculation, collection strategy control and data caching and uploading, realizing low-power intelligent scheduling on the terminal side.

[0025] The cloud platform can be deployed in public cloud, private cloud or hybrid cloud environments. It decouples tasks such as media writing, feature extraction, event fusion, diary generation and video generation through message queues, and is responsible for data verification, multimodal event fusion, structured content generation and full-link data management.

[0026] User terminals, delivered via apps, web pages, or mini-programs, are used to display trajectories, event timelines, and generated content to users. They also support sending remote shooting commands and configuring acquisition strategies to smart recording terminals.

[0027] Digital memory is used for long-term storage of structured event data, media assets, and relationship graphs, supporting content retrieval, data tracing, and content reproduction.

[0028] Example 2: Hardware Composition of the Intelligent Recording Terminal like Figure 2 As shown, the intelligent recording terminal is centered around a main controller, which connects to and manages the image acquisition module, audio acquisition module, positioning module, motion sensing module, communication module, power management module, local cache, and buttons / indicators.

[0029] The main controller can be a microcontroller, application processor, or system-on-a-chip with image signal processing capabilities, and is responsible for the overall operation scheduling and logic control of the terminal.

[0030] The image acquisition module may include one or more cameras, image sensors, lenses, and video encoders for acquiring still image data and dynamic video data. Still image data can be a single photograph or a series of photographs, while dynamic video data can be continuous video recordings or video clips acquired according to a preset duration or an adaptive duration. It may include video frame sequences, encoding parameters, acquisition time range, and an optional synchronized audio track.

[0031] The audio acquisition module can use a single microphone or a microphone array to write continuous audio into a circular buffer in segments according to a preset duration. Adjacent segments can retain an overlap of 0.1 seconds to 5 seconds to reduce sound loss at the segment boundaries.

[0032] The positioning module supports positioning from multiple systems such as GPS, BeiDou, GLONASS, and Galileo. It can also integrate cellular positioning, wireless LAN positioning, Bluetooth beacon, or inertial estimation results to improve the availability and accuracy of positioning in different scenarios.

[0033] The motion sensing module can include a three-axis or six-axis IMU and can also be expanded with a magnetometer, barometer or speed sensor to sense the motion state, attitude and changes in the environment of the device.

[0034] The communication module supports at least one of Bluetooth Low Energy communication and Wi-Fi communication, and can also be expanded with cellular communication capabilities to meet the needs of short-range and long-range data interaction.

[0035] The power management module dynamically controls the sleep and wake-up of the camera, positioning module, and communication module based on the acquisition status, upload status, and event scoring, thereby enabling the terminal to operate with low power consumption.

[0036] The local cache includes a circular audio cache and a media file cache, which are used to temporarily store the raw data collected and the processed data to be uploaded.

[0037] The buttons / indicators support manual triggering of operations such as taking photos, starting recording, and stopping recording, and the indicator lights can provide feedback on the device's working status.

[0038] Example 3: Timed, manual, and remote photo or video recording. like Figure 3 As shown, after the device starts up, it first performs clock synchronization to provide a unified benchmark for the time alignment of subsequent multimodal data.

[0039] The system sets a static image acquisition baseline interval T0, which can be configured from 10 seconds to 30 minutes, with a default value of 60 seconds. The system also supports multiple dynamic video recording modes, including continuous recording, segmented recording according to preset duration, and event-triggered recording. In event-triggered recording mode, video data is written to a circular buffer in time segments, and dynamic video clips before and after the event can be retained according to the event trigger time.

[0040] In addition to automatic data collection using preset strategies, the system supports two manual triggering methods: one is to directly trigger taking photos, starting or stopping recording via the physical buttons or touch input components on the terminal; the other is to send remote shooting commands through the user terminal. In close-range scenarios, the commands can be sent directly via Bluetooth Low Energy communication, while in long-range scenarios, the commands can be sent from the user terminal to the cloud platform and then transmitted via cellular network or message channels maintained by the device.

[0041] Each remote shooting command carries a command identifier, command type, generation time, and validity period. It may also include parameters such as video clip duration, frame rate, resolution, and bitrate. After receiving the command, the intelligent recording terminal first verifies the device identity and command validity period. If the verification is successful, it executes the corresponding shooting operation and returns the shooting result to the user terminal. When the terminal is offline, the cloud can discard expired commands and reissue valid commands that have not expired once the terminal reconnects to the network.

[0042] Example 4: Unified Event Importance Scoring and Adaptive Resource Control like Figure 10 As shown, this embodiment achieves collaborative adaptive control of multiple types of collected resources through a unified event importance score. The specific process is as follows: 1. Calculation of single-mode significance and reliability Within a continuous time window, the acquisition controller calculates motion scores, position scores, sound scores, and visual scores, all normalized to the range of 0 to 1. Motion scores are calculated based on acceleration differences, angular velocity differences, and the degree of attitude change; position scores are calculated based on displacement per unit time, velocity changes, changes in dwell points, or geofence crossings; sound scores are calculated based on acoustic event confidence, audio energy changes, and sound category novelty; and visual scores are calculated based on the feature distance between adjacent static images or video keyframes, motion changes in dynamic videos, object detection confidence, and the degree of scene change.

[0043] For each mode, a corresponding reliability coefficient is calculated synchronously, with a value ranging from 0 to 1. Motion reliability is determined by the sensor's self-test status and sampling continuity; positioning reliability is determined by positioning accuracy, the number of available satellites, and time continuity; audio reliability is determined by the signal-to-noise ratio and the proportion of silent audio; visual reliability is determined by at least one of the following: sharpness of static images or dynamic videos, exposure suitability, occlusion ratio, frame loss ratio, and coding integrity.

[0044] 2. Calculation of Original Uniform Event Importance Score Let A be the set of currently available modalities. The system first calculates a basic weighted value based on the significance score and reliability coefficient of the available modalities. Then, it adds cross-modal consistency correction, historical continuity correction, and user trigger priority, and subtracts the quality defect suppression item to finally obtain the original unified event importance score. The calculation formula is as follows:

[0045] In the formula, These are the weighting coefficients for the corresponding modes. The reliability coefficient is the corresponding modality. For the significance score of the corresponding modality, For cross-modal consistency terms, For historical continuity, Trigger priority for users, For quality defect suppression, , , , These are the weight coefficients for the corresponding correction terms. This is a truncation function that restricts the calculation results to the range of 0 to 1. For modes that are missing or have a reliability below a preset threshold, they are not included in the set of available modes, and the weights of the remaining modes are renormalized to avoid scoring distortion caused by missing modes.

[0046] 3. Time smoothing processing To reduce frequent jitter in the acquisition strategy caused by transient noise, the original unified event importance score is exponentially smoothed to obtain the final unified event importance score used for strategy control. The calculation formula is as follows:

[0047] In the formula, The smoothness coefficient ranges from 0 to 1. The larger the value, the smoother the score changes and the stronger the anti-shake ability.

[0048] 4. Dynamic calculation of collected parameters The target static image acquisition interval is dynamically adjusted according to the importance of the event. The higher the importance, the shorter the acquisition interval and the higher the sampling density. The formula for calculating the target interval is as follows:

[0049] In the formula, The baseline acquisition interval for static images. For sensitivity parameters, and These are the shortest and longest data acquisition intervals allowed by the system, respectively.

[0050] For dynamic videos, parameters such as start conditions, stop conditions, single segment duration, frame rate, resolution, and bit rate are all dynamically determined based on a smoothed uniform event importance score.

[0051] 5. Hysteresis state machine and multi-resource coordinated regulation The system employs a first threshold and a second threshold L below the first threshold to construct a state switching mechanism with hysteresis characteristics, dividing the terminal's operating state into low-activity state, normal state, and important event state. When smoothed... When the rise reaches the first threshold, it enters a critical event state; when When the score drops below the second threshold and the duration reaches a preset value, the system enters a low-activity state; during other periods, it remains in a normal state or continues the previous state. This dual-threshold hysteresis design prevents the score from fluctuating repeatedly around the critical value, thus avoiding frequent hardware restarts and shutdowns.

[0052] The same unified event importance score is used simultaneously to coordinate and control five types of resources: visual media acquisition, location sampling, audio preservation, local caching level, and upload priority. In critical event states, the system shortens the static image acquisition interval, starts or extends dynamic video recording, and improves video frame rate, resolution, or bit rate; increases the positioning sampling frequency; expands the audio retention range before and after the event; and improves the data caching level and upload priority.

[0053] In low-activity states, the system extends the static image acquisition interval, reduces the dynamic video frame rate or bit rate, shortens the video retention time, or stops recording; reduces the positioning sampling frequency, switches the positioning module to periodic wake-up mode; performs downsampling, compression, or deletion on audio segments without significant acoustic features; and reduces the upload frequency of non-critical data.

[0054] Under normal conditions, the system operates according to the baseline acquisition strategy.

[0055] 6. Secondary resource constraints The power management module further applies secondary constraints to the aforementioned target acquisition strategy based on remaining battery power, device temperature, and local storage capacity to prevent device malfunctions under extreme operating conditions. When a user's photo / video recording command or a high-confidence security event is detected, the minimum necessary acquisition standards are maintained, and the triggering reason and strategy version are recorded.

[0056] Example 5: Edge Processing and Cloud Collaborative Processing like Figure 4 As shown, this embodiment covers the complete process of edge filtering, reliable uploading and cloud storage, multimodal event fusion, content generation and tracing, as detailed below: 1. Edge quality screening and duplication suppression The edge processing module first segments the raw sensor data into slices according to a unified time window, and then performs quality screening and duplication suppression to reduce invalid data from the source.

[0057] The quality screening covers four types of data: for static images and dynamic videos, it performs blur detection, exposure detection, occlusion detection, and video frame drop detection; for audio, it performs signal-to-noise ratio detection; and for location data, it performs validity detection. Data that does not meet the quality standards is filtered locally and not uploaded to the cloud.

[0058] Repetition suppression combines at least two of the following for judgment: perceptual hash, visual feature vector, video fingerprint, audio fingerprint, location distance, and temporal distance. For example, if the perceptual hash distance and feature distance of two adjacent still images are both below the threshold, and the location distance is also below the threshold, they are judged as approximately duplicates, and the one with higher clarity is retained. When adjacent dynamic video clips are similar in keyframe features, temporal range, and location range, the clip with higher quality score or more complete event coverage is retained, and the remaining data is either retained as metadata or directly deleted.

[0059] 2. Resuming interrupted downloads and prioritizing uploads Filtered data is written to the breakpoint resume queue. Each queue item must contain at least the data fragment, content validation value, event score, data type, creation time, and upload status. Upload priority is determined by the event score, data type priority, network compatibility score, power consumption compatibility score, and local storage pressure, calculated using the following formula:

[0060] In the formula, , , , , These are the weight coefficients for the corresponding dimensions. Rate the event For data type precedence, For network compatibility scoring, Rate the battery compatibility. Score storage stress.

[0061] The system dynamically adjusts its upload strategy based on the network environment: when Wi-Fi is available, all data to be transmitted can be uploaded in full; under weak network or cellular network conditions, priority is given to uploading location data, event metadata, representative images, video keyframes, low-bitrate video preview clips, and low-bitrate audio summaries, while uploading high-resolution still images and original dynamic videos is delayed, reducing transmission traffic while ensuring event traceability. The upload process employs a fragment numbering and content checksum mechanism; the terminal only deletes the corresponding queue item after the cloud confirms the fragment's completeness, ensuring transmission reliability.

[0062] 3. Cloud-based tiered storage architecture like Figure 5 As shown, the cloud access service first completes device authentication, upload session establishment, fragment integrity verification, and duplicate object detection, and then writes the data to the hierarchical storage system. The storage system includes object storage and a metadata database. Object storage is used to store media resources such as still images, raw dynamic videos, audio, generated videos, and trajectory rendering files; the metadata database is used to store structured metadata such as device identifiers, unified timestamps, trajectory points, media object identifiers, media content checksums, event identifiers, access permissions, and processing versions. In addition, the cloud is configured with a vector index library, a timeline database, graph relation storage, and lifecycle management: the vector index library stores semantic feature vectors of static images, dynamic video keyframes, audio, transcribed text, and event summaries to support semantic retrieval; the timeline database maintains the order and trajectory segments of events by user, device, and date; the graph relation storage maintains the associations between original media, multimodal data units, events, subjects, locations, objects, behaviors, time, and generated content to form a hierarchical event memory graph; lifecycle management configures media objects as hot storage, warm storage, or cold storage based on event importance, access frequency, retention period, and user selection, and performs full lifecycle management such as deduplication and redundant backup of high-value data.

[0063] 4. Multimodal event boundary detection and hierarchical fusion like Figure 6 , Figure 7 As shown, after the cloud-based multimodal processing system reads the multimodal data unit of the same device, it first performs timestamp calibration: based on the GNSS time, device clock, cloud reception time and clock calibration record, it calculates the time offset within the same upload session, corrects the data timestamp, and retains the original timestamp and correction parameters to ensure data traceability.

[0064] The system generates candidate event boundaries by detecting change points from multiple dimensions: location change points are generated when the location of the dwelling changes, the speed crosses the threshold, or the mode of movement changes; visual change points are generated when the scene feature distance between adjacent static images or video keyframes exceeds the limit, or when a camera switch is detected; audio change points are generated when the category of ambient sound or acoustic event changes; and motion change points are generated when the device activity state changes.

[0065] For the multimodal data units on both sides of the candidate event boundary, the system performs fusion decision according to a three-layer progressive logic: The first layer is time constraint filtering, which determines whether the time interval between the two data units meets the preset threshold requirements. If the time interval exceeds the limit, it is directly determined as different events.

[0066] The second layer is a spatial continuity constraint filter, which calculates the positioning distance between the two sides of the data and dynamically generates a spatial distance threshold based on the movement speed. The threshold calculation formula is as follows:

[0067] In the formula, Based on the basic distance threshold, The average moving speed within the time window. This represents the time interval between the data units on both sides. If the positioning distance exceeds the dynamic space threshold, it is determined to be a different event.

[0068] The third layer is the reliability-weighted fusion similarity calculation. When the hard constraints of the first two layers are satisfied, the fusion similarity is calculated based on the available modes. The calculation formula is as follows:

[0069] In the formula, For the set of available modalities, at least two categories should be selected from visual, audio, motion, and historical event continuity. For the fusion weights of the corresponding modes, The reliability coefficient is the corresponding modality. Similarity scores are assigned to corresponding modalities, including visual similarity, audio similarity, consistency of motion states, and continuity of historical events.

[0070] The system uses a threshold higher than θ. split Merging threshold θ mergePerform hysteresis decision: when fusion similarity ≥θ merge When merging candidate boundary data units, they are grouped into the same event; when merging similarity... <θ split When a change in the location of stay occurs, or when at least two modalities change significantly at the same time, the boundaries of candidate events are preserved; when the score is between two thresholds, the judgment result of the previous state is continued to avoid repeated merging and segmentation of events in the critical state.

[0071] After event segmentation and merging, the system creates an event object for each event. Each event object is associated with at least a start time, end time, trajectory segment, representative image / video clip, audio clip, event tag, event score, media object reference, and processed version. All events are sorted by time to form an event timeline, which serves as a unified input for subsequent content generation.

[0072] 5. Content generation and evidence consistency verification The content generation system uses the same version of event objects and event timelines as unified inputs to generate digital diaries, itinerary summaries, and short videos, ensuring that the time, location, and event logic of different types of content are consistent.

[0073] For digital diaries and itinerary summaries, the system first converts event objects into structured event representations containing time, location, subject, action, audio summary, and evidence identifiers, and then generates natural language content. After generation, the system breaks down the content into multiple content statement units, associating each unit with at least one event object identifier or original media object identifier, and calculates evidence coverage and content conflict degree. When the evidence coverage is below a preset threshold, or the content conflict degree is above a preset threshold, the corresponding content statement unit is deleted, corrected, regenerated, or marked as pending verification to ensure that all generated content is supported by collected data.

[0074] For short video generation, such as Figure 8 As shown, the system constructs a media candidate set from event-related static images and original video clips, and calculates a performance score for each candidate media. The score is based on at least two of the following: clarity, event rarity, subject salience, motion variation, acoustic salience, and temporal coverage. Under constraints such as target video duration, adjacent content repetition limits, privacy authorization limits, and event temporal coverage limits, the optimal media clip is selected. The selected static image or video clip is then cropped, spliced, or its duration adjusted, and then arranged and output with trajectory animation, time and location captions, event summaries, ambient sound, and narration. In the generated short video, each shot saves the corresponding event object identifier and original media object identifier, distinguishing between the originally captured video and the cloud-generated video.

[0075] 6. Event lineage tracing and version management like Figure 11 As shown, the system constructs a full-link event lineage tracing diagram between the original media object, multimodal data unit, event object, event timeline and generated content. Each level of object stores the object's unique identifier, content verification value, parent object identifier and processing version, realizing step-by-step reverse positioning from generated content to original media.

[0076] The original media object record must include at least the media object identifier, content verification value, device identifier, acquisition time, and storage address; the multimodal data unit record must include at least the data unit identifier, the parent object identifier set, and the referenced original media identifier set; the event object record must include at least the event identifier, event version, parent event identifier, referenced data units and media sets, and fusion parameter version; the event timeline record must include at least the timeline identifier, the referenced event set, and the timeline version; and the generated content record must include at least the generated content identifier, parent generated content identifier, referenced timeline / event / media identifier, generation model or rule version, generation configuration version, generation time, the mapping relationship between content statement units and evidence, and the generated content verification value.

[0077] For any generated content, its complete traceability tuple can be represented as structured data containing generated content identifier, parent generated content identifier, timeline identifier, event identifier set, data unit identifier set, media identifier set, media content verification value set, event version, model version, configuration version, generation time, and generated content verification value.

[0078] When responding to a source verification request, the system traces the lineage back to the original media object, recalculates the content verification value of the accessible original media and compares it with the recorded value. At the same time, it verifies whether the event version, timeline version, model version, configuration version and evidence mapping relationship are consistent with the generated record, thus completing the end-to-end consistency verification.

[0079] When the fusion algorithm, generation model, and configuration template are upgraded, the system reprocesses the original media and multimodal data units, assigning new version numbers to new event objects, timelines, or generated content while retaining the parent version identifier. Simultaneously, it locates affected downstream nodes based on the lineage map, performing partial regeneration only on affected event entries, diary segments, or video clips, without requiring full reprocessing. Older versions can be archived or replaced according to user policies, and version change and operation records are kept entirely online.

[0080] When a user deletes the original media object, the system synchronously updates the associated metadata, vector index, event relationships, and the availability status of the generated content. If the generated content is still authorized to be retained, the missing source is marked. If complete deletion is required, the corresponding generated content is deleted and the deletion audit record is retained.

[0081] 7. Security, Privacy, and Anomaly Handling The system employs two-way authentication between the terminal and the cloud, with end-to-end data transmission encrypted. Both local cache and cloud media objects support encrypted storage. Users can set geofences to automatically disable audio capture, reduce the frequency of still image capture, prohibit or restrict dynamic video recording, or retain only local data within designated areas. The system can detect sensitive subjects such as faces, license plates, and screens in still images and dynamic videos, and perform masking processing on sensitive areas in the generated content based on user permission configurations.

[0082] Access permissions can be configured in a fine-grained manner, such as by account, family member, device, time range, event type, and individual media object. All operations, including reading, exporting, sharing, deleting, source verification, and model processing, are written to the audit log.

[0083] For abnormal scenarios, the system has corresponding consistency guarantee mechanisms: When the same media segment is uploaded repeatedly, the cloud identifies the duplicate object based on the content verification value and device identifier, writes it to the object storage only once, and associates multiple upload records with the same object; if some segments fail verification, the cloud requests the terminal to retransmit the corresponding segments. When a sensor's data is temporarily missing, the event score and fusion similarity only use available modalities and renormalize the weights, while writing the missing modalities and their time ranges into the data unit and event object to avoid misinterpreting missing data as no event. When the content generation task fails, the message queue records the task status and the number of retries, and the retry task continues to use the same event version and generation configuration; if the model or configuration changes, a new generation version is created. When the consistency verification of the generated content fails, the system records the failed content statement unit, the corresponding evidence identifier, evidence coverage, and content conflict degree. Without changing the unaffected content, the failed unit is deleted, corrected, regenerated, or marked as pending verification to avoid the same version identifier corresponding to different processing results.

[0084] Example 6: Pet Camera Application This embodiment applies the above system to a pet monitoring scenario, with the intelligent recording terminal mounted on the pet's collar or harness.

[0085] The device captures a still image every 60 seconds by default. The positioning module records location information at configurable intervals, and audio is continuously captured and written to a circular buffer. Recording is disabled by default. Users can remotely control the device to take photos, start recording, stop recording, or capture video clips of preset durations via their mobile phones at close or long distances.

[0086] When the motion sensing module detects that the pet changes from stationary to running, the positioning module detects that the pet enters a new area, the microphone detects a significant sound, or the vision module detects other animals / people, the device automatically shortens the static image acquisition interval, starts dynamic video recording, extends the duration of the current video segment, increases the video frame rate or bit rate, and retains dynamic video and audio segments before and after the event.

[0087] When the device is connected to the home Wi-Fi network, it automatically uploads all collected data in full; when away from home, it prioritizes uploading location information, representative images, video keyframes, and low-bitrate video preview clips via cellular network to save data usage.

[0088] Based on uploaded multimodal data, the cloud automatically generates the pet's daily activity trajectory, location distribution, interaction event records, event timeline, digital diary, and exciting short videos. Users can view all content on their devices and perform source verification on any generated content to trace the corresponding original media data.

[0089] Example 7, Action Cameras and Other Applications This embodiment applies the above system to outdoor sports recording scenarios, with the terminal installed as an action camera on a helmet, chest strap, bicycle, backpack, or vehicle.

[0090] In action camera scenarios, continuous recording, segmented recording, or event-triggered recording modes can be enabled to increase the sampling frequency of motion and positioning. When key motion states such as speed peaks, turning movements, altitude changes, and bumps are detected, the static image acquisition frequency is automatically increased, or the frame rate, resolution, and bit rate of dynamic videos are increased. Static images and original dynamic video clips near turning, speed peaks, altitude changes, stillness, and significant sounds are prioritized for preservation.

[0091] After the exercise, the cloud automatically generates the exercise route, exercise data statistics, event timeline, exercise diary, and highlight video. Users can manually mark key events, and the system automatically associates all media and route data for the corresponding time period.

[0092] Besides pet and exercise recording, the terminal can also be used as a wearable camera, a children's activity recording device, an elderly care recording device, or a vehicle recording device. The relationship between the terminal's form in various scenarios and the long-term digital memory stored in the cloud is illustrated in the following diagram. Figure 9 , Figure 12 As shown. In scenarios involving personnel, the system enables recording prompts, privacy area blocking, sensitive subject masking, and access control functions in accordance with legal requirements and user authorization.

[0093] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention are also within the protection scope of the present invention.

Claims

1. A smart terminal adaptive data acquisition and cloud event generation system based on multimodal fusion, characterized in that, include: Intelligent recording terminals, cloud platforms, and user terminals; The intelligent recording terminal includes an image acquisition module, an audio acquisition module, a positioning module, a communication module, a motion sensing module, a local storage module, and an edge processing module. It is used to continuously or periodically acquire positioning data and audio data, and to acquire visual media data according to a preset acquisition strategy, manual triggering commands, or remote control commands. The visual media data includes at least one of static image data and dynamic video data. The dynamic video data includes continuous recording data or video clips. Time information is written to the positioning data, audio data, and visual media data based on a unified time reference, forming a multimodal data unit and uploading it to a cloud platform. The edge processing module is used to calculate a unified event importance score based on the degree of change in motion, position, sound, and visual data, the reliability of corresponding sensor data, cross-modal consistency, and historical continuity with preceding events. Based on the unified event importance score, it collaboratively adjusts the visual media acquisition strategy, positioning sampling frequency, audio retention range, local cache level, and upload priority. The cloud platform includes an access service, a storage system, a multimodal processing system, and a content generation system. The multimodal processing system is used to sequentially filter based on time constraints and spatial continuity constraints, and then determine the event boundary based on a comparison between the reliability-corrected fusion similarity and a preset merging threshold and a segmentation threshold, wherein the merging threshold is higher than the segmentation threshold. It merges multiple multimodal data units into multiple event objects and forms an event timeline based on the event objects. The content generation system is used to generate at least one of digital diaries, itinerary summaries, and short videos based on the event timeline. It establishes an event lineage tracing relationship between the original media object, multimodal data unit, event object, and generated content, and performs consistency verification between the content statement unit in the generated content and the corresponding event object or original media evidence. The user terminal is used to display trajectories, event timelines, or generated content, and to control the intelligent recording terminal to perform static image capture, dynamic video recording, or set acquisition strategies.

2. The intelligent terminal adaptive acquisition and cloud event generation system based on multimodal fusion according to claim 1, characterized in that: The preset acquisition strategy includes static image acquisition interval and dynamic video recording parameters. The static image acquisition interval is any value from 10 seconds to 30 minutes, with a default value of 60 seconds. When the intelligent recording terminal is in a low-activity state, it extends the static image acquisition interval and performs at least one of the following operations: reducing the dynamic video frame rate or bit rate, shortening the dynamic video retention time, and stopping recording. When it detects a change in position, speed, posture, significant sound, significant visual target, or remote shooting command, it performs at least one of the following operations: shortening the static image acquisition interval, immediately capturing a static image, starting dynamic video recording, extending the video segment duration, and increasing the dynamic video frame rate or bit rate.

3. The intelligent terminal adaptive acquisition and cloud event generation system based on multimodal fusion according to claim 1, characterized in that: The audio acquisition module writes continuous audio into a circular buffer in segments according to a preset duration. The edge processing module retains audio segments before and after an acoustic event based on the occurrence time and event score of the acoustic event, and performs downsampling, compression, or deletion processing on audio segments for which no effective acoustic features are detected. The edge processing module is also used to perform quality screening and duplication suppression on multimodal data units. The quality screening includes performing at least one of the following on static images or dynamic videos: blur detection, exposure detection, occlusion detection, and video frame drop detection; performing signal-to-noise ratio detection on audio; and performing validity detection on positioning data. The duplication suppression identifies duplicate or nearly duplicate data based on at least two of the following: perceptual hash of static image or video keyframes, visual feature vector, video fingerprint, audio fingerprint, location distance, and time distance, and prioritizes retaining data with higher quality scores.

4. The intelligent terminal adaptive acquisition and cloud event generation system based on multimodal fusion according to claim 1, characterized in that: The user terminal sends remote shooting commands via at least one of Bluetooth Low Energy communication, Wi-Fi communication, or cellular network. These commands include at least one of a photo capture command, a start recording command, a stop recording command, and a preset duration video clip acquisition command. Each remote shooting command has a command identifier, command type, generation time, and validity period. After verifying the device identity and validity period, the smart recording terminal performs the corresponding shooting and returns the shooting result. The smart recording terminal also includes physical buttons or touch input components for triggering manual photo capture, starting recording, or stopping recording. Data to be uploaded is written to a breakpoint resume queue. Queue items include at least data fragments, content verification values, event scores, data types, creation time, and upload status. Upload priority is determined by at least two of the following: event score, data type, network quality, remaining battery power, and local storage capacity. Under weak network or cellular network conditions, at least one of the following is prioritized for upload: location data, event metadata, representative images, video keyframes, low-bitrate video preview clips, and low-bitrate audio summaries. Uploading high-resolution still images or original dynamic videos is delayed.

5. The intelligent terminal adaptive acquisition and cloud event generation system based on multimodal fusion according to claim 1, characterized in that: The edge processing module calculates motion scores, location scores, sound scores, and visual scores respectively, and determines a reliability coefficient based on the validity of the sensor data corresponding to each score. Based on the normalized weighted result of the available scores and their corresponding reliability coefficients, cross-modal consistency within the same time window (where cross-modal consistency refers to the degree of matching between visual, audio, and motion data in terms of event semantics or scene changes within the same time window, the historical continuity of current data with preceding events in terms of time, trajectory, or semantics, user trigger priorities, and data quality defect suppression items), a unified event importance score is obtained. The edge processing module performs time smoothing on the unified event importance score and uses a first threshold. The system switches between states with hysteresis characteristics based on the value and a second threshold below the first threshold. When the smoothed unified event importance score reaches the first threshold, it enters the important event state. When the smoothed unified event importance score is below the second threshold and continues to be below the preset duration, it enters the low activity state. The start conditions, stop conditions, segment duration, frame rate, resolution, or bit rate of dynamic video recording are determined based on the smoothed unified event importance score, and secondary constraints are applied based on at least one of the remaining battery power, device temperature, and local storage capacity. The same unified event importance score is used to collaboratively determine the visual media acquisition strategy, positioning sampling frequency, audio retention range, local cache level, and upload priority.

6. The intelligent terminal adaptive acquisition and cloud event generation system based on multimodal fusion according to claim 1, characterized in that: The storage system includes object storage and a metadata database. The object storage is used to save static images, raw dynamic videos, audio, generated videos, and trajectory rendering files. The metadata database is used to save device identifiers, unified timestamps, trajectory points, media object identifiers, media content verification values, event identifiers, access permissions, and processing versions. Each event object is associated with at least a start time, end time, trajectory segment, representative image or video clip, audio clip, event tag, event score, and media object reference. The content generation system establishes an event lineage tracing graph between the original media object, multimodal data unit, event object, event timeline, and generated content. Each level of object in the event lineage tracing graph stores a unique object identifier, content verification value, parent object identifier, processing version, and processing time. The generated content also stores the event object identifier and media object identifier it references, the generation model or rule version, the generation configuration version, and the generated content verification value. When event processing or content generation is re-executed, a new version is created for the new result while retaining the parent version identifier, enabling the generated content to be back located to the corresponding event object, multimodal data unit, and original media object. The content generation system also splits the generated content into multiple content statement units, associates at least one event object identifier or media object identifier with each content statement unit, and calculates the evidence coverage rate and content conflict degree. When the evidence coverage rate is lower than a preset threshold or the content conflict degree is higher than a preset threshold, the corresponding content statement unit is deleted, corrected, regenerated, or marked as pending verification.

7. A method for generating multimodal events and digital content using a smart recording terminal and a cloud platform, characterized in that, include: The intelligent recording terminal continuously or periodically collects positioning data and audio data, and collects visual media data according to a preset collection strategy, manual triggering command or remote shooting command. The visual media data includes at least one of static image data and dynamic video data, and the dynamic video data includes continuous recording data or video clips. Time information is written to positioning data, audio data, and visual media data based on a unified time reference, and time alignment is performed on the collected data to form multimodal data units; A unified event importance score is calculated based on the degree of change in motion, position, sound, and visual data, the reliability of corresponding sensor data, cross-modal consistency, and historical continuity with preceding events. The visual media acquisition strategy, location sampling frequency, audio retention range, local cache level, and upload priority are then adjusted in a coordinated manner based on the unified event importance score. Upload the multimodal data units to the cloud platform; The cloud platform sequentially filters data based on time constraints and spatial continuity constraints, and then determines the event boundary by comparing the reliability-corrected fusion similarity with preset merging and segmentation thresholds, wherein the merging threshold is higher than the segmentation threshold. Multiple multimodal data units are merged into multiple event objects, and an event timeline is formed based on the event objects. The system generates at least one of digital diaries, itinerary summaries, and short videos based on the event timeline, establishes an event lineage tracing relationship between the original media object, multimodal data unit, event object, and generated content, and performs consistency verification on the content statement unit in the generated content with the corresponding event object or original media evidence.

8. An intelligent recording terminal, characterized in that, It includes a processor, memory, image acquisition module, audio acquisition module, positioning module, communication module, and motion sensing module; The memory stores a computer program. When the computer program is executed by the processor, the intelligent recording terminal continuously or periodically collects positioning data and audio data, and collects visual media data according to a preset collection strategy, manual triggering command or remote shooting command. The visual media data includes at least one of static image data and dynamic video data. The dynamic video data includes continuous recording data or video clips. Time information is written into various types of collected data based on a unified time reference to form multimodal data units; A unified event importance score is calculated based on the degree of change in motion, position, sound, and visual data, the reliability of corresponding sensor data, cross-modal consistency, and historical continuity with preceding events. The visual media acquisition strategy, location sampling frequency, audio retention range, local cache level, and upload priority are then adjusted in a coordinated manner based on the unified event importance score. Multimodal data units are sent to the cloud platform in a data format that includes time information, location information, media object identifier, content verification value, and unified event importance score.

9. A cloud server system, characterized in that, It includes at least one processor and a memory communicatively connected to the at least one processor; The memory stores a computer program. When the computer program is executed by the processor, the cloud server system receives location data, audio data, visual media data, and unified event importance scores uploaded by the smart recording terminal. The visual media data includes at least one of static image data and dynamic video data. The dynamic video data includes continuous recording data or video clips. The event boundaries are determined by filtering according to time constraints and spatial continuity constraints, and then by comparing the reliability-corrected fusion similarity with the preset merging threshold and segmentation threshold, wherein the merging threshold is higher than the segmentation threshold. Multiple multimodal data units are merged into multiple event objects, and an event timeline is formed based on the event objects. Generate at least one of the following based on the event timeline: digital diary, itinerary summary, and short video; Establish an event lineage tracing relationship between the original media object, multimodal data unit, event object, and generated content, and perform consistency verification on the content statement unit in the generated content and the corresponding event object or original media evidence.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method of claim 7.

Images