Multi-device intelligent collaborative interaction system based on distributed soft bus

Abstract

The invention discloses a multi-device intelligent collaborative interaction system based on a distributed soft bus, which relates to the technical field of collaborative interaction, and comprises a fusion module for fusing multi-modal context data of a sensor in a personal device network by utilizing a lightweight intention reasoning model based on a global virtual function center resource directory, the method comprises the following steps of: generating a structured collaborative task descriptor by a collaborative task description module, decomposing the collaborative task descriptor into atomic sub-tasks by an adaptation module, matching a virtual function center from a global virtual function center resource directory, dynamically evaluating the adaptability according to a network state and an equipment load, and generating an optimal collaborative task execution path; according to the method, the cooperation task descriptor is analyzed and decomposed into the atomic subtasks, the virtual function center is matched, the adaptability is evaluated based on the optimization algorithm of the real-time network state, the load and the electric quantity, the optimal cooperation task execution path is generated, the limitation of single index optimization is overcome, and the resources are distributed in a balanced mode by integrating the multi-dimensional performance.

Description

Technical Field

[0001] This invention relates to the field of collaborative interaction technology, and in particular to a multi-device intelligent collaborative interaction system based on a distributed soft bus. Background Technology

[0002] In recent years, with the rapid development of IoT technology, the interconnection and collaborative interaction between heterogeneous devices has become a core research direction in the field of intelligent interaction. Distributed soft buses, as a lightweight, cross-platform communication framework, are widely used to achieve dynamic discovery and data interaction between devices. In existing systems, device capabilities are typically registered to a resource pool in the form of hardware functions, supporting task allocation and execution. For example, in smart home scenarios, smart speakers, cameras, and mobile phones form a network via Wi-Fi or Bluetooth protocols to collaboratively complete voice control or monitoring tasks. Furthermore, lightweight neural network models are increasingly being applied to edge devices to process multimodal data and infer user intent or environmental states. Task decomposition and execution path optimization techniques are also being applied in distributed systems.

[0003] However, existing collaborative interaction systems still have room for improvement: current intent reasoning methods are mostly based on single-modal data (such as voice or image), and have limited ability to fuse multimodal contextual data, making it difficult to comprehensively capture user behavior patterns and complex environmental states, thus affecting the accuracy of task generation. In addition, task allocation and execution path optimization often only consider single indicators such as network latency or device load, lacking a comprehensive and dynamic evaluation of multi-dimensional performance such as network status, load, and power consumption, which can easily lead to uneven resource allocation or low task execution efficiency. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a multi-device intelligent collaborative interaction system based on a distributed soft bus to solve the problem of difficulty in fully capturing user behavior patterns and complex environmental states, which affects the accuracy of task generation.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] This invention provides a multi-device intelligent collaborative interaction system based on a distributed soft bus, comprising:

[0008] The registration module establishes a network of personal devices and abstracts the device hardware capabilities into a virtual function center before registration, resulting in a dynamically updated global virtual function center resource catalog.

[0009] The fusion module, based on the global virtual function center resource catalog, uses a lightweight intent reasoning model to fuse multimodal context data from sensors in the personal device network to generate a structured collaborative task descriptor.

[0010] The adaptation module decomposes the collaborative task descriptor into atomic subtasks, matches virtual function centers from the global virtual function center resource directory, and dynamically evaluates the adaptability based on network status and device load to generate the optimal collaborative task execution path.

[0011] The execution module uses a distributed soft bus to establish a secure data flow channel for the optimal collaborative task execution path, then instantiates and executes the task flow, and continuously monitors it to optimize the path.

[0012] As a preferred embodiment of the multi-device intelligent collaborative interaction system based on a distributed soft bus described in this invention, the step of establishing a personal device network, abstracting the device hardware capabilities into a virtual function center, and registering them to obtain a dynamically updated global virtual function center resource catalog specifically involves:

[0013] The distributed soft bus is used to discover heterogeneous IoT devices within the communication range via multicast domain name service protocol, and the basic identification information of the heterogeneous IoT devices is recorded.

[0014] Based on the basic identification information recorded, public key infrastructure is used to authenticate devices to each other and generate trusted authentication tokens;

[0015] A personal device network is established based on a trusted authentication token. The hardware capabilities of the devices in the personal device network are abstracted to generate a virtual function center. The virtual function center is then registered to a distributed soft bus and aggregated to form a global virtual function center resource catalog.

[0016] As a preferred embodiment of the multi-device intelligent collaborative interaction system based on a distributed soft bus according to the present invention, the step of generating a structured collaborative task descriptor by fusing multimodal context data from sensors in a personal device network using a lightweight intent reasoning model based on a global virtual function center resource catalog is as follows:

[0017] Obtain the function type and network address of the sensors within the personal device network from the global virtual function center resource directory;

[0018] Based on the sensor function type, the sensors are scheduled to collect multimodal context data and input into a lightweight intent reasoning model to extract user behavior patterns and environmental state features.

[0019] User behavior patterns and environmental state characteristics are mapped to predefined task templates to generate structured collaborative task descriptors.

[0020] As a preferred embodiment of the multi-device intelligent collaborative interaction system based on a distributed soft bus described in this invention, the step of decomposing the collaborative task descriptor into atomic subtasks, matching virtual function centers from the global virtual function center resource directory, and dynamically evaluating adaptability based on network status and device load to generate the optimal collaborative task execution path specifically involves:

[0021] The task objective of the collaborative task descriptor is parsed and decomposed into atomic subtasks in a time sequence;

[0022] Query the global virtual function center resource catalog, match the virtual function centers corresponding to the functional requirements of the atomic subtasks, and collect the real-time network status of personal device networks and the current load information of the virtual function centers.

[0023] Based on real-time network status and load information, an optimization algorithm is used to evaluate the matching and adaptability of virtual functional centers and generate the optimal collaborative task execution path.

[0024] As a preferred embodiment of the multi-device intelligent collaborative interaction system based on a distributed soft bus according to the present invention, the step of establishing a secure data flow channel for the optimal collaborative task execution path using the distributed soft bus, and then instantiating and executing the task flow, specifically involves:

[0025] Based on the optimal collaborative task execution path, determine the virtual functional centers and network addresses participating in the task;

[0026] By using a distributed soft bus and a transport layer security protocol, an encrypted data stream channel is established between virtual function centers, and the optimal collaborative task execution path is instantiated in the encrypted data stream channel to generate task stream instances.

[0027] The scheduling virtual function center executes atomic subtasks according to task flow instances.

[0028] As a preferred embodiment of the multi-device intelligent collaborative interaction system based on a distributed soft bus described in this invention, the scheduling virtual function center executes atomic subtasks according to task flow instances, and transmits and processes data streams, specifically as follows:

[0029] Analyze the task flow instance and extract the execution logic and data flow order of each atomic subtask;

[0030] Based on the execution logic and data flow sequence, the virtual function center is invoked to process the data flow required by the atomic subtask. The processed data flow is then transmitted and aggregated between the virtual function centers via a distributed soft bus to complete the execution of the atomic subtask.

[0031] As a preferred embodiment of the multi-device intelligent collaborative interaction system based on a distributed soft bus described in this invention, the continuous monitoring to optimize the path specifically includes:

[0032] The topology changes of personal device networks and the resource status of virtual function centers are monitored in real time through a distributed soft bus. If a change in resource status is detected, the adaptability of the virtual function centers is reassessed based on the change in resource status, and the collaborative task execution path is updated.

[0033] The updated execution path is instantiated into a new task flow instance through a distributed soft bus.

[0034] As a preferred embodiment of the multi-device intelligent collaborative interaction system based on a distributed soft bus according to the present invention, the extraction of user behavior patterns and environmental state features specifically includes:

[0035] After preprocessing the multimodal contextual data, it is input into a lightweight intent reasoning model to extract time series features and spatial behavior features, and then fuse them into a user behavior pattern.

[0036] By combining user behavior patterns and raw environmental information from multimodal context data, environmental state features are generated.

[0037] As a preferred embodiment of the multi-device intelligent collaborative interaction system based on a distributed soft bus according to the present invention, the step of evaluating the adaptability of the virtual functional center based on real-time network status and load information using an optimization algorithm specifically includes:

[0038] Quantify real-time network status, calculate weighted performance metrics for latency and bandwidth, and assess the current load and power status of the virtual function center;

[0039] Based on weighted performance metrics, current load, and power status, a suitability scoring matrix for virtual function centers is constructed, and the combination with the highest score is selected as the optimal combination of virtual function centers.

[0040] As a preferred embodiment of the multi-device intelligent collaborative interaction system based on a distributed soft bus according to the present invention, the step of re-evaluating the adaptability of the virtual function center and updating the collaborative task execution path according to changes in resource status specifically includes:

[0041] Identify changes in resource status, record the affected virtual function centers and performance parameters, and filter for alternative virtual function centers from the global virtual function center resource directory;

[0042] Based on the service quality requirements of the atomic subtasks, the adaptability of alternative virtual function centers is reassessed, new collaborative task execution paths are generated, and the affected virtual function centers are replaced.

[0043] The beneficial effects of this invention are as follows: It extracts user behavior patterns and environmental state features through a lightweight intent reasoning model, maps them to task templates to generate structured collaborative task descriptors, and integrates video, voice, acceleration, and location data. Compared to existing single-modal reasoning, it comprehensively captures the rider's line of sight and road conditions, improving the accuracy of task generation. Furthermore, by parsing the collaborative task descriptor into atomic subtasks and matching them with a virtual function center, it evaluates adaptability based on optimization algorithms using real-time network status, load, and battery power, generating the optimal collaborative task execution path. This overcomes the limitations of single-index optimization, comprehensively balances resource allocation across multiple dimensions, and improves the execution efficiency and real-time performance of navigation and safety prompt tasks in cycling scenarios. Attached Figure Description

[0044] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a flowchart of a multi-device intelligent collaborative interaction system based on a distributed soft bus.

[0046] Figure 2 This is a flowchart for the registration module.

[0047] Figure 3 The flowchart is for integration and adaptation.

[0048] Figure 4 A flowchart for execution and optimization. Detailed Implementation

[0049] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0050] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0051] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0052] Reference Figures 1-4 This is one embodiment of the present invention, which provides a multi-device intelligent collaborative interaction system based on a distributed soft bus, comprising the following steps:

[0053] The registration module establishes a network of personal devices and abstracts the device hardware capabilities into a virtual function center before registration, resulting in a dynamically updated global virtual function center resource catalog.

[0054] The distributed soft bus is used to discover heterogeneous IoT devices within communication range via multicast domain name service protocol, and the basic identification information of the devices is recorded, specifically:

[0055] The distributed soft bus utilizes a multicast DNS service protocol to discover heterogeneous IoT devices within communication range. For example, a rider's smartphone, smart helmet, smartwatch, and vehicle dashboard might be located on the same Wi-Fi network. The distributed soft bus broadcasts multicast DNS service protocol messages, and the smartphone, smart helmet, smartwatch, and vehicle dashboard respond, returning their respective device identification information, including device type, MAC address, and IP address. This device identification information is recorded in a temporary device list on the distributed soft bus.

[0056] Based on the basic identification information recorded, public key infrastructure is used to authenticate devices to each other and generate trusted authentication tokens, specifically:

[0057] Public and private key pairs are generated for smartphones, smart helmets, smartwatches, and vehicle dashboards. Public keys are exchanged via a distributed soft bus, and the legitimacy of the other party's public key is verified using a public key infrastructure. After successful verification, a trusted authentication token containing the user ID and the authentication status of the smartphone, smart helmet, smartwatch, and vehicle dashboard is generated based on the rider's user identity.

[0058] Establishing a personal device network based on a trusted authentication token involves the following steps:

[0059] The distributed soft bus associates smartphones, smart helmets, smartwatches, and vehicle dashboards into a user-centric personal device network based on the user ID in the trusted authentication token. It assigns network address A to smartphones, network address B to smart helmets, network address C to smartwatches, and network address D to vehicle dashboards, and stores the network topology structure containing network addresses A, B, C, and D.

[0060] The hardware capabilities of devices in a personal device network are abstracted to generate virtual functional centers. The process is as follows: The high-performance AI computing power and high-precision GPS positioning function of a smartphone are abstracted into a high-performance AI computing center and a high-precision GPS positioning center. The function type of the high-performance AI computing center is recorded as AI computing, computing performance parameters, and current availability. The function type of the high-precision GPS positioning center is recorded as positioning, positioning performance parameters, and current availability. The front-facing camera and noise reduction processing function of a smart helmet are abstracted into a front-facing camera center and a noise reduction processing center. The function type of the front-facing camera center is recorded as video acquisition, video acquisition parameters, and current availability. The function type of the noise reduction processing center is recorded as audio processing, audio processing parameters, and current availability. The accelerometer function of a smartwatch is abstracted into an accelerometer center. The function type of the accelerometer center is recorded as motion detection, motion detection parameters, and current availability. The recognition display and touch control functions of an in-vehicle instrument panel are abstracted into a display output center and a touch interaction center. The function type of the display output center is recorded as display, display parameters, and current availability. The function type of the touch interaction center is recorded as interaction, interaction parameters, and current availability.

[0061] Smartphones, smart helmets, smartwatches, and in-vehicle dashboards integrate a high-performance AI computing center, a high-precision GPS positioning center, a front-facing camera center, a noise reduction frequency processing center, an accelerometer center, a display output center, and a touch interaction center, forming a virtual functional center set that includes function types, performance parameters, and current availability status.

[0062] The virtual function centers are registered to the distributed soft bus, and a global virtual function center resource catalog is formed, specifically as follows:

[0063] The distributed soft bus integrates information from the high-performance AI computing center, high-precision GPS positioning center, front-facing camera center, noise reduction frequency processing center, accelerometer center, display output center, and touch interaction center to generate a global virtual function center resource catalog containing the function types, performance parameters, current available status, and network addresses of all virtual function centers. When the status of a smartphone, a smart helmet, smartwatch, or vehicle instrument changes, or a new device joins the personal device network, the distributed soft bus adjusts the global virtual function center resource catalog based on the received update information.

[0064] The fusion module, based on the global virtual function center resource catalog, uses a lightweight intent reasoning model to fuse multimodal context data from sensors in the personal device network to generate a structured collaborative task descriptor.

[0065] Obtain the function type and network address of the sensors within the personal device network from the global virtual function center resource directory, specifically:

[0066] In the scenario where a rider is preparing to start riding, the distributed soft bus queries the global virtual function center resource directory for the function types and network addresses of the smartphone, the smart helmet, the smartwatch, and the vehicle instrument panel. It extracts the function type of the front camera center of the smart helmet as video acquisition and the network address as B, the function type of the noise reduction frequency processing center of the smart helmet as audio processing and the network address as B, the function type of the accelerometer center of the smartwatch as motion detection and the network address as C, and the function type of the high-precision GPS positioning center of the smartphone as positioning and the network address as A.

[0067] Based on the sensor function type, the sensors are scheduled to collect multimodal context data. The specific process is as follows:

[0068] The distributed soft bus schedules the front-facing camera center of the smart helmet to collect video data based on its function of video acquisition; it schedules the noise reduction audio processing center of the smart helmet to collect voice commands based on its function of audio processing; it schedules the accelerometer center of the smart watch to collect acceleration data based on its function of motion detection; and it schedules the high-precision GPS positioning center of the smartphone to collect location data based on its function of positioning.

[0069] The collected video data, voice commands, acceleration data, and location data are combined to form multimodal context data.

[0070] Multimodal contextual data is input into a lightweight intent reasoning model to extract user behavior patterns and environmental state features. The specific process is as follows:

[0071] The video data, voice commands, acceleration data, and location data in the multimodal context data are aligned with timestamps, unified into a standard time format, and their values ​​are normalized to a unified dimension to generate preprocessed multimodal context data. This preprocessed multimodal context data is then input into a lightweight intent reasoning model, which is built on an embedded neural network and includes an input layer, convolutional layers, fully connected layers, and an output layer. The input layer receives the preprocessed multimodal context data. The convolutional layers perform convolution operations on the pixel value sequences of the video data, combining image segmentation techniques to extract the rider's eye region, tracking changes in the pixel coordinates of the eye region, and generating time-series features representing changes in gaze direction over time. Convolution operations are also performed on the audio feature sequences of the voice commands, combining speech recognition techniques to extract keywords and semantic structure. Convolution operations are performed on the three-axis value sequences of the acceleration data to extract motion direction and velocity change patterns. Finally, convolution operations are performed on the latitude and longitude coordinate sequences of the location data, combining differential processing to compare latitude and longitude values ​​at adjacent time points, extracting changes in geographic movement trajectory direction and distance, and generating spatial behavior features representing the movement path. The fully connected layer integrates the gaze direction features, keyword features, semantic structure features, motion pattern features, and movement trajectory features extracted by the convolutional layer to generate a comprehensive feature vector; the output layer maps the comprehensive feature vector into time series features (representing the gaze direction change pattern) and spatial behavior features (representing the rider's movement trajectory pattern).

[0072] It should also be noted that keywords refer to high-information words in voice commands that are directly related to the rider's intentions or task objectives, such as "turn left," "turn right," "slow down," "stop," "navigate," and "obstacles," reflecting the rider's navigation or safety operation needs.

[0073] Time-series features and spatial behavior features are weighted and fused to generate user behavior patterns, including riders' gaze direction patterns and movement trajectory patterns.

[0074] User behavior patterns are combined with video data, voice commands, acceleration data, and location data from preprocessed multimodal context data to generate environmental state features, including road conditions and location changes in the cycling environment.

[0075] The training process of the lightweight intent reasoning model is as follows: A labeled dataset is collected, containing gaze direction data from video frames, keywords for voice commands, triaxial acceleration values, and latitude / longitude coordinates of location. In the labeled dataset, video frames are labeled with pixel coordinate changes in gaze direction, voice commands with keywords and semantic structures, acceleration values ​​with motion patterns along the triaxial acceleration values, and location with movement trajectories along the latitude / longitude coordinates. Gradient descent is used to adjust the parameters of the input, convolutional, fully connected, and output layers of the lightweight intent reasoning model, enabling it to learn the pixel coordinate change patterns in gaze direction, keywords and semantic structures in voice commands, triaxial acceleration motion patterns, and movement trajectory features along the latitude / longitude coordinates of location within video frames. Twenty percent of the labeled dataset is used as a validation set. The weights of the lightweight intent reasoning model are updated based on the prediction error of the validation set, ensuring that the lightweight intent reasoning model accurately recognizes gaze direction patterns, voice keywords, motion patterns, and movement trajectories on resource-constrained devices.

[0076] The user behavior patterns and environmental state features are mapped to predefined task templates to generate structured collaborative task descriptors. The specific process is as follows:

[0077] User behavior patterns and environmental state characteristics are matched with predefined task templates, which include navigation task templates and safety prompt task templates. Based on the matching results, navigation task objectives and safety prompt task objectives are extracted, real-time and reliability requirements are configured, and a structured collaborative task descriptor containing navigation task objectives, safety prompt task objectives, and real-time and reliability requirements is generated.

[0078] It should also be noted that the predefined task templates are rule sets that are pre-designed and stored based on typical task requirements and historical data in cycling scenarios. They include navigation task templates (such as "generate the optimal navigation route") and safety prompt task templates (such as "detect obstacles and issue an alarm").

[0079] Preferably, in existing methods, intent reasoning often relies on single-modal data (such as voice) processed by high-performance devices or on device interconnection using standard protocols in distributed IoT, making it difficult to efficiently integrate multimodal data. In contrast, this invention integrates multimodal contextual data such as video, voice, acceleration, and location data through a lightweight intent reasoning model to generate user behavior patterns and environmental state features, accurately match predefined task templates, generate structured collaborative task descriptors, achieve efficient intent recognition on resource-constrained devices, and improve the accuracy and real-time performance of intelligent collaboration in dynamic cycling scenarios.

[0080] The adaptation module decomposes the collaborative task descriptor into atomic subtasks, matches virtual function centers from the global virtual function center resource directory, and dynamically evaluates the adaptability based on network status and device load to generate the optimal collaborative task execution path.

[0081] The task objective of parsing the cooperative task descriptor is decomposed into sequential atomic subtasks, as follows:

[0082] The distributed soft bus parses the navigation task objective in the structured cooperative task descriptor and decomposes it into time-sequential atomic subtasks, including localization and tracking and path planning; at the same time, it parses the safety prompt task objective in the structured cooperative task descriptor and decomposes it into time-sequential atomic subtasks, which include obstacle detection and alarm output.

[0083] The global virtual function center resource directory is queried to match the virtual function center corresponding to the functional requirements of the atomic subtask, as follows:

[0084] The distributed soft bus queries the global virtual function center resource catalog to extract the high-precision GPS positioning center for smartphones corresponding to the positioning and tracking function requirements (function type: positioning, network address: A); the high-performance AI computing center corresponding to the path planning function requirements (function type: AI computing, network address: A); the smart helmet front camera center corresponding to the obstacle detection function requirements (function type: video acquisition, network address: B); and the vehicle instrument display output center corresponding to the alarm output function requirements (function type: display, network address: D).

[0085] Collect real-time network status of personal device networks and current load information of the virtual function center, as detailed below;

[0086] The distributed soft bus sends requests to smartphones, smart helmets, and vehicle dashboards via network addresses A, B, and D to obtain the real-time network status of personal device networks, including network latency and bandwidth from smartphones to smart helmets and from smart helmets to vehicle dashboards. This allows the system to obtain the current load information and power status of the high-precision GPS positioning center, high-performance AI computing center, front-facing camera center, and display output center. The load information includes the task queue length, and the power status includes the remaining battery percentage.

[0087] Based on real-time network status and load information, an optimization algorithm is used to evaluate the matching suitability of virtual functional centers and generate the optimal collaborative task execution path, as follows:

[0088] Weighting factors are assigned to the network latency and bandwidth from the smartphone to the smart helmet, and to the network latency and bandwidth from the smart helmet to the vehicle's instrument panel. Based on the assigned weighting factors, the network latency and bandwidth from the smartphone to the smart helmet and from the smart helmet to the vehicle's instrument panel are linearly combined to generate a weighted performance index of the real-time network status, expressed as:

[0089] ;

[0090] in, A weighted performance metric representing the real-time network status. A weighting factor representing the network latency from the smartphone to the smart helmet. This indicates the network latency from the smartphone to the smart helmet. A weighting factor representing the bandwidth from the smartphone to the smart helmet. This indicates the bandwidth from the smartphone to the smart helmet. The weighting factor represents the network latency from the smart helmet to the vehicle's dashboard. This indicates the network latency from the smart helmet to the vehicle's dashboard. The weighting factor represents the bandwidth from the smart helmet to the vehicle's instrument panel. This indicates the bandwidth from the smart helmet to the vehicle's instrument panel.

[0091] It should also be noted that the weighting factors are allocated based on the requirements of task real-time performance and data transmission efficiency. and The weighting factors are 0.4 and 0.3 respectively. If a higher weighting factor is chosen, the latency will be overemphasized, which may lead to the selection of a path with insufficient bandwidth, resulting in video data transmission stuttering and affecting the accuracy of obstacle detection. If the weighting factor is lower than 0.4 and 0.3, the latency effect will be ignored, delaying localization tracking and alarm output, and reducing the timeliness of safety prompts. and The values ​​are set to 0.2 and 0.1 respectively. If the values ​​are greater than 0.2 and 0.1, bandwidth will be overemphasized, resulting in the selection of high-latency paths, increasing the positioning and tracking response time, and affecting the real-time performance of navigation. If the values ​​are lower than 0.2 and 0.1, low-bandwidth paths will be selected because bandwidth requirements are ignored, causing data transmission congestion and affecting the integrity of video data in path planning.

[0092] The task queue length and remaining battery percentage of the high-precision GPS positioning center, high-performance AI computing center, front camera center, and display output center are integrated to generate the current load status and current battery status.

[0093] The distributed soft bus normalizes the reciprocal of the weighted performance metric, the reciprocal of the task queue length, and the remaining battery percentage to generate network performance values, load values, and battery values ​​for the high-precision GPS positioning center, high-performance AI computing center, front-facing camera center, and display output center. After assigning weight factors to the positioning and tracking task, path planning task, obstacle detection task, and alarm output task, an adaptability score is calculated based on the network performance value, load value, and battery value, using the following formula:

[0094] ;

[0095] in, Indicates virtual function center For the task The suitability score indicates the degree of matching. Indicates task Weighting factors Indicates virtual function center Normalized network performance values Indicates virtual function center The normalized load value, Indicates virtual function center The normalized energy value.

[0096] It should also be noted that the weighting factors for tasks are set based on their real-time performance and resource requirements. Specifically: the weighting factor for the location tracking task is 0.4. A weighting greater than 0.4 will result in excessive priority, consuming bandwidth and affecting obstacle detection video transmission; a weighting less than 0.4 will increase latency and cause untimely navigation. The weighting factor for the path planning task is 0.3. A weighting greater than 0.3 will lead to excessive allocation of computing resources and delayed location tracking; a weighting less than 0.3 will result in insufficient computation and inaccurate path planning. The weighting factor for the obstacle detection task is 0.2. A weighting greater than 0.2 will consume video resources and reduce the real-time performance of navigation; a weighting less than 0.2 will result in slow video processing and missed obstacle detection. The weighting factor for the alarm output task is 0.1. A weighting greater than 0.1 will waste resources and affect the execution of other tasks; a weighting less than 0.1 will cause alarm delays and reduce security.

[0097] The adaptability scores of high-precision GPS positioning centers for positioning and tracking tasks, high-performance AI computing centers for path planning tasks, front-facing camera centers for obstacle detection tasks, and display output centers for alarm output tasks are aggregated to generate an adaptability score matrix.

[0098] Select the combination of virtual function centers with the highest adaptability score from the adaptability score matrix, assign the positioning and tracking task to the high-precision GPS positioning center, the path planning task to the high-performance AI computing center, the obstacle detection task to the front camera center, and the alarm output task to the display output center, and generate the optimal collaborative task execution path that includes task allocation and data flow order.

[0099] Preferably, in existing interactive systems, IoT task allocation often relies on static resource scheduling or single performance metrics, making it difficult to dynamically adapt to the real-time status and load of heterogeneous device networks. This invention, however, decomposes atomic subtasks by parsing structured collaborative task descriptors, matches function centers based on a global virtual function center resource directory, and combines weighted performance metrics of real-time network status and load / power status to generate the optimal collaborative task execution path. This achieves dynamic adaptation, improves task allocation efficiency and execution real-time performance in resource-constrained environments, and optimizes navigation and safety prompt collaboration in cycling scenarios.

[0100] The execution module uses a distributed soft bus to establish a secure data flow channel for the optimal collaborative task execution path, then instantiates and executes the task flow, and continuously monitors it to optimize the path.

[0101] Based on the optimal collaborative task execution path, the virtual functional centers and network addresses participating in the task are determined. An encrypted data stream channel is established between the virtual functional centers via a distributed soft bus using a transport layer security protocol. The optimal collaborative task execution path is then instantiated within this encrypted data stream channel, generating task flow instances, as detailed below:

[0102] The distributed soft bus identifies the high-precision GPS positioning center, high-performance AI computing center, front-facing camera center, and display output center participating in the task through network addresses A, B, and D. Utilizing transport layer security protocols, encrypted data stream channels are established between the high-precision GPS positioning center and the high-performance AI computing center, between the high-performance AI computing center and the front-facing camera center, and between the front-facing camera center and the display output center. The optimal collaborative task execution path is loaded into the encrypted data stream channel to generate a task flow instance, which includes the execution logic and data flow sequence of the positioning and tracking task, path planning task, obstacle detection task, and alarm output task.

[0103] The scheduling virtual function center executes atomic subtasks according to task flow instances, and transmits and processes data streams, as follows:

[0104] The distributed soft bus parsing task flow example extracts the execution logic of the positioning and tracking task as collecting location data, the execution logic of the path planning task as calculating the navigation route, the execution logic of the obstacle detection task as analyzing video frames, and the execution logic of the alarm output task as displaying alarm information. The data flow order is as follows: location data flows to the high-performance AI computing center, navigation route flows to the front camera center, and video analysis results flow to the display output center.

[0105] According to the execution logic, the system calls a high-precision GPS positioning center to collect location data and generate a location data stream, calls a high-performance AI computing center to process the location data stream and generate a navigation route, calls a front-facing camera center to analyze video data in the navigation route and generate obstacle detection results, and calls a display output center to process the obstacle detection results and generate alarm information. Through an encrypted data stream channel, the system transmits the location data stream between the high-precision GPS positioning center and the high-performance AI computing center, the navigation route between the high-performance AI computing center and the front-facing camera center, and the obstacle detection results between the front-facing camera center and the display output center. Finally, it aggregates the location data stream from the high-precision GPS positioning center, the navigation route from the high-performance AI computing center, the obstacle detection results from the front-facing camera center, and the alarm information from the display output center to complete the execution of the positioning and tracking task, the path planning task, the obstacle detection task, and the alarm output task.

[0106] Continuously monitor changes in the network topology of personal devices and optimize the path, as follows:

[0107] The distributed soft bus continuously sends probe messages via network addresses to monitor changes in the topology of personal device networks, including whether smartphones, smart helmets, or vehicle dashboards are disconnected, as well as the task queue length and remaining battery percentage of high-precision GPS positioning centers, high-performance AI computing centers, front-facing camera centers, and display output centers. For example, if an increase in the task queue length of the front-facing camera center is detected, a change in resource status is confirmed. The affected front-facing camera centers and their task queue lengths are recorded, and smartphone front-facing camera centers with video capture functionality are selected from the global virtual function center resource directory as alternative virtual function centers. Based on the service quality requirements of the obstacle detection task, including low latency and high resolution, the network performance, load, and battery levels of the smartphone front-facing camera center are evaluated to generate a suitability score for the obstacle detection task (in the same way as the suitability score for the adaptation module). The suitability score of the smartphone front-facing camera center is compared with the suitability score of the front-facing camera center; if the smartphone front-facing camera center has a higher suitability score, it is replaced.

[0108] The collaborative task execution paths have been updated to assign the positioning and tracking task to the high-precision GPS positioning center, the path planning task to the high-performance AI computing center, the obstacle detection task to the smartphone front camera center, and the alarm output task to the display output center.

[0109] The distributed soft bus loads the updated collaborative task execution path in the encrypted data stream channel, generating a new task flow instance.

[0110] In summary, this invention extracts user behavior patterns and environmental state features through a lightweight intent reasoning model, maps them to task templates to generate structured collaborative task descriptors, and integrates video, voice, acceleration, and location data. Compared to existing single-modal reasoning, it comprehensively captures the rider's line of sight and road conditions, improving the accuracy of task generation. Furthermore, by parsing the collaborative task descriptor into atomic subtasks and matching them with a virtual function center, it evaluates adaptability based on optimization algorithms using real-time network status, load, and battery power, generating the optimal collaborative task execution path. This overcomes the limitations of single-index optimization, comprehensively balances resource allocation across multiple dimensions, and improves the execution efficiency and real-time performance of navigation and safety prompt tasks in cycling scenarios.

[0111] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A multi-device intelligent collaborative interaction system based on a distributed soft bus, characterized in that: include, The registration module establishes a network of personal devices and abstracts the device hardware capabilities into a virtual function center before registration, resulting in a dynamically updated global virtual function center resource catalog. The fusion module, based on the global virtual function center resource catalog, uses a lightweight intent reasoning model to fuse multimodal context data from sensors in the personal device network to generate a structured collaborative task descriptor. The adaptation module decomposes the collaborative task descriptor into atomic subtasks, matches virtual function centers from the global virtual function center resource directory, and dynamically evaluates the adaptability based on network status and device load to generate the optimal collaborative task execution path. The execution module uses a distributed soft bus to establish a secure data flow channel for the optimal collaborative task execution path, then instantiates and executes the task flow, and continuously monitors it to optimize the path.

2. The multi-device intelligent collaborative interaction system based on a distributed soft bus as described in claim 1, characterized in that: The process of establishing a personal device network, abstracting device hardware capabilities into a virtual function center, and registering them to obtain a dynamically updated global virtual function center resource catalog is as follows: The distributed soft bus is used to discover heterogeneous IoT devices within the communication range via multicast domain name service protocol, and the basic identification information of the heterogeneous IoT devices is recorded. Based on the basic identification information recorded, public key infrastructure is used to authenticate devices to each other and generate trusted authentication tokens; A personal device network is established based on a trusted authentication token. The hardware capabilities of the devices in the personal device network are abstracted to generate a virtual function center. The virtual function center is then registered to a distributed soft bus and aggregated to form a global virtual function center resource catalog.

3. The multi-device intelligent collaborative interaction system based on a distributed soft bus as described in claim 1, characterized in that: The method, based on the global virtual function center resource catalog, utilizes a lightweight intent reasoning model to fuse multimodal context data from sensors in personal device networks to generate a structured collaborative task descriptor, specifically: Obtain the function type and network address of the sensors within the personal device network from the global virtual function center resource directory; Based on the sensor function type, the sensors are scheduled to collect multimodal context data and input into a lightweight intent reasoning model to extract user behavior patterns and environmental state features. User behavior patterns and environmental state characteristics are mapped to predefined task templates to generate structured collaborative task descriptors.

4. The multi-device intelligent collaborative interaction system based on a distributed soft bus as described in claim 1, characterized in that: The process of decomposing the collaborative task descriptor into atomic subtasks, matching virtual function centers from the global virtual function center resource directory, and dynamically evaluating adaptability based on network status and device load to generate the optimal collaborative task execution path specifically involves: The task objective of the collaborative task descriptor is parsed and decomposed into atomic subtasks in a time sequence; Query the global virtual function center resource catalog, match the virtual function centers corresponding to the functional requirements of the atomic subtasks, and collect the real-time network status of personal device networks and the current load information of the virtual function centers. Based on real-time network status and load information, an optimization algorithm is used to evaluate the matching and adaptability of virtual functional centers and generate the optimal collaborative task execution path.

5. The multi-device intelligent collaborative interaction system based on a distributed soft bus as described in claim 1, characterized in that: After establishing a secure data flow channel for the optimal collaborative task execution path using a distributed soft bus, the task flow is instantiated and executed, specifically as follows: Based on the optimal collaborative task execution path, determine the virtual functional centers and network addresses participating in the task; By using a distributed soft bus and a transport layer security protocol, an encrypted data stream channel is established between virtual function centers, and the optimal collaborative task execution path is instantiated in the encrypted data stream channel to generate task stream instances. The scheduling virtual function center executes atomic subtasks according to task flow instances.

6. The multi-device intelligent collaborative interaction system based on a distributed soft bus as described in claim 5, characterized in that: The scheduling virtual function center executes atomic subtasks according to task flow instances, and transmits and processes data streams, specifically as follows: Analyze the task flow instance and extract the execution logic and data flow order of each atomic subtask; Based on the execution logic and data flow sequence, the virtual function center is invoked to process the data flow required by the atomic subtask. The processed data flow is then transmitted and aggregated between the virtual function centers via a distributed soft bus to complete the execution of the atomic subtask.

7. The multi-device intelligent collaborative interaction system based on a distributed soft bus as described in claim 1, characterized in that: The continuous monitoring to optimize the path specifically includes: The topology changes of personal device networks and the resource status of virtual function centers are monitored in real time through a distributed soft bus. If a change in resource status is detected, the adaptability of the virtual function centers is reassessed based on the change in resource status, and the collaborative task execution path is updated. The updated execution path is instantiated into a new task flow instance through a distributed soft bus.

8. The multi-device intelligent collaborative interaction system based on a distributed soft bus as described in claim 3, characterized in that: The extraction of user behavior patterns and environmental state features specifically includes: After preprocessing the multimodal contextual data, it is input into a lightweight intent reasoning model to extract time series features and spatial behavior features, and then integrate them into a user behavior pattern. By combining user behavior patterns and raw environmental information from multimodal context data, environmental state features are generated.

9. The multi-device intelligent collaborative interaction system based on a distributed soft bus as described in claim 4, characterized in that: The process of evaluating the adaptability of virtual function centers based on real-time network status and load information using an optimization algorithm is as follows: Quantify real-time network status, calculate weighted performance metrics for latency and bandwidth, and assess the current load and power status of the virtual function center; Based on weighted performance metrics, current load, and power status, an adaptability scoring matrix for virtual function centers is constructed, and the combination with the highest score is selected as the optimal combination of virtual function centers.

10. The multi-device intelligent collaborative interaction system based on a distributed soft bus as described in claim 7, characterized in that: The process of reassessing the adaptability of the virtual function center and updating the collaborative task execution path based on changes in resource status specifically involves: Identify changes in resource status, record the affected virtual function centers and performance parameters, and filter for alternative virtual function centers from the global virtual function center resource directory; Based on the service quality requirements of the atomic subtasks, the adaptability of alternative virtual function centers is reassessed, new collaborative task execution paths are generated, and the affected virtual function centers are replaced.

Images