Data acquisition method, master device, slave device, server, system and medium

Abstract

This application relates to the field of data acquisition technology, specifically to a data acquisition method, master device, slave device, server, system, and medium, aiming to solve the technical problems of control link being affected by traffic and mutual interference with the data link during the data acquisition process. To this end, the backpack master device and acquisition slave device of this application are connected via a wired network, while the backpack master device and server, and the acquisition slave device and server are connected via a wireless network. This enables low-latency data acquisition control between the backpack master device and acquisition slave device via a wired network connection, and also enables the uploading of data acquisition results via a wireless network. It solves the problems of delay and large jitter in the process of sending acquisition control commands, avoids mutual interference between the control process and the data upload process, and meets the comprehensive requirements of mobile, multi-device, high synchronization accuracy, and high upload bandwidth in the data acquisition process.

Description

Technical Field

[0001] This application relates to the field of data acquisition technology, specifically to a data acquisition method, a master device, a slave device, a server, a system, and a medium. Background Technology

[0002] Existing multi-device, multi-modal data acquisition solutions typically fall into the following categories: Multiple mobile phones, head-mounted displays, development boards, or cameras receive control commands and upload data through the same Wi-Fi network; the backpack master device sends commands such as start acquisition, stop acquisition, reacquisition, and time synchronization to multiple devices via Wi-Fi; all acquisition slave devices, including the backpack master device, iPhone, Pico, and devices running the Android system, are connected to the same Wi-Fi network, and acquisition commands, status feedback, video upload, audio upload, IMU upload, and business interface calls are all completed through this Wi-Fi network.

[0003] However, the existing solutions often have the following problems: Defect 1: Pure Wi-Fi control links are affected by upload traffic. • Reason: In existing solutions, control commands, heartbeats, status feedback, video uploads, audio uploads, IMU uploads, and cloud API calls typically share the same Wi-Fi link.

[0004] • Defect: When multiple devices upload high-definition video, audio, depth data or logs at the same time, the Wi-Fi bandwidth is occupied, and control commands such as start acquisition, stop acquisition, time synchronization, and re-acquisition will be delayed and jittered, resulting in unstable joint acquisition.

[0005] Defect 2: Interference between control link and data link • Reason: The existing solution does not distinguish between the control plane and the data plane, and low-latency small packet control messages and large-capacity acquisition file uploads share the same wireless link.

[0006] •Deficiencies: External network bandwidth, router load, cloud response, wireless interference, OSS upload queue and other factors may affect the real-time performance of data acquisition and control.

[0007] Accordingly, a new data acquisition scheme is needed in this field to address the above problems. Summary of the Invention

[0008] In order to overcome the above-mentioned defects, this application is made to solve or at least partially solve the technical problems of control link being affected by traffic and mutual interference with data link during data acquisition.

[0009] In a first aspect, a data acquisition method is provided, the method being applied to a backpack master device, wherein the backpack master device is connected to a server via a wireless network, and the backpack master device is connected to a data acquisition slave device via a wired network; the method includes: In response to receiving a data acquisition request from a mobile control client, a data acquisition control command is sent to the data acquisition slave device via the wired network to control the data acquisition of the slave device; and, Based on the aforementioned data collection requirements, data is collected, and the collected data results are uploaded to the server via the wireless network.

[0010] In one technical solution of the above data acquisition method, before sending the acquisition control command, the method further includes: Establish a wired network-based communication connection with the data acquisition slave device; Start the Dynamic Host Configuration Protocol (DHCP) service, and assign the wired network address to the acquisition slave device based on the DHCP service.

[0011] In one technical solution of the above data acquisition method, the method further includes: Based on the Dynamic Host Configuration Protocol (DHCP) service, the data acquisition slave device is provided with the network address, subnet mask, lease time, and DHCP server identifier of the wired network.

[0012] In one technical solution of the above data acquisition method, the method further includes: The default gateway, domain name system, and external network route of the wired network are not sent to the data collection device.

[0013] In one technical solution of the above data acquisition method, sending acquisition control commands to the acquisition slave device via the wired network includes: Based on the wired network, the acquisition slave device is issued an acquisition preparation command and an acquisition start command, so that the acquisition slave device responds to the acquisition start command, performs data acquisition, and records the local monotonic clock timestamp corresponding to the actual acquisition start point of the acquisition start command.

[0014] In one technical solution of the above data acquisition method, before sending the acquisition control command to the acquisition slave device through the wired network, the method further includes: Based on the wired network, a clock synchronization command is sent to the data acquisition slave device; The clock synchronization response sent by the acquisition slave device in response to the clock synchronization command is obtained, and the round-trip delay between the backpack master device and the acquisition slave device is obtained based on the sending time of the clock synchronization command of the master device, the receiving time of the clock synchronization command of the acquisition slave device, the sending time of the clock synchronization response of the acquisition slave device, and the receiving time of the clock synchronization response of the master device.

[0015] In one technical solution of the above data acquisition method, before sending the acquisition control command to the acquisition slave device through the wired network, the method further includes: The quality of the data acquisition link is assessed based on the round-trip time.

[0016] In one technical solution of the above data acquisition method, the method further includes: The master device time anchor point of the backpack master device is uploaded to the server so that the server can achieve time alignment of the data collection results based on the master device time anchor point; The master device time anchor point includes the master device system time and the master device monotonic clock.

[0017] In a second aspect, a data acquisition method is provided, the method being applied to a data acquisition slave device, wherein the data acquisition slave device is connected to a server via a wireless network, and the data acquisition slave device is connected to a backpack master device via a wired network; the method includes: Receive acquisition control commands sent by the backpack master device through the wired network; The acquisition control command controls the acquisition slave device to acquire data, and the acquired data acquisition results are uploaded to the server via the wireless network.

[0018] In one technical solution of the above data acquisition method, before receiving the acquisition control command, the method further includes: The network address of the wired network assigned to the data acquisition slave device by the backpack master device through the Dynamic Host Configuration Protocol service.

[0019] In one technical solution of the above data acquisition method, the method further includes: The system receives the network address, subnet mask, lease time, and DHCP server identifier sent by the backpack master device to the data acquisition slave device through the DHCP service.

[0020] In one technical solution of the above data acquisition method, the method further includes: Based on the network connection management interface of the data acquisition slave device, the control communication connection is bound to the wired network, and the data upload connection is bound to the wireless network.

[0021] In one technical solution of the above data acquisition method, the step of controlling the acquisition slave device to perform data acquisition according to the acquisition control command includes: Receive the data collection preparation command and the data collection start command sent by the backpack master device; In response to the start acquisition command, data acquisition is performed, and the local monotonic clock timestamp corresponding to the actual acquisition start point of the start acquisition command is recorded.

[0022] In one technical solution of the above data acquisition method, before receiving the acquisition control command, the method further includes: In response to the clock synchronization command issued by the backpack master device, a clock synchronization response is sent to the backpack master device, so that the backpack master device can obtain the round-trip delay between the backpack master device and the acquisition slave device based on the sending time of the clock synchronization command of the master device, the receiving time of the clock synchronization command of the acquisition slave device, the sending time of the clock synchronization response of the acquisition slave device, and the receiving time of the clock synchronization response of the master device.

[0023] In one technical solution of the above data acquisition method, the method further includes: The slave device time anchor point of the data acquisition device is uploaded to the server so that the server can achieve time alignment of the data acquisition results based on the slave device time anchor point; The slave device time anchor point includes the slave device system time and the slave device monotonic clock.

[0024] In a third aspect, a data acquisition method is provided, the method being applied to a server, the server being connected to a backpack master device and a data acquisition slave device via a wireless network; the method includes: The system receives data collection results uploaded to the server by the backpack master device and the data collection slave device based on the wireless network.

[0025] In one technical solution of the above data acquisition method, the method further includes: Receive the master time anchor point of the backpack master device and the slave time anchor point of the acquisition slave device; Based on the time anchor points of the master device and the slave device, the data acquisition results of the backpack master device and the acquisition slave device are time-aligned.

[0026] In a fourth aspect, a backpack master device is provided, the backpack master device including at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program, which, when executed by the at least one processor, implements the method described in any of the above-described data acquisition methods.

[0027] In a fifth aspect, a data acquisition slave device is provided, the data acquisition slave device including at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program, which, when executed by the at least one processor, implements the method described in any of the above-described data acquisition methods.

[0028] In a sixth aspect, a server is provided, the server including at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program that, when executed by the at least one processor, implements the method described in any of the above-described data acquisition methods.

[0029] In a seventh aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, the program codes being adapted to be loaded and run by a processor to perform the method described in any of the above-described technical solutions of the data acquisition method.

[0030] In an eighth aspect, a data acquisition system is provided, the system comprising the backpack master device described in the above-mentioned backpack master device technical solution, the acquisition slave device described in the above-mentioned acquisition slave device technical solution, and the server described in the above-mentioned server technical solution; the backpack master device and the acquisition slave device are respectively connected to the server via wireless network communication, and the backpack master device and the acquisition slave device are connected via wired network communication.

[0031] The above-described technical solutions of this application have at least one or more of the following beneficial effects: In implementing the data acquisition method provided in this application, the backpack master device and the acquisition slave device are connected via a wired network, while the backpack master device and the server, and the acquisition slave device and the server are connected via a wireless network. This enables the backpack master device and the acquisition slave device to connect via a wired network, thereby achieving a low-latency data acquisition control process. It also enables the backpack master device and the acquisition slave device to communicate with the server via a wireless network, thereby achieving the uploading of data acquisition results. This effectively solves the problems of delay and large jitter in the process of sending acquisition control commands, avoids the problem of mutual interference between the control process and the data acquisition uploading process, and meets the comprehensive requirements of mobile, multi-device, high synchronization accuracy and high upload bandwidth in the data acquisition process.

[0032] Furthermore, this application assigns a wired network address to the data acquisition slave device based on the Dynamic Host Configuration Protocol (DHCP) service, and distributes the wired network address, subnet mask, lease time, and DHCP server identifier, but does not distribute the default gateway, domain name system (DNS), and external network route of the wired network. This effectively solves the problem of conflict between the default gateway, DNS, and external network route of the data acquisition slave device in the dual-network connection state.

[0033] Furthermore, based on the network connection management interface of the data acquisition slave device, this application binds the control communication connection to the wired network and the data upload connection to the wireless network, which can effectively solve the problem that some data acquisition slave devices have a default network selection, resulting in the inability to reliably separate the data transmission protocol and the transmission control protocol.

[0034] Furthermore, this application can synchronize the first frame of multimodal data among multiple acquisition slave devices during the data acquisition process based on the round-trip delay between the backpack master device and the acquisition slave device, the start acquisition command and the local monotonic clock timestamp corresponding to the actual acquisition starting point, effectively improving the accuracy and effectiveness of the data acquisition results.

[0035] Furthermore, the server in this application can perform time alignment of multimodal data acquisition results based on the time anchor points of the master device and the slave device. It can achieve effective alignment of data acquisition results from multiple devices without relying on forced system time synchronization or server-side upload time as the acquisition time, thereby further improving the accuracy of data acquisition results. Attached Figure Description

[0036] The disclosure of this application will become more readily understood with reference to the accompanying drawings. It will be readily understood by those skilled in the art that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this application. Wherein: Figure 1 This is a schematic flowchart of the main steps of a data acquisition method according to an embodiment of this application; Figure 2 This is a schematic diagram of the main components of a data acquisition system according to an embodiment of this application. Detailed Implementation

[0037] Some embodiments of this application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of this application and are not intended to limit the scope of protection of this application.

[0038] In the description of this application, "module" and "processor" can include hardware, software, or a combination of both. A module can include hardware circuitry, various suitable sensors, communication ports, and memory, and may also include software components, such as program code, or a combination of software and hardware. The term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B. The terms "at least one A or B" or "at least one of A and B" have a similar meaning to "A and / or B" and can include only A, only B, or A and B. The singular forms of the terms "a" and "this" can also include plural forms.

[0039] Here we will first explain some of the terms used in this application.

[0040] Dynamic Host Configuration Protocol (DHCP) is a network management protocol used to automatically assign IP addresses and other network parameters to devices in a network. In the embodiments of this application, the DHCP service on the backpack master device assigns local area network IP addresses to the collection slave devices connected via wired network, and selectively provides network parameters to achieve network isolation.

[0041] Backpack Master Device: The main control device deployed in a data acquisition backpack, human carrying device, mobile robot, or edge computing device, such as a Jetson, RK development board, Linux development board, or other embedded computing device. It can serve as both a data acquisition node and a control node.

[0042] Data acquisition slave device: A device that participates in the same data acquisition task as the main backpack device. This can be an iPhone, Pico, an Android device, a VR / AR headset, a camera, a depth camera, or other mobile data acquisition terminal. The underlying system of the Pico device can be based on Android or an Android-compatible system, and therefore can adopt a network path selection mechanism similar to that of Android devices.

[0043] Mobile Control Client: Control software used to send commands such as login, task start, task stop, live stream preview, device query, and upload control to the main backpack device. This software can run on mobile phones, tablets, headsets, or other mobile devices. A single device can simultaneously function as both a mobile control client and a joint acquisition slave device.

[0044] Dnsmasq: A lightweight DHCP / DNS (Domain Name System) service program commonly used in Linux systems. In the embodiments of this application, dnsmasq can be used to implement DHCP service, but this application is not limited to dnsmasq.

[0045] DHCP option:router: DHCP default gateway option, corresponding to the common DHCP option 3. In this application, the backpack master device does not issue a valid router option to the joint data collection device to avoid the wired network becoming the device's default external network exit.

[0046] DHCP option: dns-server: DHCP DNS server option, corresponding to the common DHCP option 6. In this application, the backpack master device does not issue a valid dns-server option to the joint collection device to avoid the DNS resolution path being switched to the wired control network.

[0047] WebSocket: A full-duplex communication protocol based on TCP (Transmission Control Protocol). In this application, it is used for wired low-latency command communication, heartbeat detection, status feedback, and clock synchronization between the backpack master device and the joint acquisition device.

[0048] Request-Response: A request-response communication pattern. The sender sends a request containing a request_id, and the receiver returns a response containing the same request_id. This is used for non-blocking command calls.

[0049] Notification: An asynchronous notification message used for reporting events such as upload progress, image update progress, live streaming errors, data collection errors, and status changes.

[0050] Wi-Fi: Wireless Local Area Network. In this application, it is used by various data acquisition devices to access the cloud server, call business interfaces, and upload collected data.

[0051] Dual-network isolation refers to the separation of the command and control link from the data acquisition and upload link. Specifically, the wired network is used for low-latency, high-reliability control commands, heartbeats, status feedback, and clock synchronization; while the wireless network (such as Wi-Fi) is used for devices to access cloud service interfaces and upload large-capacity multimodal data such as video and audio.

[0052] A monotonic clock is a locally incrementing clock that does not change regardless of user modifications to the system time, Network Time Protocol (NTP) synchronization, or time zone changes. In the embodiments of this application, a monotonic clock is preferably used for accurate timestamp recording of acquired data and clock synchronization between multiple devices to ensure the continuity and stability of time measurements.

[0053] Round-Trip Time (RTT) refers to the total time it takes for a data packet to travel from the sender to the receiver and back. In the embodiments of this application, it is accurately calculated using a four-timestamp model to evaluate the quality of the wired control link and serves as a key parameter for sample selection and weight calculation in the clock synchronization algorithm.

[0054] A time anchor point is a timestamp pair containing both the system time (Wall Clock) and the monotonic clock, used to establish conversion relationships between different time bases. In the embodiments of this application, the backpack master device and the data acquisition slave device each upload their respective time anchor points to the server so that the server can perform global time alignment and map the data collected by all devices onto a unified time axis.

[0055] Wall Clock: System time, i.e., the date and time provided by the operating system, is typically used for logs, file naming, task display, and UTC time stamping. This application does not rely on forcibly modifying the system time of each device to achieve precise synchronization.

[0056] First frame synchronization: In the same joint acquisition task, multiple devices align data such as the first frame of video, the start of audio sampling, the start of IMU sampling, and the start of pose recording on a unified time axis.

[0057] snake_case: Snake-like naming convention, where field names, command names, and notification names use lowercase letters with underscores, such as request_id, device_id, and start_capture.

[0058] WebRTC: A low-latency real-time audio and video communication technology. In this application, it can be used as an optional preview live streaming module, but is not a necessary component of the formal data acquisition and upload link.

[0059] IMU (Inertial Measurement Unit): An inertial measurement unit, including accelerometers, gyroscopes, magnetometers, etc.

[0060] Multimodal data: different types of data such as video, audio, IMU, pose, depth, point cloud, device status, task labels, camera parameters, synchronization index, etc.

[0061] See appendix Figure 1 , Figure 1 This is a schematic flowchart illustrating the main steps of a data acquisition method according to an embodiment of this application. Figure 1 As shown, in this embodiment of the application, the backpack master device and the server are connected via wireless network communication, the backpack master device and the data acquisition slave device are connected via wired network communication, and the data acquisition slave device and the server are connected via wireless network communication.

[0062] The data acquisition method of this application embodiment mainly includes the following steps S101 to S105.

[0063] Step S101: In response to receiving the data acquisition request sent by the mobile control client, the backpack master device sends a data acquisition control command to the data acquisition slave device via a wired network to realize the data acquisition control of the data acquisition slave device.

[0064] In this embodiment, the backpack master device can send acquisition control commands to the acquisition slave device that has established a wired network connection.

[0065] In one embodiment, the backpack master device can be a wearable or portable embedded computing device, which may include Jetson, RK development board, Linux development board, edge computing box or other devices with wired network interface, wireless network interface and computing capabilities.

[0066] In one implementation, a wired local area control network can be established between the backpack master device and multiple data acquisition slave devices. This wired local area control network enables a low-latency, low-jitter, and highly deterministic local area control network between the backpack master device and the data acquisition slave devices.

[0067] In one implementation, the wired network may include, but is not limited to: Ethernet cable; USB-C to Ethernet; Lightning to Ethernet; USB hub with Ethernet adapter; industrial network port; wired switch; RNDIS (Remote Network Driver Interface Specification); CDC-ECM (Communication Device Class - Ethernet Control Model); USB virtual network card; and other wired connection methods that can form a local area IP network.

[0068] In one implementation, the wired network can handle the following communication tasks between the backpack master device and the acquisition slave device: WebSocket control commands; device heartbeat; device status; clock synchronization; task scheduling; and exception notifications. The wired network generally does not handle the uploading of large files such as video, audio, and depth data generated during actual acquisition.

[0069] In one implementation, before the backpack master device issues the acquisition control command, the backpack master device can establish a wired network-based communication connection with the acquisition slave device; start the Dynamic Host Configuration Protocol (DHCP) service, and assign a wired network address to the acquisition slave device based on the DHCP service.

[0070] In this embodiment, data acquisition slave devices, such as iPhones, Pico devices, devices running Android systems, headsets, or other data acquisition slave devices, can establish a wired network communication connection with the backpack master device via wired network connections such as Ethernet cables, USB-to-Ethernet adapters, USB hubs, or wired switches. Network addresses are then assigned to multiple data acquisition slave devices based on DHCP service.

[0071] In a specific example, the backpack master device can be configured with a fixed IP address on a wired network, such as 10.10.14.1, and listen on a WebSocket control port, such as 8765. This fixed IP address is used by the data acquisition slave devices to discover the backpack master device, establish wired control connections, and receive data acquisition control commands.

[0072] In one implementation, the backpack master device can send information such as the network address, subnet mask, lease time, and DHCP server identifier of the wired network to the slave device based on the Dynamic Host Configuration Protocol (DHCP) service.

[0073] In a specific example, the DHCP address allocation module is deployed in the backpack master device to automatically assign local area network (LAN) IP addresses to multiple data collection slave devices connected to the backpack master device via a wired network. A preferred configuration is as follows: The main device for the backpack has a wired fixed IP address: 10.10.14.1 Subnet mask: 255.255.255.0 DHCP address pool: 10.10.14.20 - 10.10.14.200 WebSocket control port: 8765.

[0074] The DHCP address allocation module can assign fixed or semi-fixed IP addresses to the data acquisition slave devices based on the following information: 1. MAC address; 2. Device serial number; 3. Application registration device ID; 4. USB device identifier; 5. Equipment type; 6. Historical binding records.

[0075] For example, the IP address assigned to the data acquisition device could be: iphone_001 -> 10.10.14.21 pico_001 -> 10.10.14.22 android_001 -> 10.10.14.23 In one implementation, the backpack master device does not send the default gateway, domain name system, and external network route of the wired network to the data collection slave device.

[0076] In this embodiment, to prevent the data collection slave device from routing internet access such as data transmission to the backpack master device, the DHCP service differs from ordinary internet access scenarios. It does not send information such as the default gateway, domain name system, and external network routing for the wired network to the data collection slave device, thereby effectively preventing the data collection slave device from routing internet access such as data transmission to the backpack master device. The core feature of the DHCP service is: 1. Send IP addresses to the data acquisition slave devices; 2. Send the subnet mask to the data acquisition slave device; 3. The lease term can be issued; 4. Can issue DHCP server identifiers; 5. Do not issue a valid default gateway; 6. Do not issue valid DNS servers; 7. Do not issue static routes pointing to the external network; 8. Optionally, routes may be issued only to wired control subnets.

[0077] In one implementation, the DHCP service can be provided by dnsmasq, isc-dhcp-server, udhcpd, or other DHCP server programs. Preferably, dnsmasq can be used as a lightweight DHCP server program.

[0078] In a specific example, dnsmasq configuration can be implemented based on the following code: JSON interface=eth0 bind-interfaces except-interface=wlan0 dhcp-authoritative port=0 dhcp-range=10.10.14.20,10.10.14.200,255.255.255.0,12h Explicitly provide the subnet mask dhcp-option=option:netmask,255.255.255.0 Explicitly not issuing a valid default gateway dhcp-option=option:router Explicitly not issuing valid DNS servers dhcp-option=option:dns-server in: dhcp-option=option:router This is used to prevent the acquisition slave device from being issued a valid default gateway, so that the acquisition slave device will not generate a default route pointing to the wired IP of the backpack master device.

[0079] dhcp-option=option:dns-server This is used to prevent the acquisition slave device from being assigned a valid DNS server, thus preventing the slave device from switching its DNS resolution path to the wired network.

[0080] Here, port=0 indicates that dnsmasq does not provide DNS resolution services and is only used as a DHCP service. Different dnsmasq versions or system distributions may handle empty options slightly differently. In practice, the acceptance criteria are that the slave device's DHCP ACK does not contain a valid router / dns-server, and the slave device does not generate a wired default route or use wired DNS.

[0081] Verification can be performed in the following ways: 1. Packet capture confirmed that there was no valid router option in the DHCP ACK; 2. Packet capture confirmed that there was no valid dns-server option in the DHCP ACK; 3. Check the routing table of the slave device to confirm that there is no default via 10.10.14.1; 4. Check the DNS resolution path to confirm that Wi-Fi DNS is still being used; 5. Use a wired network when accessing 10.10.14.1; 6. Accessing cloud (server) domains uses Wi-Fi (i.e., wireless network).

[0082] It should be noted that dnsmasq is only one Linux implementation of this application. This application is not limited to dnsmasq; any DHCP service that can "assign wired LAN IP addresses to data collection slave devices without issuing valid default gateways and valid DNS" can achieve the purpose of this application.

[0083] In one implementation, in some systems of data collection slave devices, even if IPv4 DHCP does not assign a default gateway, the wired interface may still generate a default route due to IPv6 Router Advertisement. Therefore, in a preferred embodiment, the backpack master device may also disable IPv6 route advertisements on the wired interface, or not publish IPv6 default routes and RDNSS DNS information, thereby preventing the data collection slave device from generating a default route for the wired network.

[0084] Specifically, the backpack master device: does not publish an IPv4 default gateway; does not publish an IPv4 DNS; does not publish an IPv6 default route; and does not publish an IPv6 RDNSS DNS, thereby further preventing the external network path of the collection slave device from being taken over by the wired network.

[0085] Step S102: The acquisition slave device receives the acquisition control command sent by the backpack master device through a wired network.

[0086] In this embodiment, the data acquisition slave device can receive data acquisition control commands sent by the backpack master device via a wired network.

[0087] In one implementation, there can be multiple data acquisition slave devices, such as iPhones, Pico devices, devices running Android systems, head-mounted displays, or other combined data acquisition devices.

[0088] Step S103: The backpack master device collects data based on the collection requirements and uploads the obtained data collection results to the server via wireless network.

[0089] In this embodiment, the backpack main device can serve as one of the data acquisition devices and can upload the data acquisition results to the server via a wireless network.

[0090] Step S104: The data acquisition slave device controls the data acquisition slave device to perform data acquisition according to the data acquisition control command, and uploads the obtained data acquisition results to the server through the wireless network.

[0091] In this embodiment, the data acquisition slave device responds to the data acquisition control command, which controls the data acquisition slave device to perform data acquisition and upload the data acquisition results to the server. That is, the data acquisition results do not need to be transmitted back to the backpack master device through a wired network. Each device (including the backpack master device and the data acquisition slave device) independently uploads its own data acquisition results through its own wireless network (Wi-Fi) with the server.

[0092] In one implementation, the upload protocol of the wireless network may include, but is not limited to: HTTPS; MQTT; OSS / S3 fragmented upload; custom TCP upload; resume interrupted upload; background upload; retry in weak network conditions; upload speed limit; upload cancellation, etc.

[0093] In one implementation, the data acquisition results can be multimodal data, specifically including but not limited to: RGB video; wide-angle video; ultra-wide-angle video; depth video; audio; IMU data; pose data; point cloud data; camera intrinsic parameters; distortion parameters; device status logs; time synchronization records; task metadata; synchronization indexes or upload lists, etc.

[0094] One implementation method involves binding the control communication connection to a wired network and the data upload connection to a wireless network based on the network connection management interface of the data acquisition slave device.

[0095] In this implementation, relying solely on the DHCP service of the backpack master device without issuing routers and DNS servers can avoid routing conflicts in most scenarios. However, for data collection slave devices running Android systems and Android-compatible systems, mechanisms such as default network, network rating, network capability, and network verification still exist. Since Pico's underlying system is based on Android-compatible devices, it also needs to adopt a similar network path selection mechanism as devices running Android systems.

[0096] Therefore, this application embodiment further sets a network path selection module in the mobile software based on the Android system and devices running Android-compatible systems to ensure that different service traffic is sent according to the preset network path. Devices running the Android system or Pico devices based on the Android system can obtain two types of networks through the network connection management interface (ConnectivityManager or equivalent system interface): Wired network: TRANSPORT_ETHERNET Wireless network: TRANSPORT_WIFI The mobile software on the data acquisition device can perform the following steps: 1. Monitor network changes; 2. Obtain a network with Ethernet capability (i.e., a wired network). 3. Obtain a network with Wi-Fi and Internet capabilities (i.e., a wireless network); 4. To control the WebSocket connection, use the Ethernet Network's SocketFactory or an equivalent interface to create the connection; 5. For HTTPS, MQTT, and OSS upload connections, use Wi-Fi Network's SocketFactory or equivalent interface to create the connection; 6. WebSocket should preferably connect directly to the backpack's host device IP, such as 10.10.14.1, to avoid using DNS; 7. Cloud (i.e., server) business requests and upload requests use the DNS resolution capabilities of the Wi-Fi network; 8. Do not globally bind the entire process to Ethernet to avoid cloud requests being incorrectly sent to the wired network; 9. If Ethernet is unavailable, prevent the high-precision joint data acquisition process from starting or degrade to Wi-Fi control and display a prompt; 10. If Wi-Fi is unavailable, wired network control can continue, but data collection results will be cached locally and await uploading.

[0097] The application layer binding example logic is shown in the following code: JSON Network ethernet_network = find_network(TRANSPORT_ETHERNET) Network wifi_network = find_network(TRANSPORT_WIFI + INTERNET) websocket_client.socket_factory = ethernet_network.getSocketFactory() websocket_client.target = ws: / / 10.10.14.1:8765 cloud_http_client.socket_factory = wifi_network.getSocketFactory() cloud_http_client.dns = wifi_network.getAllByName mqtt_client.socket_factory = wifi_network.getSocketFactory() oss_upload_client.socket_factory = wifi_network.getSocketFactory() For WebSocket: WebSocket control connection -> Ethernet Network For cloud services and uploads: HTTPS / MQTT / OSS upload connection -> Wi-Fi Network When using OkHttp, WebSocket SDK, MQTT SDK, OSS SDK or other network libraries, if the SDK supports SocketFactory, DNS, Network or equivalent interfaces, then create client instances with different network bindings for controlling the connection and uploading the connection, respectively.

[0098] If some SDKs do not support connection-level network binding, one of the following methods can be used: 1. Replace with a network library that supports SocketFactory; 2. Configure policy routing for the target IP / domain at the system network layer; 3. Create a socket using the platform's native network API; 4. Execute a single request via a controlled, short-lived bindProcessToNetwork and resume immediately after the request is completed; 5. For SDKs that do not support network binding, use them only for non-critical paths. Critical WebSocket control links must be ensured to be wired.

[0099] It is not recommended to execute the following commands for an extended period of time: bindProcessToNetwork(ethernet_network) The reason is that this causes all traffic, including cloud (server) login, HTTPS, MQTT, and OSS uploads, to incorrectly go through the wired network. Since the wired network does not have a default gateway and DNS, it may cause the external network to be unreachable.

[0100] In some preferred embodiments, networks can be bound by connection or by service type: WebSocket connection control: Binding to Ethernet Cloud (server) requests and uploads: Wi-Fi binding This module ensures that dual-network isolation extends beyond the DHCP routing configuration layer to the application connection layer, guaranteeing: WebSocket command communication is sent over a wired network; The sync_clock clock synchronization is sent via a wired network; Ping / pong heartbeats are sent over a wired network; Cloud service requests are sent via Wi-Fi; Video / audio / IMU / pose / depth upload via Wi-Fi; DNS resolution is done via Wi-Fi.

[0101] In one implementation, for data acquisition slave devices on iOS and other systems, network path selection can be performed using one of the following methods: 1. Relying on DHCP configuration service without a valid router or DNS server, traffic accessing the main device IP of the backpack goes through the wired network, while traffic accessing the cloud (server) goes through Wi-Fi (wireless network). 2. Use the network path selection interface provided by the system to bind the local area control connection to the wired network; 3. Use the fixed backpack master device IP to access the WebSocket service, so that the system selects the wired interface according to the routing table; 4. Use domain names and Wi-Fi DNS for cloud access to maintain a wireless link (wireless network) for cloud services. 5. At the application layer, detect the current interface and actual exit point. If an abnormal path is found, refuse to collect data or prompt the user.

[0102] In one implementation, the WebSocket communication protocol uses the snake_case naming convention. Examples include: request_id; device_id; job_id; start_capture; stop_capture; sync_clock; upload_captured_files_progress_changed; preview_broadcast_failed, etc.

[0103] WebSocket messages can include at least the following types: 1. request: Request message; 2. response: Response message; 3. Notification: A notification message; 4. ping: heartbeat request; 5. pong: heartbeat response.

[0104] The request format can be set based on the following code: JSON { "type": "request", "request_id": "550e8400-e29b-41d4-a716-446655440000", "device_id": "device_001", "command": "start_capture", "params": {} } The response format can be set based on the following format: JSON { "type": "response", "request_id": "550e8400-e29b-41d4-a716-446655440000", "result": { "code": 200, Message: "Success" }, "data": {} } Preferred error codes include: 200: Success; 400: Parameter error; 401: Unauthorized; 404: Resource does not exist; 408: Request timed out; 409: State conflict; 500: Internal Error; 503: The device is busy or the service is unavailable; The notification format can be set based on the following code: JSON { "type": "notification", "notification_id": "550e8400-e29b-41d4-a716-446655440000", "name": "upload_captured_files_progress_changed", "data": { "job_id": "550e8400-e29b-41d4-a716-446655440000", "percent": 50, "error_message": "" } } Notifications can be used for proactive reporting of long-running tasks, abnormal events, and status changes, so that tasks such as image updates, file uploads, and live stream previews do not need to block the request-response waiting process.

[0105] Ping / Pong heartbeats can be set based on the following code: JSON { "type": "ping", "device_id": "device_001 } JSON { "type": "pong", "device_id": "device_001 } Heartbeats can be handled by a separate thread and do not carry status information such as CPU, memory, or disk. Device status is actively retrieved by the client via get_device_info to avoid excessively heavy heartbeat packets.

[0106] Non-blocking communication mechanism: The WebSocket I / O thread is only responsible for sending and receiving messages and does not perform time-consuming business logic.

[0107] Upon receiving a request, the system can perform the following steps: 1. Validate request_id, device_id, and command; 2. Submit the task to the business thread pool; 3. Return a response directly for short tasks; 4. For long tasks, first return the accepted response; 5. Long-running tasks are notified of their progress asynchronously via notifications; 6. Responses can be returned in out-of-order, but they can be matched precisely by request_id; 7. Retry if critical instructions time out without response; 8. The receiver ensures idempotency through request_id, command, and job_id.

[0108] This mechanism ensures that time-consuming operations such as mirror updates, file uploads, and live stream previews will not block high-priority messages such as stop collection, heartbeats, and exception notifications.

[0109] Step S105: The server receives the data collection results uploaded to the server by the backpack master device and the collection slave device via the wireless network.

[0110] In this embodiment, the server can receive data collection results uploaded by the backpack master device and the collection slave device via a wireless network.

[0111] In one implementation, before sending the acquisition control command to the acquisition slave device, the backpack master device can send an acquisition preparation command and an acquisition start command to the acquisition slave device via a wired network, so that the acquisition slave device responds to the acquisition start command, starts data acquisition, and records the local monotonic clock timestamp corresponding to the actual acquisition start point.

[0112] In one implementation, the backpack master device can send a clock synchronization command to the data acquisition slave device via a wired network; after receiving the clock synchronization command, the data acquisition slave device will send a clock synchronization response to the backpack master device; after receiving the clock synchronization response sent by the clock synchronization command, the backpack master device can obtain the round-trip time between the backpack master device and the data acquisition slave device based on the sending time of the master device's clock synchronization command, the receiving time of the data acquisition slave device's clock synchronization command, the sending time of the data acquisition slave device's clock synchronization response, and the receiving time of the master device's clock synchronization response.

[0113] In one implementation, the quality of the data acquisition link can be assessed based on round-trip time.

[0114] In one implementation, the backpack master device and the data acquisition slave device can respectively upload the master device time anchor and the slave device time anchor to the server via a wireless network, so that the server can realize time alignment of the data acquisition results according to the master device time anchor and the slave device time anchor respectively; wherein, the master device time anchor and the slave device time anchor respectively include the device system time and monotonic clock of the corresponding device.

[0115] Specifically, data acquisition slave devices, including those running iOS and Android systems, typically lack permission to modify the system time; data acquisition slave devices like Pico running Android-compatible systems may also not have permission to modify the system time; furthermore, modifying the system time may lead to abnormal HTTPS certificates, login tokens, and log order; the system time may also be affected by NTP, user settings, or time zone changes. Furthermore, frame-level synchronization is more suitable for using a monotonic clock. Simultaneously, multimodal acquisition requires a stable, incrementally increasing sampling timestamp, rather than a system time that may fluctuate. Therefore, this application's embodiments employ a two-layer time system: System time: used for coarse-grained time stamping of logs, file names, cloud display, task time, etc.; Monotonic clock: used for local timestamp recording of acquired data such as video frames, audio samples, IMU, pose, depth, etc.

[0116] The backpack's main device can synchronize its system time with an NTP server or cloud server (i.e., a server) via Wi-Fi, ensuring it has a relatively accurate UTC time. The backpack's main device records a time anchor point: host_wall_time_ns, host_monotonic_time_ns Among them, host_wall_time_ns is the master device's system time, and host_monotonic_time_ns is the master device's monotonic clock.

[0117] The cloud can use this time anchor to convert the monotonic clock time of the backpack's main device to approximate UTC time: host_utc_time = host_wall_time_ns + (host_monotonic_time_ns_current -host_monotonic_time_ns_anchor) `host_utc_time` is the master device's UTC time, `host_monotonic_time_ns_current` is the master device's current monotonic clock, and `host_monotonic_time_ns_anchor` is the master device's anchor monotonic clock.

[0118] The device only needs to record its own system time and monotonic clock time, for example: device_wall_time_ns; device_monotonic_time_ns Among them, device_wall_time_ms is the slave device's system time, which can be used for coarse-grained marking such as logs, file naming, and task display; device_monotonic_time_ns is the slave device's monotonic clock, which can be used for precise timestamp recording of locally collected data.

[0119] During the acquisition process, the device uniformly records local monotonic clock timestamps for data such as video frames, audio samples, IMU, pose, and depth. For example: device_first_frame_time_ns (time of the first video frame captured from the slave device) device_first_audio_time_ns (time of the first audio frame captured from the slave device) device_first_imu_time_ns (time of the first IMU frame captured from the slave device) This application does not rely on forced synchronization of system time across devices, nor does it use server upload time as the collection time. When aligning data collection results from multiple devices, the server can prioritize using the local monotonic clock of each device to collect timestamps, and combine this with the RTT (round-trip time) measured by the wired network WebSocket sync_clock command, the link quality before collection begins, the actual first frame timestamp, and cloud post-processing algorithms for synchronization alignment.

[0120] The alignment mechanism for data acquisition results based on RTT and the actual first frame timestamp is explained below: This mechanism can be used to measure the round-trip time (RTT) of the link via a WebSocket control link over a wired network between the backpack master device and the acquisition slave device, in the absence of a dedicated hardware synchronization line or timecode equipment, and to evaluate the latency and jitter of the current synchronization link. The RTT result is not used to forcibly modify the device's system time, nor does it require the establishment of a complete clock drift model. Instead, it is used to determine the synchronization quality of the control link before acquisition begins and to assist in the starting point alignment of subsequent video, audio, IMU, pose, and depth data.

[0121] The master backpack device sends a sync_clock to the slave data acquisition device via a wired WebSocket, as follows: JSON { "type": "request", "request_id": "550e8400-e29b-41d4-a716-446655440000", "device_id": "iphone_001", "command": "sync_clock", "params": { "sync_group_id": "sync_001", "sync_index": 1, "host_send_time_ns": 123456789000000 } } The acquisition device immediately returns a clock synchronization response after receiving the sync_clock data. JSON { "type": "response", "request_id": "550e8400-e29b-41d4-a716-446655440000", "result": { "code": 200, Message: "Success" }, "data": { "device_receive_time_ns": 987654321000000, "device_send_time_ns": 987654321200000, "clock_type": "monotonic" } } When the backpack master device receives a response (clock synchronization response), it records the local reception time (i.e., the reception time of the clock synchronization response): host_receive_time_ns For the i-th sync_clock measurement, four timestamps can be defined, namely: T1_i: The time when the backpack master device sends the sync_clock request (i.e., the time when the clock synchronization command is sent); T2_i: The time when the acquisition slave device receives the sync_clock request (the time when the acquisition slave device receives the clock synchronization command); T3_i: The time when the acquisition slave device sends the sync_clock response (i.e., the time when the clock synchronization response is sent); T4_i: The time when the backpack master device receives the sync_clock response (i.e., the time when the clock synchronization response is received).

[0122] Where: T1_i and T4_i belong to the monotonic clock of the backpack master device; T2_i and T3_i belong to the monotonic clock of the data acquisition slave device.

[0123] The calculation method for RTT is as follows: RTT_i = (T4_i - T1_i) - (T3_i - T2_i) Where: T4_i - T1_i is the total time taken for a single sync_clock request-response as observed by the backpack master device; T3_i - T2_i is the local processing time taken from receiving the request to issuing the response.

[0124] After deducting the processing time of the data acquisition slave device, RTT_i is closer to the round-trip communication latency of the wired control link (i.e., the wired network) between the backpack master device and the data acquisition slave device.

[0125] If the data acquisition device does not return T2_i and T3_i, a simplified RTT calculation process can be used as follows: RTT_i = T4_i - T1_i This simplified RTT includes communication time and processing time from the acquisition slave device, and can be used as an approximate indicator of synchronization quality.

[0126] In one implementation, RTT can be obtained based on the following steps: 1. The backpack master device sends multiple sync_clocks before starting data acquisition; 2. Each sync_clock sends T1, T2, T3, and T4; 3. Calculate RTT_i for each sync_clock; 4. Remove obviously abnormal high RTT samples (e.g., those exceeding a preset threshold); 5. Calculate RTT_min (minimum RTT), RTT_median (median RTT), and RTT_jitter (RTT jitter) based on the retained samples; 6. Write the above calculation results as synchronization quality results into the synchronization metadata of this data acquisition task.

[0127] More preferably, the master device of the synchronized knapsack sends no less than 5 sync_clocks per round, such as 7 or 9 times, and uses the median and MAD (Median Absolute Deviation) for anomaly filtering, as shown in the following formula: median_RTT = median(RTT_i); MAD = median(|RTT_i - median_RTT|) The rules for removing outlier samples are shown in the following formula: |RTT_i - median_RTT|>k × MAD Where k can be 2.5 or 3.

[0128] Synchronization quality can be represented by RTT jitter, as shown in the following formula: RTT_jitter = sqrt(Σ(RTT_i - mean_RTT)^2 / n) The following metrics can also be recorded: RTT_min: Minimum RTT; RTT_median: Median RTT; RTT_jitter: RTT jitter; RTT_valid_count: Number of valid RTT samples.

[0129] In a specific example, for 30 fps video, one frame is approximately 33.33 ms; for 60 fps video, one frame is approximately 16.67 ms. Preferred synchronization quality requirements can be set as follows: 30 fps: RTT_jitter<8 ms; 60 fps: RTT_jitter<4 ms If the synchronization quality requirements are not met, the backpack master device can perform the following steps: 1. Resend sync_clock; 2. Increase the number of sync_clock measurements; 3. Prompt for wired network connection error; 4. Expand the subsequent video alignment search window; 5. Downgrade to normal acquisition; 6. Disable high-precision joint acquisition.

[0130] RTT is mainly used in the following aspects in this mechanism: 1. Assess the quality of the wired control link (i.e., the wired network); 2. Filter out abnormal synchronization samples; 3. Evaluate link latency and jitter before the start of acquisition; 4. Mark the synchronization reliability of this joint acquisition task; 5. Provide a basis for subsequent video start frame cropping, frame sequence offset compensation, multimodal start point correction, and cloud synchronization index generation.

[0131] Therefore, the sync_clock in this application should not be interpreted as a forced calibration of the system time, nor should it rely on clock drift fitting. Its core function is to evaluate the quality of the synchronization link between the backpack master device and the acquisition slave device by measuring RTT multiple times through the wired control link (i.e., wired network) before the start of acquisition, and to use the quality result for subsequent multimodal data synchronization alignment.

[0132] In a joint data collection mission, the main backpack device can execute the following procedure: 1. Before data collection begins, multiple `sync_clock` messages are sent to the slave device via a wired WebSocket (i.e., wired network); 2. The RTT_i is calculated based on the four timestamps of the `sync_clock`; 3. The RTT_min, RTT_median, RTT_jitter, and number of valid samples are counted; 4. It is determined whether the current wired control link meets the requirements for high-precision joint data collection; 5. If the link quality meets the requirements, the master device sends a data collection control command; 6. After receiving the data collection control command, the slave device enters the local data collection process; 7. The slave device records the timestamps related to the actual data collection start point; 8. After data collection is completed, the slave device uploads the collected data and synchronization metadata; 9. The cloud generates a multimodal synchronization index based on the synchronization metadata and collected data.

[0133] After data acquisition begins from the device and enters the local acquisition process, the following steps can be performed: 1. Turn on the camera; 2. Turn on the microphone; 3. Turn on the IMU; 4. Initialize the encoder; 5. Create a file; 6. Discard the warm-up frame or mark the warm-up frame as invalid; 7. Start writing the actual acquisition data; 8. Record the first real frame, the first audio segment, and the local monotonic clock timestamp of the first set of IMUs.

[0134] Among them, the data acquisition device can record at least the following synchronization metadata: device_first_frame_time_ns device_first_audio_time_ns device_first_imu_time_ns device_capture_start_monotonic_time_ns sync_clock_RTT_min sync_clock_RTT_median sync_clock_RTT_jitter sync_clock_valid_count clock_type Wherein: device_first_frame_time_ns: Local monotonic clock timestamp corresponding to the first valid video frame from the device; device_first_audio_time_ns: Local monotonic clock timestamp corresponding to the first valid audio data from the device; device_first_imu_time_ns: Local monotonic clock timestamp corresponding to the first set of valid IMU data from the device; device_capture_start_monotonic_time_ns: Local monotonic clock timestamp when the device enters the formal acquisition process; sync_clock_RTT_min, sync_clock_RTT_median, and sync_clock_RTT_jitter: Link quality metrics obtained by sync_clock statistics before the start of acquisition.

[0135] The cloud server or backpack main device post-processing module can be aligned using one of the following reference starting points: 1. The time of sending the acquisition control command of the backpack master device; 2. The actual first frame time of the acquisition slave device; 3. The earliest valid video frame time; 4. The first frame time of the specified master device or specified slave device; 5. Audio abrupt change points, IMU events, image features or other detectable synchronization events.

[0136] Then, the following operations can be performed on the data acquisition results of each device: 1. Video start frame cropping; 2. Video frame sequence offset compensation; 3. Audio sample level offset correction; 4. IMU time window truncation; 5. Pose data interpolation or truncation; 6. Depth frame nearest neighbor matching; 7. Multimodal synchronization index generation.

[0137] When sync_clock_RTT_jitter is small and sync_clock_RTT_min and sync_clock_RTT_median meet preset thresholds, the system can assume that the control link is stable before the start of data acquisition, and the subsequent data start-point alignment has high reliability. When RTT jitter is large or there are insufficient effective RTT samples, the system can expand the post-processing alignment search window, or combine audio features, image features, IMU events, etc. for secondary correction.

[0138] Through this mechanism, each device does not rely on forced system time synchronization, nor does it rely on the arrival time uploaded from the cloud as the collection time. Instead, it relies on the local monotonic clock to record the actual collection timestamp, and uses the sync_clockRTT quality assessment results before the start of collection to assist in the alignment of the starting point of the data collection results of multiple devices.

[0139] After receiving the data collection results from each device, the server can retrieve the following information: Audio: The sampling range covering this time point; IMU: The sampling segment near this time point; Pose: Interpolation of the closest two points; Depth: The most recent depth frame; Video from other devices: the frame closest to host_time; And generate a synchronized index of the data acquisition results based on the following code: { "job_id": "job_id_placeholder", "frame_id": 120, "host_time_ns": 123456789000000, "devices": [ { "device_id": "iphone_001", "video_frame": "video / 000120.jpg", "audio_range": [192000, 193600], "imu_range": [3001, 3010], "pose_index": 120 }, { "device_id": "pico_001", "video_frame": "video / 000119.jpg", "audio_range": [191980, 193580], "imu_range": [2998, 3008], "pose_index": 119 } ] } For IMU or pose data, if the target time lies between two sampling points: (t_a, value_a) (t_b, value_b) Linear interpolation can then be used to obtain the value value_t at time t: value_t = value_a + (value_b - value_a) × (t - t_a) / (t_b - t_a) For attitude quaternions, spherical linear interpolation (SLERP) can be used.

[0140] In one implementation, after receiving a data collection request, the backpack master device can call the cloud service interface via the wireless network to pull or create a joint data collection task, generate a job_id, and confirm the list of participating devices; check the online status of the devices; check the storage, battery level, camera, IMU, and Wi-Fi; perform wired clock synchronization for all data collection slave devices; issue a data collection preparation command; issue a start data collection command; each device starts data collection locally; and each device uploads the data collection results via its own Wi-Fi after collection.

[0141] In one implementation, a stop-collection request can be sent via a mobile control client. Upon receiving the stop-collection request, the backpack master device can immediately send a stop-collection command to all devices (including the backpack master device and the collection slave devices); alternatively, a unified future stop time can be generated so that all devices stop collecting on a unified timeline.

[0142] In one implementation, the data collection results can be used to generate a local directory, which is then aggregated according to job_id after being uploaded to the server, and a cloud directory is generated.

[0143] In one implementation, a visual preview of the data acquisition results can be achieved based on the WebRTC preview module.

[0144] In this embodiment, the control terminal can view the camera footage in real time through the WebRTC preview module to help confirm the viewing angle, acquisition status, and task execution.

[0145] It's important to note the WebRTC preview stream: 1. Not the sole source of formal training data; 2. Does not replace high-quality locally acquired files; 3. Does not affect the WebSocket instruction thread; 4. Low priority can be set; 5. Can limit traffic, reduce bitrate, or disable; 6. New live stream requests can close old live streams, ensuring that only one preview stream is pushed at a time; 7. You can choose between wired LAN preview or wireless preview depending on your network policy.

[0146] Based on the methods described in steps S101 to S105 above, in this embodiment of the application, the backpack master device and the acquisition slave device are connected via a wired network, and the backpack master device and the server, as well as the acquisition slave device and the server, are connected via a wireless network. This enables the backpack master device and the acquisition slave device to connect via a wired network, thereby achieving a low-latency data acquisition control process. It also enables the backpack master device and the acquisition slave device to communicate with the server via a wireless network, thereby achieving the uploading of data acquisition results. This effectively solves the problems of delay and large jitter in the process of sending acquisition control commands, avoids the problem of mutual interference between the control process and the data acquisition uploading process, and meets the comprehensive requirements of mobile, multi-device, high synchronization accuracy and high upload bandwidth in the data acquisition process.

[0147] Furthermore, in this embodiment of the application, a wired network address is allocated to the data acquisition slave device based on the Dynamic Host Configuration Protocol (DHCP) service, and the network address, subnet mask, lease time, and DHCP server identifier of the wired network are distributed. However, the default gateway, domain name system (DNS), and external network route of the wired network are not distributed. This can effectively solve the problem of conflict between the default gateway, DNS, and external network route of the data acquisition slave device in the dual-network connection state.

[0148] Furthermore, this application embodiment, based on the network connection management interface of the data acquisition slave device, binds the control communication connection to the wired network and the data upload connection to the wireless network, which can effectively solve the problem that some data acquisition slave devices have a default network selection, resulting in the inability to reliably separate the data transmission protocol and the transmission control protocol.

[0149] Furthermore, the embodiments of this application can synchronize the first frame of multimodal data among multiple acquisition slave devices during the data acquisition process based on the round-trip delay between the backpack master device and the acquisition slave device, the start acquisition command, and the local monotonic clock timestamp corresponding to the actual acquisition starting point, effectively improving the accuracy and effectiveness of the data acquisition results.

[0150] Furthermore, the server in this application embodiment can perform time alignment of multimodal data acquisition results based on the master device time anchor point and the slave device time anchor point. It can achieve effective alignment of data acquisition results from multiple devices without relying on forced system time synchronization or server-side upload time as the acquisition time, thereby further improving the accuracy of data acquisition results.

[0151] It should be noted that although the steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effect of this application, different steps do not necessarily have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders. These adjusted solutions are equivalent to the technical solutions described in this application and therefore will also fall within the protection scope of this application.

[0152] Those skilled in the art will understand that all or part of the processes in the method of the above-described embodiment can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium can include any entity or device capable of carrying the computer program code, a medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0153] Another aspect of this application provides a computer-readable storage medium.

[0154] In one embodiment of a computer-readable storage medium according to this application, the computer-readable storage medium can be configured to store a program that performs the data acquisition method of the above-described method embodiments. This program can be loaded and run by a processor to implement the data acquisition method. For ease of explanation, only the parts related to the embodiments of this application are shown; for specific technical details not disclosed, please refer to the method section of the embodiments of this application. The computer-readable storage medium can be a storage device comprising various electronic devices, such as a magnetic disk, hard disk, optical disk, flash memory, read-only memory, random access memory, etc. Optionally, in the embodiments of this application, the computer-readable storage medium is a non-transitory computer-readable storage medium.

[0155] Another aspect of this application provides a backpack main device.

[0156] In one embodiment of a backpack master device according to this application, the backpack master device may include at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program, which, when executed by the at least one processor, implements the methods described in any of the above embodiments. In some embodiments of this application, the processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. The processor has data and / or signal processing capabilities. The processor may be implemented in software, in hardware, or a combination of both.

[0157] Another aspect of this application provides a data acquisition device.

[0158] In one embodiment of a data acquisition slave device according to this application, the data acquisition slave device may include at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program, which, when executed by the at least one processor, implements the method described in any of the above embodiments. In some embodiments of this application, the processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, in hardware, or a combination of both.

[0159] Another aspect of this application provides a server.

[0160] In one embodiment of a server according to this application, the server may include at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program that, when executed by the at least one processor, implements the methods described in any of the above embodiments. In some embodiments of this application, the processor may be a central processing unit, a microprocessor, a graphics processor, a digital signal processor, or any other suitable processor. The processor has data and / or signal processing capabilities. The processor may be implemented in software, in hardware, or a combination of both.

[0161] Furthermore, this application also provides a data acquisition system.

[0162] In one embodiment of a data acquisition system according to this application, such as Figure 2 As shown, the data acquisition system mainly includes the backpack master device described in the above backpack master device embodiment, the acquisition slave device described in the above acquisition slave device embodiment, and the server described in the above server embodiment. The backpack master device and the acquisition slave device are respectively connected to the server via wireless network communication, and the backpack master device and the acquisition slave device are connected via wired network communication.

[0163] The technical solution of this application has been described above with reference to one embodiment shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of this application is obviously not limited to these specific embodiments. Without departing from the principles of this application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of this application.

Claims

1. A data acquisition method, characterized in that, The method is applied to a backpack master device, wherein the backpack master device is connected to a server via a wireless network, and the backpack master device is connected to a data acquisition slave device via a wired network; the method includes: In response to receiving a data acquisition request from a mobile control client, a data acquisition control command is sent to the data acquisition slave device via the wired network to control the data acquisition of the slave device; and, Based on the aforementioned data collection requirements, data is collected, and the collected data results are uploaded to the server via the wireless network.

2. The data acquisition method according to claim 1, characterized in that, Before sending the acquisition control command, the method further includes: Establish a wired network-based communication connection with the data acquisition slave device; Start the Dynamic Host Configuration Protocol (DHCP) service, and assign the wired network address to the acquisition slave device based on the DHCP service.

3. The data acquisition method according to claim 2, characterized in that, The method further includes: Based on the Dynamic Host Configuration Protocol (DHCP) service, the data acquisition slave device is provided with the network address, subnet mask, lease time, and DHCP server identifier of the wired network.

4. The data acquisition method according to claim 2, characterized in that, The method further includes: The default gateway, domain name system, and external network route of the wired network are not sent to the data collection device.

5. The data acquisition method according to claim 1, characterized in that, Sending acquisition control commands to the acquisition slave device via the wired network includes: Based on the wired network, the acquisition slave device is issued an acquisition preparation command and an acquisition start command, so that the acquisition slave device responds to the acquisition start command, performs data acquisition, and records the local monotonic clock timestamp corresponding to the actual acquisition start point of the acquisition start command.

6. The data acquisition method according to claim 1 or 5, characterized in that, Before sending the acquisition control command to the acquisition slave device via the wired network, the method further includes: Based on the wired network, a clock synchronization command is sent to the data acquisition slave device; The clock synchronization response sent by the acquisition slave device in response to the clock synchronization command is obtained, and the round-trip delay between the backpack master device and the acquisition slave device is obtained based on the sending time of the clock synchronization command of the master device, the receiving time of the clock synchronization command of the acquisition slave device, the sending time of the clock synchronization response of the acquisition slave device, and the receiving time of the clock synchronization response of the master device.

7. The data acquisition method according to claim 6, characterized in that, Before sending the acquisition control command to the acquisition slave device via the wired network, the method further includes: The quality of the data acquisition link is assessed based on the round-trip time.

8. The data acquisition method according to any one of claims 1 to 5 or 7, characterized in that, The method further includes: The master device time anchor point of the backpack master device is uploaded to the server so that the server can achieve time alignment of the data collection results based on the master device time anchor point; The master device time anchor point includes the master device system time and the master device monotonic clock.

9. A data acquisition method, characterized in that, The method is applied to a data acquisition slave device, which communicates with a server via a wireless network and with a backpack master device via a wired network; the method includes: Receive acquisition control commands sent by the backpack master device through the wired network; The acquisition control command controls the acquisition slave device to acquire data, and the acquired data acquisition results are uploaded to the server via the wireless network.

10. The data acquisition method according to claim 9, characterized in that, Before receiving the acquisition control command, the method further includes: The network address of the wired network assigned to the data acquisition slave device by the backpack master device through the Dynamic Host Configuration Protocol service.

11. The data acquisition method according to claim 10, characterized in that, The method further includes: The system receives the network address, subnet mask, lease time, and DHCP server identifier sent by the backpack master device to the data acquisition slave device through the DHCP service.

12. The data acquisition method according to claim 9, characterized in that, The method further includes: Based on the network connection management interface of the data acquisition slave device, the control communication connection is bound to the wired network, and the data upload connection is bound to the wireless network.

13. The data acquisition method according to claim 9, characterized in that, The step of controlling the acquisition slave device to acquire data according to the acquisition control command includes: Receive the data collection preparation command and the data collection start command sent by the backpack master device; In response to the start acquisition command, data acquisition is performed, and the local monotonic clock timestamp corresponding to the actual acquisition start point of the start acquisition command is recorded.

14. The data acquisition method according to claim 9 or 13, characterized in that, Before receiving the acquisition control command, the method further includes: In response to the clock synchronization command issued by the backpack master device, a clock synchronization response is sent to the backpack master device, so that the backpack master device can obtain the round-trip delay between the backpack master device and the acquisition slave device based on the sending time of the clock synchronization command of the master device, the receiving time of the clock synchronization command of the acquisition slave device, the sending time of the clock synchronization response of the acquisition slave device, and the receiving time of the clock synchronization response of the master device.

15. The data acquisition method according to claim 14, characterized in that, The method further includes: The slave device time anchor point of the data acquisition device is uploaded to the server so that the server can achieve time alignment of the data acquisition results based on the slave device time anchor point; The slave device time anchor point includes the slave device system time and the slave device monotonic clock.

16. A data acquisition method, characterized in that, The method is applied to a server, which is connected to both the backpack master device and the data acquisition slave device via a wireless network; the method includes: The system receives data collection results uploaded to the server by the backpack master device and the data collection slave device based on the wireless network.

17. The data acquisition method according to claim 16, characterized in that, The method further includes: Receive the master time anchor point of the backpack master device and the slave time anchor point of the acquisition slave device; Based on the time anchor points of the master device and the slave device, the data acquisition results of the backpack master device and the acquisition slave device are time-aligned.

18. A backpack main device, characterized in that, include: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores a computer program, which, when executed by the at least one processor, implements the data acquisition method according to any one of claims 1 to 8.

19. A data acquisition device, characterized in that, include: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores a computer program, which, when executed by the at least one processor, implements the data acquisition method according to any one of claims 9 to 15.

20. A server, characterized in that, include: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores a computer program, which, when executed by the at least one processor, implements the data acquisition method according to any one of claims 16 to 17.

21. A computer-readable storage medium storing a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by a processor to perform the data acquisition method according to any one of claims 1 to 17.

22. A data acquisition system, characterized in that, The system includes the backpack master device as described in claim 18, the data acquisition slave device as described in claim 19, and the server as described in claim 20; the backpack master device and the data acquisition slave device are respectively connected to the server via a wireless network, and the backpack master device and the data acquisition slave device are connected via a wired network.

Images