Clock synchronization method, system, device and medium for human-machine environment data acquisition

Abstract

The embodiment of the present application provides a clock synchronization method, system, device and medium for collecting human-machine environment data. In the above method, after the first device in the local area network obtains the first human-machine environment data collection task issued by the second device, it interacts with the second device through messages, and uses the clock information of the second device as the calibration clock to calibrate the collection time of the first human-machine environment data, thereby realizing clock synchronization between multiple devices in the local area network environment. Finally, the first device binds the first human-machine environment data with the calibrated collection time and sends it to the second device, so that the second device can align the first human-machine environment data with the second human-machine environment data according to the calibrated collection time, thereby realizing the alignment of the human-machine environment data collected by different data collectors according to the calibration time, which is convenient for analyzing factors such as human factors and ergonomics of people, machines, human-machine interaction, and environment.

Description

Technical Field

[0001] The embodiments of the present application relate to the fields of electrical digital data processing, bioinformatics technology, human intelligence, and human factors and ergonomics technology, and in particular to a clock synchronization method, system, device, and medium for collecting human-machine environment data. Background Art

[0002] In virtual reality and simulation environments as well as various complex natural environments or real conditions, multiple host computers can be connected to acquisition devices to collect the user's multimodal physiological data, thereby synchronously recording the user's psychological, physiological and / or behavioral multimodal physiological data changes in real time, and synchronously analyzing the user's behavioral performance, cognitive load, fatigue and / or emotional arousal and other aspects.

[0003] In the process of connecting multiple host computers to the collection equipment to collect the physiological data of the user, in order to ensure the consistency and accuracy of the physiological data, the clocks of the multiple host computers need to be calibrated to keep the clocks of each host computer synchronized. In the existing related technology, the clocks of each host computer are generally calibrated by connecting each host computer to an Internet server through the clock of the server, but this solution relies on the Internet and cannot be used in an environment without the Internet. Even in an environment with the Internet, if the Internet network condition is not good, it will also affect the accuracy of clock synchronization between the host computers. Summary of the invention

[0004] The embodiments of the present application provide a clock synchronization method, system and device for collecting human-machine environment data. The embodiments of the present application also provide a computer-readable storage medium to achieve clock synchronization in a local area network environment and improve the accuracy and stability of clock synchronization.

[0005] In a first aspect, an embodiment of the present application provides a clock synchronization method for human-machine environment data collection, comprising: a first device acquires a first human-machine environment data collection task; wherein the first human-machine environment data collection task is initiated by a second device in a local area network where the second device is located, and the second device and the first device are in the same local area network; message interaction is performed with the second device to acquire a timestamp during the message interaction; the timestamp includes clock information of the second device, and the clock information of the second device is a calibration clock for this collection task; according to the timestamp during the message interaction, a time deviation of the first device relative to the second device is determined; according to the time deviation, a collection time of the first human-machine environment data acquired by the first device is calibrated; wherein the first human-machine environment data is collected by the first device and / or at least one data collector connected to the first device, and the data collector is used to collect human-related data, machine-related data, human-machine interaction-related data and / or environment-related data; after binding the first human-machine environment data with the calibrated collection time, it is sent to the second device, so that the second device can align the first human-machine environment data with the second human-machine environment data acquired by the second device according to the calibrated collection time.

[0006] In the clock synchronization method for collecting human-machine environment data, after the first device in the local area network obtains the first human-machine environment data collection task issued by the second device, it interacts with the second device through messages to obtain the timestamp in the message interaction process, and then the first device uses the clock information of the second device as the calibration clock to calibrate the collection time of the first human-machine environment data obtained by the first device, thereby realizing clock synchronization between multiple devices in the local area network environment, avoiding the dependence of clock synchronization on the Internet, and improving the accuracy and stability of clock synchronization. Finally, the first device binds the first human-machine environment data with the calibrated collection time and sends it to the second device, so that the second device can align the first human-machine environment data with the second human-machine environment data obtained by the second device according to the above calibrated collection time, thereby realizing the alignment of human-machine environment data collected by different data collectors according to the calibration time, which is convenient for analyzing factors such as human factors and ergonomics of people, machines, human-machine interaction, and environment.

[0007] In one possible implementation, the message interaction with the second device and obtaining the timestamp during the message interaction includes: sending a clock synchronization request message to the second device; wherein the clock synchronization request message carries a first timestamp, and the first timestamp is the moment when the first device sends the clock synchronization request message; receiving a clock synchronization response message sent by the second device and obtaining a second timestamp; wherein the second timestamp is the moment when the first device receives the clock synchronization response message, and the clock synchronization response message carries a third timestamp and a fourth timestamp, and the third timestamp is the moment when the second device starts to receive the clock synchronization request message, and the fourth timestamp is the moment when the second device sends the clock synchronization response message.

[0008] In one possible implementation method, determining the time deviation of the first device relative to the second device based on the timestamp in the message interaction process includes: determining the time deviation of the first device relative to the second device based on the first timestamp, the second timestamp, the third timestamp and the fourth timestamp.

[0009] In one possible implementation, the first device obtains the first human-computer environment data collection task, including: the first device obtains at least two first human-computer environment data collection tasks; wherein the at least two first human-computer environment data collection tasks are respectively initiated by different second devices in the local area network, and each first human-computer environment data collection task carries the initiating process identifier of the collection task and the clock synchronization server port of the second device.

[0010] In one possible implementation method, the message interaction with the second device includes: determining the second human-computer environment data collection task joined by the first device among the at least two first human-computer environment data collection tasks based on the initiating process identifier and clock synchronization server port carried by each first human-computer environment data collection task; and interacting with the second device that initiates the second human-computer environment data collection task.

[0011] In one possible implementation, the first device obtains the first human-machine environment data collection task including: the first device obtains the first human-machine environment data collection task broadcast by the second device in the local area network; or, the first device receives the first human-machine environment data collection task sent by the second device through the local area network.

[0012] In one possible implementation, after the first human-computer environment data collection task is executed, the first device may also initiate a third human-computer environment data collection task in the local area network, and the third human-computer environment data collection task uses the clock information of the first device as the calibration clock; or, in response to there being no human-computer environment data collection task currently being executed in the local area network, the first device initiates a third human-computer environment data collection task in the local area network, and the third human-computer environment data collection task uses the clock information of the first device as the calibration clock.

[0013] In one possible implementation, the data collector includes one or a combination of the following: a human-related data collector, a machine-related data collector, a human-computer interaction-related data collector, and an environment-related data collector; the human-related data includes one or a combination of the following: skin electrical temperature data, pulse data, blood pressure data, blood oxygen data, electrocardiogram data, electromyography data, muscle oxygen data, respiratory data, biomechanical data, near-infrared brain imaging data, electroencephalogram data, transcranial stimulation data, heart rate variability data, heart rate data, image / video data, sound data, eye movement data, gesture or motion data; the machine-related data includes One or a combination of the following: machine operation data, fault alarm data, machine control data, machine model data, machine communication data, machine positioning data; the human-machine interaction related data includes one or a combination of the following: human-machine voice interaction data, human-machine text interaction data, human-machine touch interaction data, human-machine gesture or action interaction data, human-machine EEG interaction data, human-machine eye movement interaction data, human-machine expression interaction data; the environment-related data includes one or a combination of the following: position data, humidity data, temperature data, chromaticity data, brightness data, weather data, road condition data, traffic data, stimulus signal data, event or signal marking data.

[0014] In a second aspect, an embodiment of the present application further provides a clock synchronization method for human-machine environment data collection, comprising: a second device initiates a human-machine environment data collection task in a local area network where the second device is located, so that at least one first device obtains the human-machine environment data collection task; wherein the first device and the second device are in the same local area network; message interaction is performed with the first device so that the first device obtains a timestamp during the message interaction process, and a time deviation of the first device relative to the second device is determined based on the timestamp; the timestamp includes the clock information of the second device, and the clock information of the second device is a calibration clock for this collection task; receiving the first human-machine environment data sent by the first device, and the first human-machine environment data; The calibration moment for binding environmental data; wherein, the first human-machine environmental data is collected by the first device and / or at least one data collector connected to the first device, and the data collector is used to collect human-related data, machine-related data and / or environment-related data; the calibration moment for binding the first human-machine environmental data is the moment after the first device calibrates the collection moment of the first human-machine environmental data acquired by the first device according to the time deviation; according to the calibration moment for binding the first human-machine environmental data, aligning the first human-machine environmental data with the second human-machine environmental data acquired by the second device; wherein, the second human-machine environmental data is collected by the second device and / or at least one data collector connected to the second device.

[0015] In the clock synchronization method for collecting human-machine environment data, the second device initiates a human-machine environment data collection task in the local area network where the second device is located, so that at least one first device obtains the human-machine environment data collection task, and then the second device interacts with the first device through messages, so that the first device can obtain the timestamp in the message interaction process, and the time deviation of the first device relative to the second device is determined according to the timestamp, so that the clock synchronization between the devices in the local area network environment can be realized, avoiding the dependence of the clock synchronization on the Internet, and improving the accuracy and stability of the clock synchronization. Afterwards, the second device receives the first human-machine environment data sent by the first device, as well as the collection calibration time bound to the first human-machine environment data, and aligns the first human-machine environment data with the second human-machine environment data obtained by the second device according to the collection calibration time bound to the first human-machine environment data, so that the human-machine environment data collected by different data collectors can be aligned according to the calibration time, which is convenient for analyzing factors such as human factors and ergonomics of people, machines, human-machine interaction, and environment.

[0016] In one possible implementation, the second device initiates a human-computer environment data collection task in the local area network where the second device is located, so that at least one first device obtains the human-computer environment data collection task, including: the second device broadcasts the human-computer environment data collection task in the local area network; or, the second device sends the human-computer environment data collection task to the at least one first device through the local area network.

[0017] In one possible implementation, the second device may also create a socket and bind the socket to the clock synchronization server port of the second device; then, the human-computer environment data collection task initiated by the second device carries the initiating process identifier of the collection task and the clock synchronization server port of the second device.

[0018] In one possible implementation method, before aligning the first human-computer environment data with the second human-computer environment data acquired by the second device according to the calibration time bound to the first human-computer environment data, it also includes: receiving the collection time of the first human-computer environment data sent by the first device; and acquiring the second human-computer environment data, the collection time of the second human-computer environment data and the calibration time bound to the second human-computer environment data; wherein the collection time and calibration time of the second human-computer environment data are the same.

[0019] In one possible implementation method, aligning the first human-computer environment data with the second human-computer environment data obtained by the second device according to the calibration time bound to the first human-computer environment data includes: aligning the first human-computer environment data with the second human-computer environment data according to the calibration time bound to the first human-computer environment data and the calibration time bound to the second human-computer environment data.

[0020] In one possible implementation manner, after aligning the first human-machine environment data with the second human-machine environment data, the method further includes: displaying the first human-machine environment data and the second human-machine environment data at the same calibration moment at the same moment.

[0021] In a third aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors; a memory; multiple applications; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the electronic device, cause the electronic device to execute the method provided in the first aspect, or cause the electronic device to execute the method provided in the second aspect.

[0022] In a fourth aspect, an embodiment of the present application provides a clock synchronization system for human-machine environment data collection, comprising: multiple electronic devices provided in the third aspect, at least one electronic device connected to at least one other electronic device and / or at least one data collector; wherein: each of the electronic devices and each of the data collectors are in the same local area network; any electronic device issues a human-machine environment data collection task, and other electronic devices and / or data collectors connected to any of the electronic devices use the clock information of any of the electronic devices as the calibrated clock of the human-machine environment data collection task to execute the human-machine environment data collection task.

[0023] In one possible implementation, the sending of a human-machine environment data collection task by any electronic device includes: after any electronic device completes executing other human-machine environment data collection tasks sent by the other electronic devices, the sending of the human-machine environment data collection task, wherein when any electronic device executes the other human-machine environment data collection tasks, the clock information of the other electronic devices is used as the calibration clock; or, in response to the fact that there is no human-machine environment data collection task currently being executed in the local area network, the sending of the human-machine environment data collection task by any electronic device.

[0024] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the method provided in the first aspect, or the computer executes the method provided in the second aspect.

[0025] In a sixth aspect, an embodiment of the present application provides a computer program, which, when executed by a computer, is used to execute the method provided in the first aspect or the second aspect.

[0026] In one possible design, the program in the sixth aspect may be stored in whole or in part on a storage medium packaged together with the processor, or may be stored in whole or in part on a memory not packaged together with the processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A schematic diagram of the structure of a first device provided in one embodiment of the present application;

[0028] Figure 2 A schematic diagram of the structure of a clock synchronization system for collecting human-machine environment data provided by an embodiment of the present application;

[0029] Figure 3 A schematic diagram of calculating time deviation provided for one embodiment of the present application;

[0030] Figure 4A flowchart of a clock synchronization method for collecting human-machine environment data provided by an embodiment of the present application;

[0031] Figure 5 A clock synchronization method for collecting human-machine environment data provided by another embodiment of the present application;

[0032] Figure 6 A schematic structural diagram of a first device provided in another embodiment of the present application. DETAILED DESCRIPTION

[0033] The terms used in the implementation section of this application are only used to explain the specific embodiments of this application and are not intended to limit this application.

[0034] In the process of connecting multiple host computers to data acquisition devices and collecting data, in order to ensure the consistency and accuracy of the data, the existing method is usually that each host computer connects to the Internet public network time protocol (NTP) server and corrects the local time according to the local time and the public NTP server time.

[0035] NTP is a protocol for synchronizing clocks on a computer network, allowing clients to request and receive time from a server, which in turn receives precise Coordinated Universal Time (UTC) from an authoritative clock source, such as an atomic clock or a global positioning system (GPS).

[0036] The solution of connecting to a public NTP server for clock synchronization is a common solution for time synchronization through the Internet. However, this solution cannot be implemented in a LAN environment that is not connected to the Internet, and it relies on good Internet network conditions. Poor network conditions will affect the accuracy of time deviation and thus the accuracy of clock synchronization.

[0037] The clock synchronization method for human-machine environment data collection provided in the embodiment of the present application can be applied to electronic devices, wherein the above-mentioned electronic devices can be personal computers (personal computers, PCs), smart phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (augmented reality, AR) / virtual reality technology (virtual reality, VR) devices, laptop computers or ultra-mobile personal computers (ultra-mobile personal computers, UMPCs) and other devices; the embodiment of the present application does not impose any restrictions on the specific type of electronic devices.

[0038] For example, Figure 1 A schematic diagram of the structure of an electronic device provided by an embodiment of the present application is shown as follows: Figure 1 As shown, the electronic device 100 may include one or more processors 110; a memory 120; multiple applications; and one or more computer programs, wherein the one or more computer programs are stored in the memory 120, and the one or more computer programs include instructions. When the above instructions are executed by the electronic device 100, the electronic device 100 can execute the clock synchronization method for human-computer environment data collection provided in an embodiment of the present application.

[0039] In addition, the electronic device 100 may further include a communication interface 120. The processor 110, the communication interface 120 and the memory 120 may communicate with each other through an internal connection path to transmit control and / or data signals.

[0040] The processor 110 and the memory 120 may be combined into a processing device, or more commonly, they are independent components, and the processor 110 is used to execute program codes stored in the memory 120. In specific implementation, the memory 120 may also be integrated into the processor 110, or may be independent of the processor 110.

[0041] In addition, in order to make the functions of the electronic device 100 more complete, the electronic device 100 may further include one or more of an input unit 160 and a display unit 170 .

[0042] Optionally, the electronic device 100 may further include a power supply 150 for providing power to various devices or circuits in the electronic device 100 .

[0043] It should be understood that Figure 1 The processor 110 in the electronic device 100 shown may be a system on chip SOC, and the processor 110 may include a central processing unit (CPU), and may further include other types of processors, such as a graphics processing unit (GPU) and the like.

[0044] It should be noted that when an electronic device 100 executes a data collection task initiated by another electronic device, it is equivalent to the first device in the following embodiments of the present application, but when the electronic device 100 itself issues a data collection task, it is equivalent to the second device in the following embodiments of the present application. One or both of the two functions of executing data collection tasks and issuing data collection tasks can be integrated on an electronic device 100, and multiple electronic devices 100 can be built into the following Figure 2 The clock synchronization system for human-machine environment data acquisition is shown.

[0045] Figure 2A schematic diagram of the structure of a clock synchronization system for collecting human-machine environment data provided by an embodiment of the present application is shown in FIG. Figure 2 As shown, the clock synchronization system for human-machine environment data collection may include: at least one first device 201, at least one data collector 202 connected to at least one first device 201, at least one second device 203 and at least one data collector 204 connected to at least one second device 203.

[0046] The at least one first device 201, the at least one data collector 202 connected to the at least one first device 201, the at least one second device 203 and the at least one data collector 204 connected to the at least one second device 203 are in the same local area network.

[0047] In specific implementation, the first device 201 and the second device 203 may adopt Figure 1 The structure shown is implemented, and the above-mentioned data collector includes one of the following or a combination: a human body-related data collector, a machine-related data collector, a human-computer interaction-related data collector, and an environment-related data collector; specifically, the above-mentioned data collector can be various physiological signal acquisition sensors such as electroencephalogram, electrodermal and / or heart rate, and can also be an eye tracker, an image or video sensor, a machine speed sensor, a steering wheel and pedal mechanical sensor, a temperature sensor or a humidity sensor, etc. In addition, the above-mentioned data collector can be a wearable, non-wearable, contact, non-contact sensor or other types of data collectors. This embodiment does not limit the type of the above-mentioned data collector.

[0048] In this embodiment, the data collector can be used to collect human-machine environment data, and the human-machine environment data can include one or a combination of the following: human body related data, machine related data, human-machine interaction related data, and environment related data;

[0049] The human body related data includes one or a combination of the following: skin electrical temperature data, pulse data, blood pressure data, blood oxygen data, electrocardiogram data, electromyography data, muscle oxygen data, respiratory data, biomechanical data, near-infrared brain imaging data, electroencephalography data, transcranial stimulation data, heart rate variability data, heart rate data, image / video data, sound data, eye movement data, gesture or action data;

[0050] Machine-related data includes one or a combination of the following: machine operation data, fault alarm data, machine control data, machine model data, machine communication data, and machine positioning data;

[0051] Human-computer interaction related data include one or a combination of the following: human-computer voice interaction data, human-computer text interaction data, human-computer touch interaction data, human-computer gesture or action interaction data, human-computer EEG interaction data, human-computer eye movement interaction data, and human-computer expression interaction data;

[0052] The environment-related data includes one or a combination of the following: location data, humidity data, temperature data, colorimetric data, brightness data, weather data, road condition data, traffic data, stimulus signal data, event or signal marking data.

[0053] The above data is not a complete list of various data that need to be obtained for human factors and ergonomics analysis. The above human-machine environment data may also include other data, and this embodiment does not limit the above human-machine environment data.

[0054] It should be noted that Figure 2 The first device 201 and the data collector 202 are taken as two devices respectively as an example, but the present embodiment is not limited to this. The first device 201 and the data collector 202 can also be integrated into one device, that is, the first device 201 integrates the function of the data collector, and the first device 201 itself can complete the function of data collection; similarly, the second device 203 and the data collector 204 can also be integrated into one device, that is, the second device 203 itself can complete the function of data collection.

[0055] Figure 2 In the system shown, the first device 201 is responsible for monitoring and managing the data collector 202, and the second device 203 is responsible for monitoring and managing the data collector 204. Therefore, the first device 201 is the host computer of the data collector 202; the second device 203 is the host computer of the data collector 204. It can be seen that Figure 2 The system shown can be regarded as a multi-host computer signal acquisition system in a local area network environment.

[0056] In a multi-host computer signal acquisition system under a local area network environment, a user operates the program process interface of one of the hosts (for example, the second device 203) based on the business process, thereby starting a collection task process. In a multi-host computer signal acquisition system, the host computer that initiates the collection task is used as a device that provides a calibration clock for the data related to this collection task. In this embodiment, it is assumed that the user starts the collection task process on the second device 203, and the second device 203 is used as the host computer that initiates the collection process to provide a calibration clock for the data related to this collection task.

[0057] The second device 203 continuously broadcasts relevant information of the current collection task to a specific user datagram protocol (UDP) port in the local area network, including the Internet protocol (IP) address of the collection task initiating process, the NTP server port (the default is 123 according to NTP), the current NTP server time (denoted as t1), etc. The above-mentioned UDP broadcast port can be set through a configuration file to ensure that all host computers are configured consistently during the deployment of the host computer software system.

[0058] The NTP server protocol is implemented on the second device 203, and the clock information of the host computer is designated as the calibration clock for this collection task.

[0059] NTP is an application layer protocol based on UDP in the Transmission Control Protocol (TCP) / IP protocol family. In this embodiment, the second device 203 uses the clock information of the local device as the clock source. Only a simplest NTP server implementation is required. The implementation process may include:

[0060] Step 1, create a UDP socket.

[0061] Creates a UDP socket and binds it to the specified host and port (default is localhost and port 123).

[0062] Step 2: Receive the request.

[0063] Enter an infinite loop, waiting to receive the NTP request message from the NTP client.

[0064] Step 3: Process the request and generate a response.

[0065] Once the NTP request message from the NTP client is received, the current system time t4 is recorded. After the NTP request message is received, the NTP client sending time t3 contained therein is obtained, and then the current system time t5 is obtained again.

[0066] Encapsulate t3, t4, and t5 into the NTP response message.

[0067] Step 4: Send a response.

[0068] The generated NTP response message is sent back to the NTP client.

[0069] In a multi-host signal acquisition system, the differences between the NTP server implemented in a LAN environment and the public NTP server in an Internet environment may include but are not limited to the following points:

[0070] 1) Selection of time source:

[0071] In the Internet environment: Public NTP servers usually use high-precision time sources, such as GPS receivers or atomic clocks, to ensure accurate time services.

[0072] In a LAN environment: the calibration clock is associated with the device node that publishes the collection task. When executing the collection task, the local clock of the device that publishes the collection task is used as the time source for calibration. Even if there is a deviation between the time source and the Internet standard time, it can meet the clock synchronization requirements of multiple host computers in the process of collecting human-machine environment signals.

[0073] 2) Access rights and security

[0074] In the Internet environment: Public NTP servers are open to all network users, and access control and security need to be considered to prevent malicious use and attacks.

[0075] In a LAN environment: it is only open to devices within the LAN, and the network environment is more controllable. In scenarios with high security and confidentiality requirements such as those involving secrets, military and / or politics, it is necessary to provide a clock synchronization solution that is different from the Internet environment.

[0076] Specifically, in the multi-host computer signal acquisition system in the LAN environment provided by the embodiment of the present application, the calibration clock is associated with the device node that publishes the acquisition task. Whichever device in the LAN publishes the acquisition task, the local clock of the device is used as the calibration clock for the data related to the acquisition task when executing the acquisition task. In other words, the identification of the NTP server is related to the actual business process. In the business process, each host computer may serve as the initiating process of the acquisition process, so each host computer may serve as the NTP server. Furthermore, in order to avoid acquisition conflicts, the embodiment of the present application introduces a conflict avoidance mechanism similar to device exclusivity, that is, the device that initiates the acquisition task is the NTP server, and once the acquisition task is started, the devices are occupied, so another acquisition task will not be started on the same device at the same time, that is, a host computer can only participate in one acquisition task at the same time.

[0077] Although due to the exclusivity of the device, a device can only participate in one collection task at the same time, different host computers in the same LAN may participate in different collection tasks, realizing concurrent processing of system data collection and improving the efficiency of data collection and time synchronization. Assume that in a LAN, there are two groups of host computers that start the collection process at the same time, such as group A (host computer 1, host computer 2 and host computer 3) and group B (host computer 4 and host computer 5):

[0078] In group A, host computer 1 initiates a collection task and broadcasts the collection task information to the configured UDP port;

[0079] In group B, host computer 4 initiates a collection task and broadcasts the collection task information to the configured UDP port.

[0080] Assuming that the UDP broadcast port configured for the two groups of host computers is the same, host computers 2 and 3 in group A and host computer 5 in group B all receive two collection tasks, and host computers 2, 3 and 5 all display two collection tasks in the interface list. At this time, the user can choose to join a collection task. Once a host computer joins a collection task, it will not join another collection task at the same time due to the exclusivity of the device, so there will be no conflict in NTP time synchronization during the collection process.

[0081] In a multi-host computer signal acquisition system, the user starts other host computer processes based on the business process. These processes receive the acquisition task information from the above-mentioned UDP broadcast port and display it on the interface. The user selects the acquisition task from the acquisition tasks displayed on the interface to make the current host computer join this acquisition task. These host computers are NTP clients. In this embodiment, the NTP client can be Figure 2 The first device 201 in.

[0082] In this embodiment, the NTP server (second device 203) continuously performs UDP broadcast, and the NTP client (first device 201) receives the UDP broadcast, obtains the NTP server information and the time t1 of sending the UDP broadcast message from the above UDP broadcast, and records the current system time t2 of the first device 201. Then, after the user chooses to join the collection task initiated by the second device 203, the first device 201 requests the second device 203 to obtain the standard time. When sending the NTP request message, the first device 201 obtains the current system time t3 and encapsulates it into the above NTP request message. After receiving the NTP response message sent by the second device 203, the first device 201 obtains the current system time t6 and obtains the time deviation of the local device relative to the second device 203.

[0083] Figure 3 A schematic diagram of calculating time deviation provided by an embodiment of the present application, Figure 3 In the example, the host computer initiating the collection task may be the second device 203, and the collection host computer may be the first device 201. Figure 3 The various timestamp parameters in are described below.

[0084] t1: the time when the collection task initiator sends a UDP broadcast each time;

[0085] t2: the time when the first device 201 receives the UDP broadcast;

[0086] t3: the time when the first device 201 sends the NTP request message after the user chooses to join the collection task;

[0087] t4: the time when the collection task initiation process (the second device 203) starts to receive the NTP request message;

[0088] t5: the time when the second device 203 sends the NTP response message;

[0089] t6: the time when the first device 201 receives the NTP response message.

[0090] The first device 201 can calculate the time deviation of the first device 201 relative to the second device 203 by using four of the above timestamp parameters.

[0091] First, the first device 201 calculates the round-trip delay delay of the NTP message from the first device 201 to the second device 203.

[0092] delay = (t6 – t3)-(t5 – t4) (1)

[0093] Secondly, the first device 201 calculates the time offset between the first device 201 and the second device 203 .

[0094] Specifically, the first device 201 can solve the time deviation offset according to the equation group (2), and the offset can be shown as formula (3):

[0095]

[0096] After calculating and obtaining the offset, the first device 201 can obtain the calibration time according to the local time and the above time deviation, as shown in formula (4).

[0097] t = t0 + offset (4)

[0098] In formula (4), t is the calibration time of the first device 201, and t0 is the local time of the first device 201.

[0099] Next, during the data collection process, after the first device 201 obtains the human-machine environment data collected by the data collector 202, it can calibrate the collection time of the human-machine environment data according to formula (4) to obtain the calibration time (i.e., the collection time after calibration), and then the first device 201 binds the human-machine environment data collected by the data collector 202 to the calibration time. In some examples, the first device 201 can create a data table in the database to store the above human-machine environment data. The data table can be shown in Table 1. Each row of the data table represents the human-machine environment data collected by the data collector 202 connected to the first device 201 each time. For example, data column 1 in Table 1 can be physiological data of the human body collected by the data collector 202, data column 2 can be machine operation data, data column 3 can be environmental data, and so on. In addition to the human-machine environment data collected by the data collector 202, the above data table also includes two columns: local time and calibration time, wherein the local time is the local time of the first device 201, which can also be regarded as the time when the data collector 202 collects the above human-machine environment data, and the calibration time is the time after the first device 201 calibrates the local time according to formula (4); It can be seen from Table 1 that in the data table of the first device 201, the local time is different from the calibration time.

[0100] Table 1

[0101]

[0102] After the first device 201 binds the human-machine environment data collected by the data collector 202 with the calibration time, it sends it to the second device 203. That is, the first device 201 can send Table 1 to the second device 203. While the first device 201 is performing the collection task, the second device 203 is also performing the collection task. After the second device 203 obtains the human-machine environment data collected by the data collector 204, it can store the human-machine environment data collected by the data collector 204 in the same way as Table 1, as shown in Table 2.

[0103] Table 2

[0104]

[0105] Compared with Table 1, the difference is that the second device 203 is the initiator of the collection task, so Figure 2 In the collection system shown, the clock information of the second device 203 is used as the calibration clock for this collection task, so in the data table of the second device 203, the local time and the calibration time are the same.

[0106] After the collection task is completed, the second device 203 aligns and replays the human-machine environment data in Table 1 sent by the first device 201 with the human-machine environment data in Table 2. Specifically, the second device 203 can align the human-machine environment data in Table 1 with the human-machine environment data in Table 2 according to the calibration time in Table 1 and the calibration time in Table 2, so that the human-machine environment data with the same calibration time is the human-machine environment data collected at the same time.

[0107] After aligning the human-machine environment data in Table 1 with the human-machine environment data in Table 2, the second device 203 can also play back the human-machine environment data collected in this collection task. When playing back the human-machine environment data, the second device 203 can display the human-machine environment data with the same calibration time at the same time.

[0108] In the description of the above embodiment, the second device 203 is an NTP server and the first device 201 is an NTP client. However, in the clock synchronization system for human-machine environment data collection provided by the embodiment of the present application, different devices can be understood as peer nodes, and there is no concept of client-server (CS). That is to say, in the clock synchronization system for human-machine environment data collection, any electronic device can issue a human-machine environment data collection task, and then other electronic devices and / or data collectors connected to any of the above electronic devices use the clock information of any of the above electronic devices as the calibration clock of the human-machine environment data collection task to execute the human-machine environment data collection task. Specifically, any electronic device issuing a human-machine environment data collection task can be: after any electronic device completes executing other human-machine environment data collection tasks issued by other electronic devices, it issues a human-machine environment data collection task, wherein any of the above electronic devices uses the clock information of other electronic devices as the calibration clock when executing the other human-machine environment data collection tasks; or, in response to the fact that there is no human-machine environment data collection task currently being executed in the local area network, any of the above electronic devices issues a human-machine environment data collection task.

[0109] In short, in the clock synchronization system for human-machine environment data collection provided by the embodiment of the present application, the same device can be the initiator of the data collection task at certain moments, in which case the device plays the role of the second device 203 in the above embodiment, and can also be the executor of the data collection task initiated by other devices at certain moments, in which case the device plays the role of the first device 201 in the above embodiment. Thus, in a multi-device environment based on a local area network, clock synchronization based on the granularity of the collection task can be flexibly implemented according to the collection task.

[0110] The following describes how to use Figure 2 The system shown performs an example of a human-machine environment data collection task.

[0111] In this example, the operator can specify Figure 2 The second device 203 in the system shown is an NTP server, and the first device 201 is an NTP client, which realizes the clock synchronization of multiple host computers. In virtual reality and simulation environments and various complex natural environments or real conditions, the first device 201 and the second device 203 can synchronously record the user's multimodal data changes such as psychology, physiology and behavior in real time. Using the recorded data, the second device 203 can synchronously analyze the user's behavior performance, cognitive load, fatigue and / or emotional arousal.

[0112] In this example, the second device 203 is the main operating device of the user, and the data collector 204 connected to the second device 203 may be an eye tracker, which collects the user's eye movement data in real time during the experiment. After the second device 203 obtains the eye movement data collected by the eye tracker, it records the eye movement data, which can be used to analyze the operator's behavior performance and / or physiological states such as attention;

[0113] The data collector 202 connected to the first device 201 can be a sensor device such as a physiological bracelet and / or an ear clip. The above-mentioned sensor devices such as the physiological bracelet and / or the ear clip collect the user's physiological signals in real time during the experiment. After the first device 201 obtains the physiological signals collected by the above-mentioned sensor devices such as the physiological bracelet and / or the ear clip, it records the above-mentioned physiological signals. The above-mentioned physiological signals can be used to analyze the user's physiological states such as cognitive load, fatigue and / or emotional arousal.

[0114] After the second device 203 is started, the user creates and starts a human-computer environment data collection task in the process of the second device 203. At this time, the second device 203 acts as an NTP server and broadcasts the collection task information to the configured UDP port. The above collection task information may include the collection task initiating process IP and the NTP server port (the default is 123 according to the NTP protocol), etc.

[0115] After the first device 201 is started, it continuously listens to the configured UDP port, receives the broadcast signal, obtains the information of the NTP server from the broadcast signal, and displays at least one received collection task in the interface list. Then, in the process interface of the first device 201, the user selects the collection task to be added by the first device 201 according to the information of the NTP server, and then the user clicks on the selected collection task to make the first device 201 join the selected collection task. Afterwards, the first device 201 sends an NTP request message to the second device 203 based on the information obtained from the NTP server. After receiving the NTP response message sent back by the second device 203, the first device 201 can calculate the time deviation of the first device 201 relative to the second device 203 according to formula (3). The detailed process of calculating the time deviation can be found in Figure 3 The description of the illustrated embodiment will not be repeated here.

[0116] After calculating the time deviation, the first device 201 can calibrate the collection time of the physiological signal collected by the sensor devices such as the physiological bracelet and / or ear clip according to the above-mentioned time deviation to obtain the calibration time, and then the first device 201 binds the above-mentioned physiological signal with the above-mentioned calibration time, for example: recording the above-mentioned physiological signal and the calibration time in the data table shown in Table 1, and then sending the recorded data table to the second device 203.

[0117] After the second device 203 obtains the eye movement data collected by the eye tracker, the eye movement data can be recorded in the data table shown in Table 2. When recording the eye movement data in the data table shown in Table 2, the second device 203 calibrates the time column and directly stores the local time.

[0118] After the collection task is completed, the second device 203 can align and analyze the data according to the calibration time columns of the two data tables and play back the data.

[0119] As described above, the embodiment of the present application provides a clock synchronization method for collecting human-machine environment data. Figure 4 This is a flowchart of a clock synchronization method for collecting human-machine environment data provided by an embodiment of the present application. The method provided by this embodiment can be applied to Figure 2 The first device 201 in the system shown, such as Figure 4 As shown, the clock synchronization method for collecting human-machine environment data may include:

[0120] Step 401: The first device 201 obtains a first human-machine environment data collection task.

[0121] The first human-machine environment data collection task is initiated by the second device 203 in the local area network where the second device 203 is located, and the second device 203 and the first device 201 are in the same local area network.

[0122] Specifically, the first device 201 obtains the first human-machine environment data collection task by: the first device 201 obtains the first human-machine environment data collection task broadcast by the second device 203 in the above-mentioned local area network; or, the first device 201 receives the first human-machine environment data collection task sent by the second device 203 through the above-mentioned local area network.

[0123] That is to say, the first device 201 can listen to the configured broadcast port to obtain the first human-computer environment data collection task broadcast by the second device 203 in the above-mentioned local area network; or, the second device 203 can also directly send the first human-computer environment data collection task to the first device 201, and the first device 201 receives the first human-computer environment data collection task sent by the second device 203 through the above-mentioned local area network.

[0124] In step 402, the first device 201 performs message interaction with the second device 203, and obtains a timestamp during the message interaction.

[0125] The timestamp includes the clock information of the second device 203 , and the clock information of the second device 203 is the calibration clock of this acquisition task.

[0126] Specifically, the first device 201 obtains the first human-computer environment data collection task as follows: the first device 201 obtains at least two first human-computer environment data collection tasks; wherein the above-mentioned at least two first human-computer environment data collection tasks are respectively initiated by different second devices 203 in the above-mentioned local area network, and each first human-computer environment data collection task carries the initiation process identifier of the above-mentioned collection task and the clock synchronization server port of the second device 203.

[0127] In this way, the message interaction between the first device 201 and the second device 203 can be as follows: the first device 201 determines the second human-computer environment data collection task joined by the first device 201 in the above-mentioned at least two first human-computer environment data collection tasks according to the initiating process identifier and the clock synchronization server port carried by each first human-computer environment data collection task; then, the first device 201 interacts with the second device 203 that initiates the second human-computer environment data collection task.

[0128] That is, the first device 201 may receive at least two first human-machine environment data collection tasks, and the first device 201 may determine the second human-machine environment data collection task that the first device 201 is to join based on the initiating process identifier and the clock synchronization server port, and due to the exclusivity of the device, the first device 201 can only participate in one human-machine environment data collection task at the same time. After determining that the first device 201 is to join the second human-machine environment data collection task, the first device 201 may perform message interaction with the second device 203 that initiated the second human-machine environment data collection task.

[0129] In the specific implementation, in the data collection scenario of human factors and ergonomics analysis, multiple devices in the local area network (i.e., host computers, such as mobile phones and / or PCs, etc.) can broadcast human-machine environment data collection tasks respectively. That is to say, there may be multiple human-machine environment data collection tasks in the local area network at the same time. The user of the device that publishes the collection task is usually the leader of the collection task, called the main tester, and the user of the device that listens to the collection task is usually the test subject, called the test subject. Each collection task usually includes a main tester and one or more test subjects. The main testers and test subjects of different collection tasks can be different. The test subject can input whether to join and which data collection task to join on the device used by the test subject according to the preparation of the data collector (such as a bracelet, an electroencephalogram cap, an eye tracker and / or a camera, etc.) that he wears or needs to prepare. After the data collection task is released, the confirmation right of whether to join the data collection task and which data collection task to join is handed over to the test subject to determine, which is conducive to ensuring the effectiveness and accuracy of data collection.

[0130] Step 403: The first device 201 determines a time deviation of the first device 201 relative to the second device 203 according to the timestamp in the message interaction process.

[0131] In this embodiment, the first device 201 and the second device 203 perform message interaction, and the timestamp obtained in the message interaction process can be: the first device 201 sends a clock synchronization request message to the second device 203; wherein the above-mentioned clock synchronization request message carries a first timestamp, and the first timestamp is the moment when the first device 201 sends the above-mentioned clock synchronization request message; then, the first device 201 receives the clock synchronization response message sent by the second device 203, and obtains a second timestamp; wherein the above-mentioned second timestamp is the moment when the first device 201 receives the above-mentioned clock synchronization response message, and the above-mentioned clock synchronization response message carries a third timestamp and a fourth timestamp, and the above-mentioned third timestamp is the moment when the second device 203 starts to receive the above-mentioned clock synchronization request message, and the fourth timestamp is the moment when the second device 203 sends the above-mentioned clock synchronization response message.

[0132] In this way, the first device 201 can determine the time deviation of the first device 201 relative to the second device 203 according to the first timestamp, the second timestamp, the third timestamp and the fourth timestamp. The specific manner in which the first device 201 determines the above time deviation can be shown in formula (3), which will not be repeated here.

[0133] The clock synchronization request message may be an NTP request message, and the clock synchronization response message may be an NTP response message.

[0134] Step 404 : The first device 201 calibrates the collection time of the first human-machine environment data acquired by the first device 201 according to the above time deviation.

[0135] Among them, the above-mentioned first human-machine environment data can be collected by the first device 201 and / or at least one data collector connected to the first device 201, and the above-mentioned data collector is used to collect human-related data, machine-related data and / or environment-related data.

[0136] That is to say, the first device 201 and the data collector can be two devices set separately, or they can be integrated into one device, that is, the first device 201 integrates the function of the data collector, and the first device 201 itself can complete the data collection function.

[0137] In step 405, the first device 201 binds the first human-machine environment data with the calibrated collection time and sends the data to the second device 203, so that the second device 203 can align the first human-machine environment data with the second human-machine environment data acquired by the second device 203 according to the calibrated collection time.

[0138] In some examples, after the first human-machine environment data collection task is completed, the first device 201 may also initiate a third human-machine environment data collection task in the local area network, and the third human-machine environment data collection task uses the clock information of the first device 201 as the calibration clock. In other words, after the first human-machine environment data collection task is completed, the first device 201 may initiate a new third human-machine environment data collection task, so that the first device 201 becomes a clock synchronization server, and the third human-machine environment data collection task uses the clock information of the first device 201 as the calibration time.

[0139] The first device 201 determines that the first human-machine environment data collection task has been completed by: the first device 201 determines that the first human-machine environment data collection task has been completed by listening to broadcasts. Specifically, the first device 201 does not hear the broadcast instruction of the second device 203 about the first human-machine environment data collection task within the set time, and determines that the task is completed; or the second device 203 actively broadcasts to inform the first device 201 that the first human-machine environment data collection task has been completed. Specifically, when the second device 203 determines that the first human-machine environment data collection task is to be terminated, it broadcasts the first human-machine environment data collection task end instruction in the local area network. After the first device 201 obtains the end instruction, it determines that the first human-machine environment data collection task is completed. This facilitates the initiation of other collection tasks in the local area network.

[0140] In other examples, in response to there being no human-computer environment data collection task currently being executed in the local area network, the first device 201 initiates a third human-computer environment data collection task in the local area network, and the third human-computer environment data collection task uses the clock information of the first device 201 as the calibration clock.

[0141] In the clock synchronization method for collecting human-machine environment data, after the first device 201 obtains the first human-machine environment data collection task, it interacts with the second device 203 through messages to obtain the timestamp in the message interaction process, wherein the timestamp includes the clock information of the second device 203, and the clock information of the second device 203 is the calibration clock for this collection task. Then, the first device 201 determines the time deviation of the first device 201 relative to the second device 203 according to the timestamp in the message interaction process, and calibrates the collection time of the first human-machine environment data obtained by the first device 201 according to the time deviation, thereby realizing clock synchronization between multiple devices in a local area network environment, avoiding the dependence of clock synchronization on the Internet, and improving the accuracy and stability of clock synchronization. Finally, after the first device 201 binds the first human-machine environment data with the calibrated collection time, it sends it to the second device 203, so that the second device 203 can align the first human-machine environment data with the second human-machine environment data obtained by the second device 203 according to the above-mentioned calibrated collection time, thereby realizing the alignment of the human-machine environment data collected by different data collectors according to the calibration time, which is convenient for the analysis of human factors and ergonomics of factors such as people, machines, human-computer interaction, and environment.

[0142] Figure 5 Another embodiment of the present application provides a clock synchronization method for collecting human-machine environment data. The method provided in this embodiment can be applied to Figure 2 The second device 203 in the system shown, such as Figure 5 As shown, the clock synchronization method for collecting human-machine environment data may include:

[0143] Step 501 , the second device 203 initiates a human-machine environment data collection task in the local area network where the second device 203 is located, so that at least one first device 201 obtains the above human-machine environment data collection task.

[0144] The first device 201 and the second device 203 are in the same local area network.

[0145] In some examples, the second device 203 initiates a human-computer environment data collection task in the local area network where the second device 203 is located so that at least one first device 201 obtains the above-mentioned human-computer environment data collection task. The task can be: the second device 203 broadcasts the above-mentioned human-computer environment data collection task in the above-mentioned local area network; or, the second device 203 can also send the above-mentioned human-computer environment data collection task to at least one first device 201 through the above-mentioned local area network.

[0146] In this embodiment, before the second device 203 initiates the human-machine environment data collection task, the second device 203 may also create a socket (e.g., UDP socket) and bind the socket to the clock synchronization server port (e.g., port 123) of the second device 203. In this way, the human-machine environment data collection task initiated by the second device 203 carries the initiating process identifier of the collection task and the clock synchronization server port of the second device 203.

[0147] In step 502 , the second device 203 performs message interaction with the first device 201 , so that the first device 201 obtains a timestamp during the message interaction, and determines a time deviation of the first device 201 relative to the second device 203 based on the timestamp.

[0148] The timestamp includes the clock information of the second device 203 , and the clock information of the second device 203 is the calibration clock of this acquisition task.

[0149] Step 503 : The second device 203 receives the first human-machine environment data sent by the first device 201 , and the calibration time bound to the first human-machine environment data.

[0150] Among them, the above-mentioned first human-machine environment data is collected by the first device and / or at least one data collector connected to the first device, and the above-mentioned data collector is used to collect human-related data, machine-related data and / or environment-related data; the calibration moment bound to the first human-machine environment data is the moment after the first device 201 calibrates the collection moment of the first human-machine environment data obtained by the first device 201 according to the above-mentioned time deviation.

[0151] Step 504 : The second device 203 aligns the first human-machine environment data with the second human-machine environment data acquired by the second device 203 according to the calibration time bound to the first human-machine environment data.

[0152] The second human-machine environment data is collected by the second device 203 and / or at least one data collector connected to the second device 203. That is, the second device 203 and the data collector can be two devices set separately, or they can be integrated into one device, that is, the second device 203 integrates the function of the data collector, and the second device 203 itself can complete the function of data collection.

[0153] In this embodiment, before the second device 203 aligns the first human-machine environment data with the second human-machine environment data acquired by the second device 203, the second device 203 may also receive the collection time of the first human-machine environment data sent by the first device 201; and obtain the second human-machine environment data, the collection time of the above-mentioned second human-machine environment data and the calibration time bound to the second human-machine environment data; wherein the collection time and calibration time of the above-mentioned second human-machine environment data are the same. Specifically, the first device 201 may record the first human-machine environment data, the collection time and calibration time of the first human-machine environment data into a data table, and then send the data table to the second device 203. The data table recorded by the first device 201 may be as shown in Table 1; similarly, the second device 203 may also record the second human-machine environment data, the collection time and calibration time of the second human-machine environment data into a data table. The data table recorded by the second device may be as shown in Table 2.

[0154] Specifically, the second device 203 aligns the first human-computer environment data with the second human-computer environment data obtained by the second device 203 according to the calibration time bound to the first human-computer environment data. This can be done by: the second device 203 aligns the first human-computer environment data with the second human-computer environment data according to the calibration time bound to the first human-computer environment data and the calibration time bound to the second human-computer environment data.

[0155] In some examples, after the second device 203 aligns the first human-machine environment data with the second human-machine environment data, it can also display the first human-machine environment data and the second human-machine environment data at the same calibration moment at the same moment.

[0156] In the above-mentioned clock synchronization method for human-computer environment data collection, the second device 203 initiates a human-computer environment data collection task in the local area network where the second device 203 is located, so that at least one first device 201 obtains the above-mentioned human-computer environment data collection task, and then the second device 203 interacts with the first device 201 through messages, so that the first device 201 can obtain the timestamp in the message interaction process, and the time deviation of the first device 201 relative to the second device 203 is determined according to the above-mentioned timestamp, so as to realize clock synchronization between devices in a local area network environment, avoid the dependence of clock synchronization on the Internet, and improve the accuracy and stability of clock synchronization. Afterwards, the second device 203 receives the first human-computer environment data sent by the first device 201, as well as the collection calibration time bound to the first human-computer environment data, and aligns the first human-computer environment data with the second human-computer environment data acquired by the second device 203 according to the collection calibration time bound to the first human-computer environment data. This allows the human-computer environment data collected by different data collectors to be aligned according to the calibration time, making it easier to analyze human factors and ergonomics of factors such as humans, machines, human-computer interaction, and the environment.

[0157] It is to be understood that some or all of the steps or operations in the above embodiments are merely examples, and the present application embodiments may also perform other operations or variations of various operations. In addition, the various steps may be performed in different orders presented in the above embodiments, and it is possible that not all of the operations in the above embodiments need to be performed.

[0158] It is understandable that, in order to realize the above functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to exceed the scope of this application.

[0159] This embodiment can divide the electronic device into functional modules according to the above method embodiment. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0160] Figure 6 A schematic diagram of the structure of an electronic device provided in another embodiment of the present application, in which each functional module is divided according to each function, Figure 6 A possible schematic diagram of the composition of the electronic device 600 involved in the above embodiment is shown. Figure 6 As shown, the electronic device 600 may include: a receiving unit 601, a processing unit 602 and a sending unit 603;

[0161] It should be noted that when the same electronic device 600 executes a data collection task initiated by another electronic device, it is equivalent to the first device in the above embodiment, but when the electronic device 600 itself issues a data collection task, it is equivalent to the second device in the above embodiment. One or both of the two functions of issuing data collection tasks and executing data collection tasks can be integrated on one electronic device 600, and multiple electronic devices 600 can be built. Figure 2 The clock synchronization system for human-machine environment data acquisition is shown.

[0162] When the electronic device 600 is equivalent to the first device in the above embodiment, the receiving unit 601 can be used to support the electronic device 600 to perform steps 401 and 402, and / or other processes of the technical solution described in the embodiment of the present application;

[0163] The processing unit 602 may be used to support the electronic device 600 in executing step 402, step 403, step 404, etc., and / or other processes of the technical solution described in the embodiment of the present application;

[0164] The sending unit 603 can be used to support the electronic device 600 to execute step 405, etc., and / or other processes of the technical solution described in the embodiments of the present application.

[0165] When the electronic device 600 is equivalent to the first device in the above embodiment, the present application Figure 4 All relevant contents of each step involved in the illustrated method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0166] When the electronic device 600 is equivalent to the second device in the above embodiment, the receiving unit 601 can be used to support the electronic device 600 to perform steps 501 and 503, etc., and / or other processes of the technical solution described in the embodiment of the present application;

[0167] The processing unit 602 may be used to support the electronic device 600 in executing step 502 and step 504, etc., and / or other processes of the technical solution described in the embodiment of the present application;

[0168] The sending unit 603 can be used to support the electronic device 600 to execute steps 501 and 502, etc., and / or other processes of the technical solution described in the embodiments of the present application.

[0169] When the electronic device 600 is equivalent to the second device in the above embodiment, the present application Figure 5 All relevant contents of each step involved in the illustrated method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0170] The electronic device 600 provided in this embodiment is used to execute the present application Figure 4 or Figure 5 The clock synchronization method for collecting human-machine environment data provided by the illustrated embodiment can therefore achieve the same effect as the above method.

[0171] It should be understood that the electronic device 600 may correspond to Figure 1 The electronic device 100 shown in FIG. The functions of the receiving unit 601 and the sending unit 603 can be Figure 1 The processor 110 and the communication interface 120 in the electronic device 100 are implemented; the functions of the processing unit 602 can be implemented by Figure 1 The processor 110 in the electronic device 100 is shown implemented.

[0172] In the case of adopting an integrated unit, the electronic device 600 may include a processing module, a storage module, and a communication module.

[0173] The processing module can be used to control and manage the actions of the electronic device 600, for example, it can be used to support the electronic device 600 to execute the steps performed by the above-mentioned receiving unit 601, processing unit 602 and sending unit 603. The storage module can be used to support the electronic device 600 to store program codes and data, etc. The communication module can be used to support the communication between the electronic device 600 and other devices.

[0174] Among them, the processing module can be a processor or a controller, which can implement or execute various exemplary logic boxes, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, and so on. The storage module can be a memory. The communication module can specifically be a device that interacts with other electronic devices, such as a radio frequency circuit, a Bluetooth chip and / or a Wi-Fi chip.

[0175] In one embodiment, when the processing module is a processor and the storage module is a memory, the electronic device 600 involved in this embodiment can be a Figure 1 Device of the structure shown.

[0176] The present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the present application. Figure 4 or Figure 5 The method provided by the illustrated embodiment.

[0177] The present application also provides a computer program product, which includes a computer program, which, when executed on a computer, enables the computer to execute the present application. Figure 4 or Figure 5 The method provided by the illustrated embodiment.

[0178] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.

[0179] Those of ordinary skill in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented in a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0180] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0181] In several embodiments provided in the present application, any function can be stored in a computer-readable storage medium if it is implemented in the form of a software functional unit and sold or used as an independent product. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence or in other words, the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0182] The above is only a specific implementation of the present application. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. The protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A clock synchronization method for collecting human-machine environment data, characterized in that: include: The first device acquires a first human-machine environment data collection task; wherein the first human-machine environment data collection task is initiated by the second device in a local area network where the second device is located, and the second device and the first device are in the same local area network; Perform message interaction with the second device to obtain a timestamp during the message interaction; the timestamp includes clock information of the second device, and the clock information of the second device is a calibrated clock for this collection task; Determine, according to the timestamp in the message interaction process, a time deviation of the first device relative to the second device; Calibrate the collection time of the first human-machine environment data acquired by the first device according to the time deviation; wherein the first human-machine environment data is collected by the first device and / or at least one data collector connected to the first device, and the data collector is used to collect human-related data, machine-related data, human-machine interaction-related data and / or environment-related data; After the first human-machine environment data is bound to the calibrated collection time, it is sent to the second device, so that the second device aligns the first human-machine environment data with the second human-machine environment data acquired by the second device according to the calibrated collection time.

2. The method according to claim 1, characterized in that The performing message interaction with the second device and obtaining a timestamp during the message interaction includes: Sending a clock synchronization request message to the second device; wherein the clock synchronization request message carries a first timestamp, and the first timestamp is the time when the first device sends the clock synchronization request message; Receive a clock synchronization response message sent by the second device and obtain a second timestamp; wherein the second timestamp is the moment when the first device receives the clock synchronization response message, the clock synchronization response message carries a third timestamp and a fourth timestamp, the third timestamp is the moment when the second device starts to receive the clock synchronization request message, and the fourth timestamp is the moment when the second device sends the clock synchronization response message.

3. The method according to claim 2, characterized in that The determining, according to the timestamp in the message interaction process, a time deviation of the first device relative to the second device includes: A time offset of the first device relative to the second device is determined according to the first timestamp, the second timestamp, the third timestamp, and the fourth timestamp.

4. The method according to claim 1, characterized in that The first device acquires the first human-machine environment data collection task including: The first device acquires at least two first human-computer environment data collection tasks; wherein, the at least two first human-computer environment data collection tasks are respectively initiated by different second devices in the local area network, and each first human-computer environment data collection task carries the initiating process identifier of the collection task and the clock synchronization server port of the second device.

5. The method according to claim 4, characterized in that The performing message interaction with the second device includes: Determine, according to the initiating process identifier and the clock synchronization server port carried by each first human-machine environment data collection task, the second human-machine environment data collection task added by the first device from among the at least two first human-machine environment data collection tasks; Perform message interaction with the second device that initiates the second human-machine environment data collection task.

6. The method according to any one of claims 1 to 5, characterized in that: The first device acquires the first human-machine environment data collection task including: The first device acquires a first human-machine environment data collection task broadcasted by the second device in the local area network; or The first device receives a first human-machine environment data collection task sent by the second device through the local area network.

7. The method according to any one of claims 1 to 5, characterized in that: Also includes: After the first human-machine environment data collection task is completed, the first device initiates a third human-machine environment data collection task in the local area network, and the third human-machine environment data collection task uses the clock information of the first device as a calibration clock; or, In response to there being no human-machine environment data collection task currently being executed in the local area network, the first device initiates a third human-machine environment data collection task in the local area network, and the third human-machine environment data collection task uses the clock information of the first device as a calibration clock.

8. The method according to any one of claims 1 to 5, characterized in that: The data collector includes one or a combination of the following: a human body related data collector, a machine related data collector, a human-machine interaction related data collector, and an environment related data collector; The human body related data includes one or a combination of the following: skin electrical skin temperature data, pulse data, blood pressure data, blood oxygen data, electrocardiogram data, electromyography data, muscle oxygen data, respiratory data, biomechanical data, near-infrared brain imaging data, electroencephalography data, transcranial stimulation data, heart rate variability data, heart rate data, image / video data, sound data, eye movement data, gesture or action data; The machine-related data includes one or a combination of the following: machine operation data, fault alarm data, machine control data, machine model data, machine communication data, and machine positioning data; The human-computer interaction related data includes one or a combination of the following: human-computer voice interaction data, human-computer text interaction data, human-computer touch interaction data, human-computer gesture or action interaction data, human-computer EEG interaction data, human-computer eye movement interaction data, and human-computer expression interaction data; The environment-related data includes one or a combination of the following: location data, humidity data, temperature data, chromaticity data, brightness data, weather data, road condition data, traffic data, stimulus signal data, event or signal marking data.

9. A clock synchronization method for collecting human-machine environment data, characterized in that: include: The second device initiates a human-machine environment data collection task in the local area network where the second device is located, so that at least one first device obtains the human-machine environment data collection task; wherein the first device and the second device are in the same local area network; Perform message interaction with the first device so that the first device obtains a timestamp in the message interaction process, and determines a time deviation of the first device relative to the second device according to the timestamp; the timestamp includes clock information of the second device, and the clock information of the second device is a calibration clock for this acquisition task; Receive first human-machine environment data sent by the first device, and a calibration time bound to the first human-machine environment data; wherein the first human-machine environment data is collected by the first device and / or at least one data collector connected to the first device, and the data collector is used to collect human-related data, machine-related data and / or environment-related data; the calibration time bound to the first human-machine environment data is the time after the first device calibrates the collection time of the first human-machine environment data acquired by the first device according to the time deviation; According to the calibration time bound to the first human-machine environment data, the first human-machine environment data is aligned with the second human-machine environment data acquired by the second device; wherein the second human-machine environment data is acquired by the second device and / or at least one data collector connected to the second device.

10. The method according to claim 9, characterized in that The second device initiating a human-machine environment data collection task in a local area network where the second device is located so that at least one first device obtains the human-machine environment data collection task includes: The second device broadcasts the human-machine environment data collection task in the local area network; or, The second device sends the human-machine environment data collection task to the at least one first device through the local area network.

11. The method according to claim 9 or 10, characterized in that: Also includes: Create a socket; Binding the socket to the clock synchronization server port of the second device; Then, the human-machine environment data collection task initiated by the second device carries the initiating process identifier of the collection task and the clock synchronization server port of the second device.

12. The method according to claim 9 or 10, characterized in that: Before aligning the first human-machine environment data with the second human-machine environment data acquired by the second device according to the calibration time bound to the first human-machine environment data, the method further includes: Receive the collection time of the first human-machine environment data sent by the first device; and obtain the second human-machine environment data, the collection time of the second human-machine environment data and the calibration time bound to the second human-machine environment data; wherein the collection time and calibration time of the second human-machine environment data are the same.

13. The method according to claim 12, characterized in that The aligning the first human-machine environment data with the second human-machine environment data acquired by the second device according to the calibration time bound to the first human-machine environment data includes: The first human-machine environment data is aligned with the second human-machine environment data according to a calibration time bound to the first human-machine environment data and a calibration time bound to the second human-machine environment data.

14. The method according to claim 13, characterized in that After aligning the first human-machine environment data with the second human-machine environment data, the method further includes: The first human-machine environment data and the second human-machine environment data having the same calibration time are displayed at the same time.

15. An electronic device, characterized in that: include: one or more processors; Memory; Multiple applications; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the electronic device, cause the electronic device to execute the method as described in any one of claims 1-8, or cause the electronic device to execute the method as described in any one of claims 9-14.

16. A clock synchronization system for collecting human-machine environment data, characterized in that: include: A plurality of electronic devices as claimed in claim 15 and a plurality of data collectors; wherein the plurality of electronic devices as claimed in claim 15 include a plurality of electronic devices that execute the method according to any one of claims 1 to 8 and a plurality of electronic devices that execute the method according to any one of claims 9 to 14; At least one electronic device is connected to at least one other electronic device, and / or at least one electronic device is connected to at least one data collector; wherein: Each of the electronic devices and each of the data collectors are in the same local area network; Any electronic device issues a human-machine environment data collection task, and other electronic devices and / or data collectors connected to the any electronic device use the clock information of the any electronic device as the calibration clock of the human-machine environment data collection task to execute the human-machine environment data collection task.

17. The system according to claim 16, characterized in that The human-machine environment data collection task sent by any electronic device includes: After any of the electronic devices completes the other human-machine environment data collection tasks issued by the other electronic devices, the electronic devices issue the human-machine environment data collection tasks, wherein when any of the electronic devices execute the other human-machine environment data collection tasks, the clock information of the other electronic devices is used as the calibration clock; or In response to there being no human-machine environment data collection task currently being executed in the local area network, any one of the electronic devices issues a human-machine environment data collection task.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed on a computer, enables the computer to execute the method according to any one of claims 1 to 8, or enables the computer to execute the method according to any one of claims 9 to 14.

Images