Evaluation device and evaluation method

Abstract

To evaluate a degree of motion sickness.SOLUTION: An acquisition part acquires an active index value showing a degree of the activity of the sympathetic nerve or parasympathetic nerve of a user, calculated from vital data of the user. An evaluation part evaluates a degree of motion sickness of the user based on a distribution degree relating to the time sequence of the activity index value.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an evaluation device and an evaluation method. [Background technology]

[0002] There is known technology for remotely controlling work machines. In remote control, an operator in a control room operates the machine by viewing an image. As the work machine moves as it operates, the image the operator sees is subject to shaking and other effects due to this movement. On the other hand, the remote control environment is stationary, and there is a discrepancy between the operator's visual perception and the operator's physical sensation. This can lead to visually induced motion sickness (simulator sickness), and eliminating visually induced motion sickness has become an issue (see, for example, Patent Document 1). Furthermore, motion sickness can also be induced when an operator is riding on a work machine. Symptoms such as motion sickness and visually induced motion sickness are called "motion sickness." In other words, the term motion sickness also includes pseudo-motion sickness, which does not involve actual motion sickness. The SSQ (Simulator Sickness Questionnaire), which is a subjective evaluation method, is known as a method for assessing the degree of motion sickness. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-107567 Summary of the Invention [Problem to be solved by the invention]

[0004] However, no method has been established for objectively assessing the degree of motion sickness. An object of the present disclosure is to provide an assessment device and an assessment method that can assess the degree of motion sickness. [Means for solving the problem]

[0005] According to one aspect of the present invention, the evaluation device includes an acquisition unit that acquires an activity index value that indicates the degree of activity of the user's sympathetic or parasympathetic nerves, calculated from the user's vital data, and an evaluation unit that evaluates the degree of motion sickness of the user based on the distribution of the activity index value over a time series. [Effects of the Invention]

[0006] According to the above aspect, the evaluation device can evaluate the degree of motion sickness. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a schematic diagram illustrating a configuration of a remote control system according to a first embodiment. [Figure 2] 1 is an external view of a work machine according to a first embodiment. [Figure 3] FIG. 2 is a schematic block diagram showing the configuration of a control device of the remote control device according to the first embodiment. [Figure 4] 4 is a flowchart showing a motion sickness evaluation method using a remote control device according to the first embodiment. [Figure 5] FIG. 10 is a diagram showing the relationship between φrange and SSQ factor scores, a correlation of which was confirmed in the experiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] First Embodiment "system" FIG. 1 is a schematic diagram showing the configuration of a remote control system according to the first embodiment. The remote operation system 1 comprises a work machine 100 that is operated by remote control, and a remote operation device 500. The work machine 100 is provided at a work site (for example, a mine or quarry). The remote operation device 500 is provided in a remote operation room at the work site or at a location away from the work site (for example, in a city or within the work site). The work machine 100 and remote operation device 500 are connected via a network such as the Internet. The remote control system 1 is a system for operating a work machine 100 using a remote control device 500.

[0009] The work machine 100 operates in accordance with the operation signal received from the remote control device 500 . Operation signals for operating the work machine, turning, traveling, etc. are generated by the operator O operating the levers and pedals of the operation device 530 of the remote control device 500. The generated operation signals are transmitted to the work machine 100. The operator O who drives the work machine 100 is an example of a user.

[0010] Work Machine FIG. 2 is an external view of the work machine according to the first embodiment. The work machine 100 according to the first embodiment is a hydraulic excavator. However, the work machine 100 may be a work machine other than a hydraulic excavator, such as a wheel loader or a bulldozer. The work machine 100 comprises a hydraulically operated work implement 110, a rotating body 120 that supports the work implement 110, and a running body 130 that supports the rotating body 120. The running body 130 is, for example, a crawler track.

[0011] The revolving unit 120 is provided with a cab 121. An imaging device 122 is provided above the cab 121. The imaging device 122 is installed at the front and top of the cab 121. The imaging device 122 captures an image (e.g., a moving image) in front of the cab 121 through a windshield in front of the cab 121. Examples of the imaging device 122 include imaging devices using a CCD (Charge Coupled Device) sensor and a CMOS (Complementary Metal Oxide Semiconductor) sensor. Note that the imaging device 122 does not necessarily have to be provided in the cab 121, as long as the imaging device 122 is provided in a position where it can capture images of at least the work target of the revolving unit 120 and the work machine 110. For example, the imaging device 122 may be provided outside the cab 121, or may be provided on the revolving unit, for example. Furthermore, the imaging device 122 may be provided outside the work machine 100 , that is, may be provided in a location separate from the work machine 100 .

[0012] The work machine 100 is equipped with a control device 125. The control device 125 receives operation signals from the remote operation device 500 via an interface 640 (see FIG. 3). The control device 125 drives the work implement 110, the revolving unit 120, or the traveling unit 130 in accordance with the received operation signals.

[0013] Remote Control Device As shown in FIG. 1, the remote control device 500 includes a driver's seat 510, a display device 520, an operation device 530, a control device 540, and a vital sensor 550. The display device 520 is disposed in front of the driver's seat 510. The display device 520 is located in front of the operator O when the operator O sits in the driver's seat 510. As shown in FIG. 1 , the display device 520 is configured by a left display 521, an upper display 522, a center display 523, a lower display 524, and a right display 525 arranged in a row. Note that the number of displays constituting the display device 520 is not limited to this. For example, the display device 520 may be configured by a plurality of displays arranged in a row as shown in FIG. 1 , or may be configured by a single large display. Furthermore, the display device 520 may project an image onto a flat, curved, or spherical surface using a projector or the like, or may be worn by the operator O, such as an HMD (head-mounted display).

[0014] The operating device 530 is disposed near the driver's seat 510. The operating device 530 is located within an operable range of the operator O when the operator O sits in the driver's seat 510. The operating device 530 includes a rotation lever for rotating the rotating body 120. The operating device 530 includes, for example, an electric lever and an electric pedal.

[0015] The control device 540 displays the image received from the work machine 100 on the display device 520, and transmits an operation signal representing the operation of the operating device 530 to the work machine 100. The control device 540 acquires vital data of the operator O while operating the work machine 100 from the vital sensor 550, and evaluates the degree of motion sickness of the operator O. The control device 540 displays the motion sickness evaluation result on the display device 520. For example, the control device 540 may output the motion sickness evaluation result as a warning sound. The control device 540 is an example of an evaluation device.

[0016] The vital sensor 550 is attached to the body of the operator O and measures vital data of the operator O. The vital sensor 550 according to the first embodiment is an electrocardiograph, and outputs an electrocardiogram waveform as vital data.

[0017] 3 is a schematic block diagram showing the configuration of a control device 540 of a remote operation device 500 according to the first embodiment. The control device 540 of the remote operation device 500 is a computer comprising a processor 610, a main memory 620, a storage 630, and an interface 640. The processor 610 reads a program recorded in the storage 630 from the storage 630, deploys it in the main memory 620, and executes processing in accordance with the program. The control device 540 is connected to a network via the interface 640. The interface 640 receives image data from the work machine 100 and transmits operation signals to the work machine 100.

[0018] The storage 630 has a storage area. Examples of the storage 630 include an HDD, an SSD, a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, and a semiconductor memory. The storage 630 may be an internal medium directly connected to a common communication line of the control device 540, or may be an external medium connected to the control device 540 via an interface. The storage 630 is a non-transitory tangible storage medium.

[0019] The vital sensor 550 may be a wearable terminal. The vital sensor 550 is connected to the control device 540 via an interface 640. The connection between the vital sensor 550 and the interface 640 may be wired or wireless.

[0020] The processor 610 includes an image receiving unit 611, a display control unit 612, an operation signal transmitting unit 613, an acquiring unit 614, and an evaluating unit 615 by executing a program.

[0021] The image receiving unit 611 receives image data from the work machine 100 via the network. The display control unit 612 causes the display device 520 to display the image received by the image receiving unit 611 . The operation signal transmission unit 613 generates an operation signal for operating the work machine 100 based on the amount of operation of the operation device 530, and transmits the operation signal to the work machine 100 via the network.

[0022] The acquiring unit 614 acquires an activity index value indicating the degree of activity of the sympathetic or parasympathetic nerves from the vital data measured by the vital sensor 550. When the vital data is an electrocardiogram waveform, the activity index value may be the m index of a Lorenz plot, the S index of a Lorenz plot, CVrr, or LF / HF. The m index of a Lorenz plot, the S index of a Lorenz plot, and CVrr are examples of activity index values ​​indicating the degree of activity of the parasympathetic nerves. LF / HF is an example of an activity index value indicating the degree of activity of the sympathetic nerves.

[0023] The acquisition unit 614 calculates the time series of the activity index values ​​in the following procedure. The acquisition unit 614 acquires vital data from the vital sensor 550. The acquisition unit 614 acquires the vital data according to a predetermined measurement cycle, thereby obtaining a time series of the vital data. The acquisition unit 614 records the acquired vital data in the storage 630 in association with the acquisition time.

[0024] The acquisition unit 614 extracts vital data relating to a most recent predetermined period from the time series of vital data. The extracted vital data is called a partial time series. The acquisition unit 614 extracts the occurrence timing of R waves from the acquired partial time series of vital data and calculates the occurrence interval RRI for each R wave. The acquisition unit 614 calculates CVrr by dividing the standard deviation of the occurrence interval RRI by the average value of the occurrence interval RRI and multiplying the result by 100. The acquisition unit 614 performs frequency analysis on the acquired partial time series of vital data and identifies the high-frequency power spectral density as HF and the low-frequency power spectral density as LF from the spectrum related to two peaks. The acquisition unit 614 obtains LF / HF by calculating the ratio between HF and LF. The acquisition unit 614 generates a Lorenz plot graph by plotting the occurrence interval RRI of R waves on (x, y) = (RRI(n), RRI(n+1)). The acquisition unit 614 projects all the points on the Lorenz plot onto the y=x axis and the y=-x axis, and calculates the standard deviation σ x、 y=standard deviation σ on the -x axis yThe acquisition unit 614 calculates (2σ x ×2σ y × π) as the S parameter of the Lorenz plot. The acquisition unit 614 calculates the average value of the distance from the origin of each plot on the y=x axis as the m parameter of the Lorenz plot. The acquisition unit 614 records the calculated activity index value in the storage 630 in association with the time.

[0025] The evaluation unit 615 identifies the maximum and minimum values ​​from the time series of activity index values ​​acquired by the acquisition unit 614, and calculates the difference (range) between the maximum and minimum values. The evaluation unit 615 evaluates the degree of motion sickness based on the range of activity index values. The evaluation unit 615 calculates an evaluation value of motion sickness from the range of activity index values ​​based on a relational expression prepared in advance that indicates the relationship between the range of activity index values ​​and an evaluation value that indicates the degree of motion sickness. The evaluation unit 615 determines whether the calculated evaluation value exceeds a threshold. If the evaluation value exceeds the threshold, the evaluation unit 615 causes the display device 520 to display a screen that prompts the operator O to take a break.

[0026] <<Operation of the remote control device 500>> 4 is a flowchart showing a method for evaluating motion sickness using the remote control device 500 according to the first embodiment. When the remote control device 500 starts remotely controlling the work machine 100, the display control unit 612 starts displaying the image data received by the image receiving unit 611 on the display device 520, and the operation signal transmission unit 613 starts transmitting an operation signal based on the operation of the operation device 530 (step S1). For example, the operator O starts remotely controlling the work machine 100 while viewing the display device 520, which displays an image captured by the imaging device 122 mounted on the work machine 100.

[0027] The acquiring unit 614 acquires vital data from the vital sensor 550 (step S2). For example, the acquiring unit 614 acquires vital data of the operator O while viewing the display device 520. The acquiring unit 614 records the acquired vital data in the storage 630 in association with the acquisition time. Next, the acquiring unit 614 extracts a partial time series relating to the most recent predetermined period from the time series of the vital data from the storage 630, and calculates an activity index value by analyzing the partial time series (step S3). For example, the acquiring unit 614 calculates the m parameter of a Lorenz plot. The acquiring unit 614 records the calculated activity index value in the storage 630 in association with the time.

[0028] The evaluation unit 615 identifies the maximum and minimum values ​​from the time series of activity index values ​​recorded in the storage 630 and calculates a range (step S4). The evaluation unit 615 calculates the range from the time of step S1 to the current time. The evaluation unit 615 calculates an evaluation value of motion sickness from the range of activity index values ​​in accordance with a pre-prepared relational expression (step S5). The evaluation value of motion sickness is, for example, a value corresponding to the SSQ-O score, which indicates the level of visual fatigue caused by motion sickness. Note that, as a result of experiments described below, it has been found that the larger the m parameter of the Lorenz plot, the higher the SSQ-O score, and the relational expression may be, for example, an approximation of this relationship to a linear function. A higher evaluation value of motion sickness indicates a stronger degree of motion sickness.

[0029] The evaluation unit 615 determines whether the calculated evaluation value exceeds a threshold value (step S6). The threshold value for the evaluation value may be, for example, a threshold value generally used in SSQ to determine whether or not the operator is suffering from motion sickness, or may be a threshold value indicating the degree to which operation of the work machine 100 is impaired. If the evaluation value exceeds the threshold value (step S6: YES), the evaluation unit 615 causes the display device 520 to display a warning notification screen urging the operator O to take a break (step S7). For example, if the evaluation value exceeds the threshold value, the evaluation unit 615 may notify the manager of the operator O that the operator O is suffering from motion sickness. If the operator O continues remote operation despite the display device 520 displaying a warning notification screen urging the operator O to take a break, the evaluation unit 615 may contact the manager of the operator O. If the evaluation value does not exceed the threshold value (step S6: NO), the evaluation unit 615 does not issue a warning.

[0030] Next, the control device 540 determines whether the operator O ends the remote driving (step S8). If the operator O does not end the remote driving (step S8: NO), the process returns to step S2 and the evaluation of motion sickness continues. If the operator O ends the remote driving (step S8: YES), the control device 540 ends the evaluation of motion sickness.

[0031] Through the above procedure, the control device 540 can evaluate the degree of motion sickness.

[0032] <Relationship between the range of activity index values ​​and the evaluation value of motion sickness> Here, the reason why the evaluation value of motion sickness can be calculated from the range of the activity index value will be explained. The inventors conducted the following experiment using participants who had no experience in remotely operating the work machine 100. Before remote driving, each participant completed a subjective assessment of motion sickness using the SSQ. The SSQ consists of 16 questions, and by calculating the weighted sum of the responses to each question, four factors—visual fatigue (SSQ-O: oculomotor), dizziness (SSQ-D: disorientation), nausea (SSQ-N: nausea), and a total score (SSQ-TS: total score)—were calculated. Next, participants were asked to confirm the remote driving operation method, and then practiced using a sample program under conditions where there was no delay until the results of the remote control operation were visually reflected. They then practiced remote driving until they successfully completed a routine from excavation to soil removal in under one minute. The experiment was conducted using a video device that projected stereoscopic images onto the front, left, right, and floor surfaces. Participants wore polarized glasses for stereoscopic vision and an electrocardiogram (ECG) to perform the remote driving task. The task consisted of six five-minute sessions of remote operation of a hydraulic excavator. Electrocardiograms were recorded during each session. The ECG waveforms were saved after each session. After completing the remote driving, participants were asked to subjectively evaluate their motion sickness using the SSQ.

[0033] The inventors fitted a tenth-order polynomial to the ECG waveforms obtained from each session, removed trends, and then detected R waves. The inventors calculated the RRI from the detected R waves, and then resampled the RRI for one second. Using this, they calculated the activity index values: CVrr, LF / HF, the m index of the Lorenz plot, and the S index of the Lorenz plot. The inventors removed outliers from the activity index values ​​for multiple sessions of multiple participants.

[0034] From the calculation results of the SSQ factor scores, the inventors confirmed that the scores of each SSQ factor increased after the remote control experiment. Considering that previous research had shown results in which the correspondence between sickness and time-series changes in autonomic nervous activity was unclear, the inventors conducted an analysis from a new perspective, wondering whether there might be a correlation between the range of each individual's activity index value and subjective symptoms during participation in the experiment. First, the difference φ between the maximum and minimum values ​​of each participant's activity index value was calculated. rangeThe inventors calculated φ by subtracting the later value of the maximum and minimum values ​​from the earlier value, in order to take into account the order in which the maximum and minimum values ​​appear. range The inventors calculated the scores of each factor of the SSQ and φ range The correlation coefficients (Pearson's linear correlation coefficient r, Kendall's τ coefficient, and Spearman's ρ coefficient) were calculated. Note that the inventors targeted the data in the top 50% of SSQ scores when calculating the correlation coefficients. This is because the data of participants with lower SSQ scores who did not experience motion sickness in the first place was included. As a result of the verification, for each activity index value, the range φ of the activity index value range A correlation was found between the scores of the SSQ factors and the φ. range This figure shows the relationship between the scores of the SSQ factors and the score of the SSQ factors. From these results, the inventors have found that it is possible to evaluate the degree of motion sickness from the range of the activity index value.

[0035] Actions and Effects As described above, according to the first embodiment, the control device 125 includes an acquisition unit 614 that acquires an activity index value that indicates the degree of activity of the sympathetic or parasympathetic nerves, calculated from the vital data of the operator O, and an evaluation unit 615 that evaluates the level of motion sickness of the user based on the time-series range of the activity index value. The above-mentioned experiment has shown that the level of motion sickness can be evaluated from the range of the activity index value. Therefore, the control device 125 can evaluate the level of motion sickness.

[0036] Other Embodiments Although one embodiment has been described in detail above with reference to the drawings, the specific configuration is not limited to the above, and various design modifications are possible. That is, in other embodiments, the order of the above-described processes may be changed as appropriate. Furthermore, some processes may be executed in parallel. The control device 540 according to the above-described embodiment may be configured by a single computer, or the configuration of the control device 540 may be divided into multiple computers that cooperate with each other to function as the control device 540. For example, the vital sensor 550 may calculate the activity index value, and the control device 540 may acquire the calculated activity index value from the vital sensor 550.

[0037] The control device 540 according to the embodiment described above evaluates motion sickness of the operator O who uses the remote operation system 1, but is not limited to this. For example, in other embodiments, the evaluation device may evaluate motion sickness (motion sickness) of the operator O who is on board the manned work machine 100. Furthermore, in other embodiments, the evaluation device may evaluate other types of motion sickness.

[0038] The control device 540 according to the embodiment described above calculates an evaluation value of motion sickness from the range of the activity index value and determines whether to issue a warning based on the evaluation value, but this is not limited to this. For example, the control device 540 according to another embodiment may determine whether to issue a warning directly from the range of the activity index value. Furthermore, the control device 540 according to another embodiment may evaluate motion sickness based on other distribution measures such as standard deviation, variance, or interquartile range, rather than the range of the activity index value. [Explanation of symbols]

[0039] 1...Remote control system 100...Work machine 110...Work equipment 120...Swing unit 121...Driver's cab 122...Imaging device 125...Control device 130...Traveling body 500...Remote control device 510...Driver's seat 520...Display device 521...Left display 522...Upper display 523...Central display 524...Lower display 525...Right display 530...Operation device 540...Control device 550...Vital sensor 610...Processor 611...Image receiving unit 612...Display control unit 613...Operation signal transmitting unit 614...Acquisition unit 615...Evaluation unit 620...Main memory 630...Storage 640...Interface O...Operator

Claims

1. an acquisition unit that acquires an activity index value that indicates the degree of activity of the sympathetic nerve or the parasympathetic nerve of the user, the activity index value being calculated from the vital data of the user; an evaluation unit that evaluates the degree of motion sickness of the user based on a distribution degree of the time series of the activity index value; An evaluation device comprising:

2. the user is an operator who drives a work machine, The acquisition unit acquires the activity index value calculated from vital data of the user while the work machine is being operated. The evaluation device according to claim 1 .

3. The user is an operator who remotely operates the work machine while visually checking a display that displays an image captured by an imaging device mounted on the work machine, The acquisition unit acquires the activity index value calculated from vital data of the user while the user is viewing the display. The evaluation device according to claim 2 .

4. the acquiring unit acquires vital data of the user and calculates the activity index value from the acquired vital data. The evaluation device according to claim 1 .

5. The activity index value is the m index of a Lorenz plot, the S index of a Lorenz plot, CVrr, or LF / HF; The evaluation device according to claim 1 .

6. the evaluation unit evaluates the degree of motion sickness of the user based on a difference between a maximum value and a minimum value in the time series of the activity index value. The evaluation device according to claim 1 .

7. The evaluation unit determines whether an evaluation value indicating the degree of motion sickness of the user exceeds a threshold, and outputs an evaluation result if the evaluation value exceeds the threshold. The evaluation device according to claim 6 .

8. calculating an activity index value indicating the degree of activity of the user's sympathetic or parasympathetic nerves from the user's vital data; evaluating the degree of motion sickness of the user based on a distribution degree of the time series of the activity index value; An evaluation method comprising:

Images