Data processing device and program

Abstract

To improve accuracy for estimating stress felt by a person.SOLUTION: A data processing device 1 comprises: a storage unit 12 which stores stress state data indicating a feature of a walking state in such a state that a user feels stress in association with each of the plurality of users; an image data acquisition unit 131 which acquires a plurality of pieces of photographed image data generated by a plurality of imaging devices; a detection unit 132 which specifies a position of the user; a selection unit 133 which selects one or more pieces of photographed image data corresponding to the position of the user from the plurality of pieces of photographed image data; a state specification unit 135 which specifies the feature of the walking state of the user on the basis of the image of the user included in the one or more pieces of photographed image data corresponding to the positions of the plurality of users; and an output unit 136 which outputs a result of determining whether the user feels stress on the basis of a result of comparing the feature indicated by stress state data stored in association with the user in the storage unit 12 with the feature specified by the state specification unit 135.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a data processing device and a program.

Background Art

[0002] Conventionally, a mental state estimation system that estimates the stress felt by a subject based on an image captured by a camera has been known (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional mental state estimation system, since the stress felt by a person was estimated based on an image captured by a single camera, there was a problem that the stress could not be estimated when the person was not within the imaging range of the camera.

[0005] Therefore, the present invention has been made in view of these points, and an object thereof is to improve the accuracy of estimating the stress felt by a person.

Means for Solving the Problems

[0006] A data processing device according to a first aspect of the present invention includes: a storage unit that stores stress state data indicating the characteristics of a user's walking state when they are experiencing stress, associated with each of a plurality of users walking in a predetermined space; an image data acquisition unit that acquires a plurality of image data generated by a plurality of imaging devices installed in the predetermined space; a detection unit that identifies the position of the user in the predetermined space; a selection unit that selects one or more image data corresponding to the user's position from the plurality of image data; a state identification unit that identifies the characteristics of the user's walking state based on the image of the user included in the one or more image data corresponding to the position of the plurality of users selected by the selection unit; and an output unit that outputs a result of determining whether or not the user is experiencing stress, based on a comparison between the characteristics indicated by the stress state data stored in the storage unit in association with the user and the characteristics identified by the state identification unit.

[0007] The storage unit stores the stress state data in association with the installation locations of the plurality of imaging devices, and the output unit may determine whether or not the user is experiencing stress by using the characteristics indicated by the stress state data stored in the storage unit in association with the installation locations of the imaging devices that generated the image data used by the state identification unit to identify the characteristics of the user's walking state.

[0008] The system may further include a declaration data acquisition unit that acquires data from the user indicating that the user is experiencing stress, and a data creation unit that creates stress state data indicating the characteristics of the user's walking state as shown in the image data in which the user is included at the time the declaration data acquisition unit acquires the data.

[0009] The system may further include a measurement data acquisition unit that acquires measurement data of the user's pulse wave, and a data creation unit that creates stress state data indicating the characteristics of the user's walking state as shown in the imaged image data in which the user is included at the point when the pulse wave shown in the measurement data acquired by the measurement data acquisition unit indicates that the user is feeling stressed.

[0010] A program according to a second aspect of the present invention causes a computer to function as: an image data acquisition unit that acquires a plurality of image data generated by a plurality of imaging devices installed in a predetermined space; a detection unit that identifies the position of a user in the predetermined space; a selection unit that selects one or more image data corresponding to the position of the user from the plurality of image data; a state identification unit that identifies the characteristics of the user's walking state based on the image of the user included in the one or more image data corresponding to the user's position selected by the selection unit; and an output unit that outputs a result of determining whether or not the user is in a stressed state based on a comparison between the characteristics of the walking state in which the user is feeling stressed, which are stored in a storage unit in association with the user, and the characteristics identified by the state identification unit. [Effects of the Invention]

[0011] The present invention has the effect of improving the accuracy of estimating the stress felt by a person. [Brief explanation of the drawing]

[0012] [Figure 1] This is a diagram illustrating the overview of the data processing system 100 according to this embodiment. [Figure 2] This is a schematic diagram showing the layout of room S. [Figure 3] This diagram shows the addresses of multiple areas within Room S. [Figure 4] This is a diagram showing the configuration of the data processing device 1. [Figure 5] This figure shows an example of area data. [Figure 6] This is a diagram showing an example of camera position data. [Figure 7] This is a diagram showing an example of first detection data. [Figure 8] This is a diagram showing an example of second detection data. [Figure 9] This is a diagram showing an example of stress state data. [Figure 10] This is a diagram for explaining the method by which the distance image creation unit 134 creates distance image data. [Figure 11] This is a diagram showing an example of reported data. [Figure 12] This is a diagram showing an example of measurement data of a pulse wave. [Figure 13] This is a flowchart showing the processing flow in the data processing device 1.

Mode for Carrying Out the Invention

[0013] [Overview of Data Processing System 100] FIG. 1 is a diagram for explaining the overview of the data processing system 100 according to the present embodiment. The data processing system 100 is a system for determining whether a person is in a state of feeling stress, and is used, for example, to grasp whether a person (hereinafter sometimes referred to as "employee M") in a room S inside an office or a factory is feeling stress. In the present embodiment, the case where the data processing system 100 determines whether the employee M in the room S is feeling stress is exemplified, but the data processing system 100 may be a system for determining whether a person in another location is feeling stress.

[0014] In the example shown in FIG. 1, a plurality of cameras C, which are imaging devices, are installed in the room S, and the plurality of cameras C generate imaging image data by photographing the employee M in the space. The viewing angles of the plurality of cameras C are arbitrary, but in the present embodiment, it is assumed that the camera C is a camera capable of photographing in all 360 degrees directions.

[0015] A plurality of cameras C transmit captured image data to the data processing device 1 via a communication line. The communication line may be wired or wireless. In the present embodiment, the case where the captured image data is moving image data is exemplified, but the captured image data may be still image data.

[0016] The data processing device 1 is a device for determining whether a person included in the captured image data captured by the camera C is feeling stress, and is, for example, a computer. The data processing device 1 is, for example, a computer installed in an office or a factory, but may also be a computer or a server installed at a location different from an office or a factory.

[0017] FIG. 2 is a schematic diagram showing the layout of the room S. FIG. 3 is a diagram showing the addresses of each of a plurality of areas in the room S. The addresses of the plurality of areas are area identification information (hereinafter referred to as "area ID") represented by a combination of horizontal addresses x1 to x8 and vertical addresses y1 to y6. For example, the address of the upper left area is represented as (x1, y1). The dashed arrow shown in FIG. 3 indicates the path along which the employee M1 has moved.

[0018] In the room S, a plurality of desks T (desks T11 to T34) are installed. Above desks T11 to T22, cameras C11 to C15 are installed, and above desks T23 to T34, cameras C21 to C25 are installed. C11 to C15 and C21 to C25 are used as camera identification information (hereinafter referred to as "camera ID") for identifying the camera C.

[0019] The camera C may be installed above the passage around the plurality of desks T. The plurality of cameras C generate different captured image data by photographing within the range of their respective fields of view. Employees (in FIG. 2, M1 to M5) staying in the room S can move freely along the passage.

[0020] The data processing device 1 uses one or more image data selected from multiple image data generated by multiple cameras C to identify the walking state of employee M in room S. The walking state is, for example, walking speed or stride length. The data processing device 1 determines whether or not employee M is experiencing stress by comparing the identified walking state with the walking state of employee M when she is experiencing stress, which is stored in the memory of the data processing device 1.

[0021] The data processing device 1 can select image data including employee M from multiple image data generated by multiple cameras C, so it can identify employee M's walking state regardless of which area of ​​room S employee M is in, and determine whether or not employee M is experiencing stress. The configuration and operation of the data processing device 1 will be described in detail below.

[0022] [Configuration of Data Processing Unit 1] Figure 4 shows the configuration of the data processing device 1. The data processing device 1 includes a communication unit 11, a storage unit 12, and a control unit 13.

[0023] The communication unit 11 is an interface for sending and receiving data with external devices, and includes, for example, a LAN (Local Area Network) controller or a wireless communication controller. The communication unit 11 receives captured image data from multiple cameras C.

[0024] The storage unit 12 has a storage medium such as ROM (Read Only Memory), RAM (Random Access Memory), or SSD (Solid State Drive). The storage unit 12 stores the program executed by the control unit 13. The storage unit 12 also stores multiple image data received by the communication unit 11 and data used to perform various processes based on the image data.

[0025] The storage unit 12 includes a program storage unit 121, an area data storage unit 122, a detection data storage unit 123, and a stress state data storage unit 124. The program storage unit 121 stores the program to be executed by the control unit 13. The area data storage unit 122 stores area data, in which each of the multiple area IDs is associated with a camera C, and camera position data indicating the location of each of the multiple camera Cs.

[0026] Figure 5 shows an example of area data. In the area data shown in Figure 5, the area address (i.e., area ID) is associated with the camera ID. For an area where multiple cameras C can capture images, the camera IDs of the multiple cameras C are associated with the address. Figure 6 shows an example of camera position data. In the camera position data shown in Figure 6, the camera ID is associated with the installation position, which indicates the coordinates or other location where the camera C corresponding to that camera ID is installed.

[0027] The detection data storage unit 123 stores first detection data, which associates the detected employee M with multiple image data sets in which the detected employee M exists. For example, for each identical employee identified by the detection unit 132 (described later), the detection data storage unit 123 stores the image data containing that employee in chronological order, in the order of the date and time the image data was generated.

[0028] Figure 7 shows an example of first detection data. Figure 7 shows first detection data D01, D02, and D03 corresponding to three employees (for example, employees M1, M2, and M3) detected in the captured image data, but the detection data storage unit 123 may store multiple first detection data corresponding to more employees.

[0029] In the first detection data shown in Figure 7, the start and end dates and times of the time in which employee M is included in each image data, the image data name of the image data, and the camera ID of the camera C that generated the image data are associated with the order in which the image data containing employee M corresponding to each first detection data was generated. The image data name is the file name of the image data associated with an address in the detection data storage unit 123, and the control unit 13 can access the image data by specifying the image data name.

[0030] The first detection data shown in Figure 7 corresponds to multiple image data captured while employee M1, located in room S as shown in Figure 3, moves along the path indicated by the dashed line from the moment he enters room S to the address (x4, y3). Here, it is assumed that employee M1 is moving addresses every second.

[0031] As shown in the area data in Figure 5, while employee M1 is in room S and is at addresses (x1, y1) and (x1, y2), only camera C11 can photograph employee M1. Therefore, for the first two seconds (13:45:00 to 13:45:02), only camera C11 photographs employee M1. After that, when employee M1 moves to address (x1, y3), camera C21 also becomes able to photograph employee M1, and between 13:45:02 and 13:45:05, C21 also photographs employee M1.

[0032] Subsequently, between 13:45:03 and 13:45:06, when employee M1 is at addresses (x2, y3) to (x4, y3), cameras C12 and C22 photograph employee M1. When employee M1 reaches address (x3, y3) at 13:45:04, cameras C13 and C23 also photograph employee M1. In this way, the first detection data contains multiple image data of employee M1 recorded in chronological order, so the data processing device 1 can efficiently determine whether or not employee M1 is experiencing stress.

[0033] Furthermore, the detection data storage unit 123 stores a second detection data associated with multiple distance image data in which a person has been detected in the distance image data. Distance image data is data created by the distance image creation unit 134, described later, using two image capture data and the distance between the two cameras C as parallax. Distance image data includes depth information (i.e., information indicating distance in the depth direction). For example, for each employee M, the detection data storage unit 123 stores multiple distance image data in which that employee M is pictured in chronological order, in the order of the dates and times when the two image capture data used to create the distance image data were generated.

[0034] Figure 8 shows an example of second detection data. Figure 8 shows second detection data D11, D12, and D13 corresponding to three employees (for example, employees M1, M2, and M3) detected in the captured image data, but the detection data storage unit 123 may store multiple second detection data corresponding to more employees.

[0035] In the second detection data shown in Figure 8, the start and end dates and times of the time in which employee M is included in the distance image data generated based on the captured image data, the image data name of the distance image data, and the camera ID of the camera that generated the captured image data used to create the distance image data are associated with the order in which the captured image data containing employee M corresponding to each second detection data was generated. The image data name of the distance image data includes the camera IDs of the multiple cameras C that generated the two captured image data used to create the distance image data.

[0036] Since two image capture data are used to create the distance image data, two camera IDs are associated with the distance image data in the second detection data. However, between 13:45:00 and 13:45:02, employee M1 is only visible in the image capture data generated by camera C11, so during this period, only the image capture data generated by camera C11 is recorded, not the distance image data.

[0037] As employee M1 moves from address (x1, y3) to (x4, y3), employee M1 is photographed by multiple cameras C, and distance image data created from two captured image data is recorded. During this time, if employee M1 is photographed by three or more cameras C, it is possible to create two or more distance image data.

[0038] Therefore, for example, between 13:45:03 and 13:45:05, four distance image data sets are created based on four pairs of captured image data corresponding to the combinations of camera C11 and camera C21, camera C11 and camera C12, camera C21 and camera C22, and camera C12 and camera C22.

[0039] The stress state data storage unit 124 stores stress state data, which is reference data for determining whether or not employee M is experiencing stress. The stress state data is data that indicates the characteristics of a person's walking state when they are experiencing stress, which has been identified in advance. For example, the stress state data is created based on observations of the walking state of a person who is experiencing stress.

[0040] The stress state data storage unit 124 may store stress state data that indicates the characteristics of the walking state of an employee M when they are feeling stressed, associated with each of the multiple employees M walking in a predetermined space. In this case, the stress state data is, for example, data created based on observations of the state of a person who has the same attributes as employee M and is feeling stressed when walking in the predetermined space. The stress state data may also be data created based on observations of the state of employee M when they themselves are feeling stressed when walking in the predetermined space.

[0041] As described above, the predetermined space in this embodiment is, for example, the space inside room S. The characteristics of the walking state indicate at least one of the employee M's stride length, walking speed, or the direction of their gaze while walking. In this embodiment, walking speed is used as the characteristic of the walking state.

[0042] The stress state data storage unit 124 may store stress state data in association with the installation locations of multiple cameras C. In this case, the stress state data is, for example, data created based on observations of the state of a person experiencing stress as they walk around each of the installation locations of multiple cameras C.

[0043] Figure 9 shows an example of stress state data. Figure 9 shows stress state data D21, D22, and D23 corresponding to three employees (e.g., employees M1, M2, and M3) detected in the captured image data, but the stress state data storage unit 124 may store multiple stress state data corresponding to more employees. In the stress state data shown in Figure 9, the installation location of each of the multiple cameras C is associated with the walking speed conditions corresponding to the state in which employee M is experiencing stress in the area captured by the camera C installed at the installation location.

[0044] Specifically, when employee M1 is walking at address (x1, y2), the camera C11 that is imaging employee M1 is installed at installation position P11, as shown in the area data in Figure 5 and the camera position data in Figure 6. In this case, the walking speed condition when employee M1 is experiencing stress is "less than 0.8 m / s" associated with installation position P11.

[0045] The control unit 13 functions as at least one of the following by executing a program stored in the program storage unit 121: the image data acquisition unit 131, the detection unit 132, the selection unit 133, the distance image creation unit 134, the state identification unit 135, the output unit 136, the recording processing unit 137, the declared data acquisition unit 138, the measurement data acquisition unit 139, and the data creation unit 140. The control unit 13 may perform the functions of all of these components, or it may perform the functions of some of these components.

[0046] The image data acquisition unit 131 acquires multiple captured image data generated by multiple cameras C installed in the room S. The captured image data acquired by the image data acquisition unit 131 may be moving image data or still image data. The image data acquisition unit 131 inputs the acquired multiple captured image data to the detection unit 132. The image data acquisition unit 131 may also store the multiple captured image data in the detection data storage unit 123, associating them with the camera ID of the camera C that transmitted the captured image data.

[0047] The detection unit 132 identifies the location of employee M within room S. Based on multiple captured image data, the detection unit 132 detects which of the multiple areas within room S a person is staying in and identifies which employee M the detected person is.

[0048] The method by which the detection unit 132 identifies the location of employee M is arbitrary, but the detection unit 132 identifies the location of employee M based on, for example, captured image data. Specifically, the detection unit 132 detects which of the multiple areas in room S employee M is staying in by identifying the area associated with the camera C that generated the captured image data including employee M in the area data storage unit 122.

[0049] In this case, the detection unit 132 first identifies contour lines in multiple captured image data input from the image data acquisition unit 131 where the change in brightness value is greater than or equal to a threshold. Then, the detection unit 132 identifies the captured image data containing the image of employee M because the shape of the contour line is similar to the contour lines of humans previously stored in the storage unit 12. The detection unit 132 then detects the position of employee M by referring to the area data stored in the area data storage unit 122 and identifying the position of the camera C that transmitted the captured image data containing the image of employee M.

[0050] The detection unit 132 detects the location of employee M and then identifies the employee ID corresponding to employee M. The detection unit 132 identifies the employee ID of employee M in the captured image data by, for example, comparing the image of employee M's face included in the captured image data with images of employee M's face stored in the storage unit 12, each associated with a plurality of employee IDs. The detection unit 132 notifies the selection unit 133 of the detected location of employee M and the identified employee ID.

[0051] The selection unit 133 selects one or more image data corresponding to the location of employee M from a plurality of image data based on the location of employee M detected by the detection unit 132. The selection unit 133 selects one or more image data generated by one or more cameras C stored in the area data storage unit 122, associated with the area containing the location of employee M detected by the detection unit 132.

[0052] Specifically, the selection unit 133 refers to the area data shown in Figure 5 and selects one or more cameras C corresponding to the address of the location where the detection unit 132 detected employee M, and selects one or more captured image data generated by the selected one or more cameras C. The selection unit 133 inputs the selected one or more captured image data to the distance image creation unit 134, associating it with the employee ID identified by the detection unit 132. The selection unit 133 may also notify the distance image creation unit 134 of information for identifying the one or more captured image data (for example, image data names).

[0053] Incidentally, as the area data in Figure 5 shows, depending on the location, there may be cases where three or more cameras C can photograph employee M. In such cases, the selection unit 133 selects two image data sets generated by two imaging devices whose distance from the position of employee M detected by the detection unit 132 to the position of camera C is within a predetermined range. The predetermined range is, for example, a range set as a distance suitable for creating distance image data. The predetermined range may also be a distance range suitable for photographing the target area to be detected.

[0054] For example, the selection unit 133 selects two image data generated by the two cameras C whose positions are closest to the position of employee M detected by the detection unit 132. The selection unit 133 may also select image data generated by the two cameras C whose average or median distance to employee M is the smallest. By selecting such two image data, the selection unit 133 creates distance image data in which employee M is clearly visible, thereby improving the accuracy of identifying employee M's actions.

[0055] The selection unit 133 may also select two cameras C in which the variation in the distance between the employee M's position and the two cameras C is within a range set as a suitable distance for creating distance image data. If the variation in the distance between the employee M's position and the two cameras C is small, the size of the employee M captured by the two cameras C will be similar, thus improving the quality of the created distance image data.

[0056] The selection unit 133 may select two cameras C whose distance from each other is within a range of distances suitable for creating distance image data, which is pre-stored in the storage unit 12, that can photograph employee M. For example, if employee M is at the address (x3, y3) shown in Figure 3, the selection unit 133 will select cameras C11 and C12, or C12 and C13, etc., from among the six cameras C that can photograph this location, which are closest to each other.

[0057] The distance image creation unit 134 creates distance image data including distance information based on the two captured image data selected by the selection unit 133. Distance information is information indicating the depth or distance to the subject. In this embodiment, the distance image data is, for example, data that includes color information and distance information contained in the captured image data. The two cameras C that generated the two captured image data selected by the selection unit 133 are considered to constitute a stereo camera, and the distance image creation unit 134 creates the distance image data using the same image processing method as when a stereo camera creates distance image data.

[0058] The distance image creation unit 134 creates distance image data by calculating the distance from the straight line connecting the two cameras C to the subject, assuming that the distance between the two cameras C corresponding to the two captured image data selected by the selection unit 133 is the distance between the left and right cameras in a stereo camera. The distance image data created by the distance image creation unit 134 is stored in the detection data storage unit 123 by the recording processing unit 137, which will be described later.

[0059] Figure 10 is a diagram illustrating how the distance image creation unit 134 creates distance image data. In the example shown in Figure 10, the focal lengths of cameras C11 and C12 are F, and the baseline length (distance between cameras) is B. The distance image creation unit 134 calculates the disparity S, which is the difference between the position of employee M in the image data generated by camera C11 and the position of employee M in the image data generated by camera C12, and calculates the distance D to employee M using the formula D = B × F / S.

[0060] The state identification unit 135 identifies the walking state of employee M included in the captured image data. The state identification unit 135 identifies the walking speed of employee M based on the image of employee M included in one or more captured image data corresponding to the positions of multiple employee M selected by the selection unit 133. Since the state identification unit 135 only needs to analyze the captured image data selected by the selection unit 133, the processing load can be reduced even when identifying the walking speeds of a large number of employees M.

[0061] The state identification unit 135 identifies, for example, the difference in the time when two images were captured, and the difference in the position of employee M in each of the two images, which are contained in two image data generated at different times. Based on the identified difference in employee M's position and the identified difference in time, the state identification unit 135 identifies the walking speed of employee M.

[0062] The state identification unit 135 may identify the walking speed of employee M based on multiple images of employee M included in one or more distance image data created by the distance image creation unit 134 based on the captured image data selected by the selection unit 133. For example, the state identification unit 135 identifies the difference in the time when two images included in the distance image data were captured and the difference in the position of employee M in each of the two images, and identifies the walking speed of employee M based on the identified time difference and position difference. By identifying the walking speed of employee M using distance image data, the accuracy of identifying the walking speed when employee M is walking in a direction other than the direction perpendicular to the optical axis of camera C can be improved.

[0063] The state identification unit 135 associates the identified walking speed with the employee ID associated with the image data used to identify the walking speed and inputs it to at least one of the data creation unit 140 or output unit 136. The state identification unit 135 may also store the identified walking speed in the stress state data storage unit 124, associating it with the employee ID associated with the image data.

[0064] The output unit 136 outputs a result of determining whether or not employee M is experiencing stress, based on a comparison between the characteristics indicated by the stress state data stored in the stress state data storage unit 124 in association with employee M and the characteristics identified by the state identification unit 135. For example, the output unit 136 uses the characteristics indicated by the stress state data stored in the stress state data storage unit 124 in association with the installation location of the camera C that generated the image data used by the state identification unit 135 to identify the user's walking speed, to determine whether or not employee M is experiencing stress.

[0065] Specifically, suppose the output unit 136 obtains from the state identification unit 135 the speed at which employee M1 walked inside room S from 13:45:00 to 13:45:01, which is "0.7 m / s". The output unit 136 identifies the camera C11 that generated the image data used by the state identification unit 135 to determine employee M's walking speed by referring to the first detection data shown in Figure 7. The output unit 136 identifies the installation position P11 of camera C11 by referring to the camera position data shown in Figure 6.

[0066] The output unit 136 refers to the stress state data shown in Figure 9 and identifies a walking speed of "less than 0.8 m / s" in the area captured by the camera C11 installed at installation position P11, indicating that employee M1 is experiencing stress. The output unit 136 determines that employee M1 is experiencing stress because the walking speed identified by the state identification unit 135 is less than 0.8 m / s.

[0067] By having the output unit 136 determine whether or not employee M1 is experiencing stress according to the area in which employee M1 walks, the data processing device 1 can improve the accuracy of its determination that employee M1 is experiencing stress. Furthermore, by using stress state data that includes walking speed conditions for determining whether employee M1 is experiencing stress, corresponding to each of multiple areas, the output unit 136 can further improve the accuracy of its determination that employee M1 is experiencing stress.

[0068] The output unit 136 outputs the result of the determination to at least one of the following: the information terminal used by employee M, the information terminal used by the personnel department of the company where employee M works, or the information terminal used by the manager who manages employee M's work.

[0069] The recording processing unit 137 stores the captured image data and distance image data in the detection data storage unit 123. The recording processing unit 137 creates the first detection data shown in Figure 7 by associating multiple captured image data common to the same employee M with the employee M and storing them in the detection data storage unit 123. The recording processing unit 137 creates the second detection data shown in Figure 8 by associating multiple distance image data common to the same employee M with the employee M and storing them in the detection data storage unit 123.

[0070] [Creating stress level data] In the above description, the stress state data storage unit 124 pre-stored the stress state data, but the data processing device 1 may also create stress state data in the data creation unit 140 or update the created stress state data. The data creation unit 140 creates stress state data based on the characteristics of employee M's walking state identified by the state identification unit 135, the declared data acquired by the declared data acquisition unit 138, and the measured data acquired by the measured data acquisition unit 139. The operation of the declaration data acquisition unit 138, the measurement data acquisition unit 139, and the data creation unit 140 will be described in detail below.

[0071] The declaration data acquisition unit 138 acquires data from employee M indicating that they are feeling stressed. For example, the declaration data acquisition unit 138 acquires declaration data from an information terminal (not shown) that indicates either "feeling stressed" or "not feeling stressed," which is generated by employee M operating the terminal. The declaration data acquisition unit 138 inputs the acquired declaration data into the data creation unit 140. The declaration data acquisition unit 138 may also store the acquired declaration data in the stress state data storage unit 124.

[0072] Figure 11 shows an example of declared data. In the declared data shown in Figure 11, the time when employee M declared whether or not they were feeling stressed, employee identification information to identify employee M (hereinafter referred to as "employee ID"), and the content of the declaration indicating whether or not they were feeling stressed are associated with each other. In this way, since the declared data includes the time of declaration and the content of the declaration, the data processing device 1 can efficiently identify image data of employee M that was generated at the time when employee M declared that they were feeling stressed.

[0073] The measurement data acquisition unit 139 acquires measurement data of employee M's pulse wave. The pulse wave is, for example, a fingertip volume pulse wave, a velocity pulse wave, or an acceleration pulse wave. The measurement data acquisition unit 139 acquires measurement data of the pulse wave measured at a predetermined time by a wearable terminal (not shown) worn by employee M from the wearable terminal. The measurement data acquisition unit 139 inputs the acquired pulse wave measurement data to the data creation unit 140. The measurement data acquisition unit 139 may also store the acquired pulse wave measurement data in the stress state data storage unit 124.

[0074] Figure 12 shows an example of pulse wave measurement data. In the pulse wave data shown in Figure 12, the measurement start time when the pulse wave measurement began, the measurement end time when the pulse wave measurement ended, the employee ID, and the pulse wave data name indicating the file name of the pulse wave measurement data are associated. In this way, the pulse wave measurement data contains the pulse wave of employee M at a predetermined time, so the data processing device 1 can identify the time when employee M's pulse wave changes significantly (i.e., the time when employee M is estimated to be experiencing stress) by referring to the measurement data.

[0075] The data creation unit 140 creates stress state data, which indicates the walking speed of each employee M when they are experiencing stress, and associates this data with each of the multiple employees M. The data creation unit 140 also creates stress state data indicating the walking speed of employee M as shown in the image data containing employee M at the time the declared data acquisition unit 138 acquires the declared data.

[0076] The data creation unit 140 identifies, for example, the declaration time included in the declaration data acquired from employee M by the declaration data acquisition unit 138. The data creation unit 140 acquires the installation location of camera C that generated image data of the area where employee M was walking at the identified time, and the walking speed of employee M identified by the state identification unit 135 based on multiple image data generated at the identified time. The data creation unit 140 stores stress state data, which associates the acquired camera C installation location and employee M's walking speed, in the stress state data storage unit 124, associating it with employee M's employee ID.

[0077] Specifically, as shown in Figure 11, the data creation unit 140 identifies the declaration time of 13:45:01, which is included in the declaration data that employee M1 is experiencing stress, obtained by the declaration data acquisition unit 138. The data creation unit 140 identifies the camera C11 that generated the image data including employee M1 at the identified time by referring to the first detection data shown in Figure 7. The data creation unit 140 identifies the installation location P11 of camera C11 by referring to the camera location data shown in Figure 6.

[0078] The data creation unit 140 obtains the walking speed of employee M1, identified as "0.8 m / s," from the state identification unit 135 or the storage unit 12 based on the captured image data showing employee M1 walking in the area (x1, y2) at the reported time of 13:45:01. In this case, the data creation unit 140 identifies the walking speed condition for employee M1 to be experiencing stress as "less than 0.8 m / s," and stores the stress state data, which associates the identified installation location P11 with the condition "less than 0.8 m / s," in the stress state data storage unit 124.

[0079] When the data creation unit 140 obtains multiple walking speeds from the state identification unit 135 or the storage unit 12, it identifies the largest walking speed among the obtained walking speeds and stores the stress state data using the identified walking speed as the walking speed condition in the stress state data storage unit 124.

[0080] Within room S, there are areas where the passageway is narrow and walking speed is reduced, and areas where employee M walks slowly when turning corners. In response to this, the data creation unit 140 determines walking speed conditions corresponding to the areas captured by each of the multiple cameras C installed in room S. As a result, the data processing device 1 can improve the accuracy of determining whether or not employee M is experiencing stress.

[0081] The data creation unit 140 may create stress state data that shows the characteristics of employee M's walking state as indicated by the image data containing employee M at the point when the pulse wave indicated by the measurement data acquired by the measurement data acquisition unit 139 indicates that employee M is feeling stressed. In this embodiment, the case where the pulse wave is an acceleration pulse wave (i.e., a heart rate waveform) is shown.

[0082] The data creation unit 140 determines that employee M is experiencing stress if, for example, the heart rate waveform shown in the measurement data acquired from employee M by the measurement data acquisition unit 139 exceeds a predetermined value. The data creation unit 140 acquires the installation location of the camera C that generated the image data of the area where employee M was walking at the time included in the measurement data, and the walking speed of employee M, which was identified by the state identification unit 135 based on multiple image data generated at the time included in the measurement data.

[0083] Specifically, as shown in Figure 12, the data creation unit 140 identifies the measurement start time and measurement end time if the heart rate waveform shown in the pulse wave data of employee M1 exceeds a predetermined value. The data creation unit 140 identifies the camera C that generated the image data including employee M1 from the measurement start time to the measurement end time by referring to the first detection data shown in Figure 7. The data creation unit 140 identifies the installation location of the identified camera C by referring to the camera position data shown in Figure 6.

[0084] The data creation unit 140 obtains the walking speed of employee M1, which has been identified based on the image data showing employee M1 walking in an area within room S from the start time to the end time of measurement, from the state identification unit 135 or the storage unit 12. The data creation unit 140 stores stress state data, which associates the identified camera C installation location with the acquired walking speed, in the stress state data storage unit 124.

[0085] By operating in this manner, the data creation unit 140 can identify the times when employee M is experiencing stress, even if employee M does not submit declared data, and thus create stress state data.

[0086] [Processing flow in data processing device 1] Figure 13 is a flowchart showing the processing flow in the data processing device 1. The image data acquisition unit 131 acquires multiple captured image data from multiple cameras C (S1). The detection unit 132 identifies the image of employee M in each of the multiple captured image data acquired by the image data acquisition unit 131 and detects the location of employee M (S2). Based on the location where employee M was detected in the captured image data and the location of the camera C that generated the captured image data, the detection unit 132 identifies the area where the detected employee M is located (S3).

[0087] The selection unit 133 selects one or more cameras C from among the multiple cameras C installed in room S (S4). If, for example, employee M is walking in an area captured by multiple cameras C, the selection unit 133 selects two cameras C to be used for distance image data. The selection unit 133 selects one or more captured image data generated by the selected one or more cameras C (S5). If the selection unit 133 has selected multiple captured image data, the distance image creation unit 134 creates distance image data using the selected multiple captured image data.

[0088] The state identification unit 135 identifies the walking speed of employee M included in the captured image data selected by the selection unit 133 or the distance image data created by the distance image creation unit 134 (S6). The output unit 136 determines whether or not employee M is experiencing stress by comparing the walking speed identified by the state identification unit 135 with the walking speed of employee M included in the stress state data stored in the stress state data storage unit 124.

[0089] If the walking speed identified by the state identification unit 135 falls within the conditions for walking speed when employee M is experiencing stress, as indicated by the stress state data (YES in S7), the output unit 136 determines that employee M is experiencing stress (S8). On the other hand, if the walking speed identified by the state identification unit 135 does not fall within the conditions for walking speed when employee M is experiencing stress (NO in S7), the output unit 136 determines that employee M is not experiencing stress (S9). The output unit 136 outputs the determined result to an external information terminal (S10).

[0090] If no operation to terminate processing is performed (NO in S11), the data processing device 1 repeats the processes from S1 to S10. If an operation to terminate processing is performed (YES in S11), the data processing device 1 terminates the processing.

[0091] [Effects of Data Processing Device 1] As explained above, in the data processing device 1, the detection unit 132 identifies the location of employee M using multiple image data acquired by the image data acquisition unit 131, and the state identification unit 135 identifies the characteristics of employee M's walking state using the image data in which employee M is pictured, which has been selected by the selection unit 133 based on employee M's location. Then, the output unit 136 determines whether or not employee M is experiencing stress by comparing the characteristics of the walking state of employee M when she is experiencing stress, as contained in the stress state data stored in the storage unit 12, with the characteristics of the walking state identified by the state identification unit 135.

[0092] Because the data processing device 1 operates in this manner, even if employee M walks freely within the space of room S, they are included in the imaging range of camera C. This prevents the situation where employee M is not captured in the image data, making it impossible to determine whether or not employee M is experiencing stress. Furthermore, by using the characteristics of the walking state associated with each of the multiple employee Ms when employee M is experiencing stress, the accuracy of determining whether or not employee M is experiencing stress can be improved.

[0093] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of its gist. For example, all or part of the apparatus can be configured by functionally or physically distributing and integrating in any unit. Furthermore, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effects of the new embodiments resulting from the combinations are combined with the effects of the original embodiments. [Explanation of Symbols]

[0094] 1 Data Processing Device 11 Communications Department 12 Storage section 13 Control Unit 100 Data Processing Systems 121 Program Storage Unit 122 Area Data Storage Unit 123 Detection data storage unit 124 Stress state data storage unit 131 Image Data Acquisition Unit 132 Detection unit 133 Selection Section 134 Distance Image Creation Unit 135 State Identification Unit 136 Output section 137 Recording Processing Unit 138 Declaration Data Acquisition Department 139 Measurement data acquisition unit 140 Data Creation Department C Camera

Claims

1. A storage unit that stores stress state data indicating the characteristics of the walking state in which a user is experiencing stress, associated with each of several users walking in a predetermined space, An image data acquisition unit that acquires multiple image data generated by multiple imaging devices installed in the predetermined space, A detection unit that identifies the user's position in the predetermined space, A selection unit that selects one or more image data corresponding to the user's position from the plurality of image data, A state identification unit identifies the characteristics of the user's walking state based on the image of the user included in the one or more captured image data corresponding to the multiple user positions selected by the selection unit, An output unit outputs a result of determining whether or not the user is experiencing stress, based on a comparison between the characteristics indicated by the stress state data stored in the memory unit in association with the user and the characteristics identified by the state identification unit. It has, The storage unit stores the stress state data in association with the installation positions of the plurality of imaging devices. The output unit determines whether or not the user is experiencing stress, using the characteristics indicated by the stress state data stored in the storage unit in association with the installation location of the imaging device that generated the captured image data used by the state identification unit to identify the characteristics of the user's walking state. Data processing device.

2. A storage unit that stores stress state data indicating the characteristics of the walking state in which a user is experiencing stress, associated with each of several users walking in a predetermined space, An image data acquisition unit that acquires multiple image data generated by multiple imaging devices installed in the predetermined space, A detection unit that identifies the user's position in the predetermined space, A selection unit that selects one or more image data corresponding to the user's position from the plurality of image data, A state identification unit identifies the characteristics of the user's walking state based on the image of the user included in the one or more captured image data corresponding to the multiple user positions selected by the selection unit, An output unit outputs a result of determining whether or not the user is experiencing stress, based on a comparison between the characteristics indicated by the stress state data stored in the memory unit in association with the user and the characteristics identified by the state identification unit. A declaration data acquisition unit that obtains data from the user indicating that they are experiencing stress, A data creation unit creates stress state data that indicates the characteristics of the user's walking state as shown in the image data that includes the user, at the time the declaration data acquisition unit acquires the data, A data processing device having

3. A storage unit that stores stress state data indicating the characteristics of the walking state in which a user is experiencing stress, associated with each of several users walking in a predetermined space, An image data acquisition unit that acquires multiple image data generated by multiple imaging devices installed in the predetermined space, A detection unit that identifies the user's position in the predetermined space, A selection unit that selects one or more image data corresponding to the user's position from the plurality of image data, A state identification unit identifies the characteristics of the user's walking state based on the image of the user included in the one or more captured image data corresponding to the multiple user positions selected by the selection unit, An output unit outputs a result of determining whether or not the user is experiencing stress, based on a comparison between the characteristics indicated by the stress state data stored in the memory unit in association with the user and the characteristics identified by the state identification unit. A measurement data acquisition unit that acquires measurement data of the user's pulse wave, A data creation unit creates stress state data that indicates the characteristics of the user's walking state as shown in the imaged data that includes the user, at the point when the pulse wave shown in the measurement data acquired by the measurement data acquisition unit indicates that the user is feeling stressed. A data processing device having

4. Computers, An image data acquisition unit that acquires multiple image data generated by multiple imaging devices installed in a predetermined space, A detection unit that identifies the user's position in the predetermined space, A selection unit that selects one or more image data corresponding to the user's position from the plurality of image data, A state identification unit identifies the characteristics of the user's walking state based on the image of the user included in the one or more captured image data corresponding to the multiple user positions selected by the selection unit, An output unit outputs a result of determining whether or not the user is experiencing stress, based on a comparison between the characteristics of the user's walking state when they are experiencing stress, which are stored in the memory unit in association with the user, and the characteristics identified by the state identification unit. It is a program designed to function as such. The output unit is a program that determines whether or not a user is experiencing stress, using the characteristics indicated by stress state data, which indicates the characteristics of a user's walking state when they are experiencing stress, and which is stored in the storage unit in association with the installation location of the imaging device that generated the captured image data used by the state identification unit to identify the characteristics of the user's walking state, and is associated with each of the multiple users walking in a predetermined space.

5. Computers, An image data acquisition unit that acquires multiple image data generated by multiple imaging devices installed in a predetermined space, A detection unit that identifies the user's position in the predetermined space, A selection unit that selects one or more image data corresponding to the user's position from the plurality of image data, A state identification unit identifies the characteristics of the user's walking state based on the image of the user included in the one or more captured image data corresponding to the multiple user positions selected by the selection unit, An output unit outputs a result of determining whether or not the user is experiencing stress, based on a comparison between the characteristics of the walking state when experiencing stress, as indicated by the stress state data stored in the memory unit in association with the user, and the characteristics identified by the state identification unit. A declaration data acquisition unit that obtains data from the user indicating that they are experiencing stress, A data creation unit creates stress state data associated with each of a plurality of users walking in a predetermined space, which represent the characteristics of the user's walking state as shown in the image data containing the user at the time the declaration data acquisition unit acquires the data, A program designed to function as such.

6. Computers, An image data acquisition unit that acquires multiple image data generated by multiple imaging devices installed in a predetermined space, A detection unit that identifies the user's position in the predetermined space, A selection unit that selects one or more image data corresponding to the user's position from the plurality of image data, A state identification unit identifies the characteristics of the user's walking state based on the image of the user included in the one or more captured image data corresponding to the multiple user positions selected by the selection unit, An output unit outputs a result of determining whether or not the user is experiencing stress, based on a comparison between the characteristics of the walking state when experiencing stress, as indicated by the stress state data stored in the memory unit in association with the user, and the characteristics identified by the state identification unit. A measurement data acquisition unit that acquires measurement data of the user's pulse wave, A data creation unit creates stress state data indicated by the captured image data that includes the user at the point when the pulse wave indicated by the measurement data acquired by the measurement data acquisition unit indicates that the user is experiencing stress, A program designed to function as such.

Images