Business support system and mobile device

Abstract

[Problem] To provide a business support system that can record the actions of personnel engaged in business and provide business support to the personnel. [Solution] The business support system monitors and manages the actions of staff engaged in work and supports the staff in carrying out their work, and comprises a portable device worn by the staff, an administrator terminal on which images captured by the portable device are displayed and which can communicate with the portable device, and a business support server which configures and stores staff action information based on data acquired by the portable device and controls communication between the portable device and the administrator terminal, the portable device and the administrator terminal being connected to the business support server via a communication line, and the portable device including a headset unit, a battery unit unit, and a communication power cable connecting the headset unit and the battery unit unit.

Description

[Technical field]

[0001] The present invention relates to a business support system, an administrator terminal, a business support server, and a mobile device that monitors and manages the actions of personnel engaged in business and supports the personnel in carrying out the business. [Background technology]

[0002] Conventionally, the actions of workers have been recorded, and a known action recording system is described in Patent Document 1. This action recording system has a terminal device worn by the worker, and is configured to capture images of the surroundings using an imaging unit built into the terminal device, and record the captured images. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-037951 A Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, due to a shortage of human resources, there are more and more situations where inexperienced staff members are working alone, and there is a growing demand for providing real-time advice to such staff members and supporting their work. However, the behavior recording system described in Patent Document 1 does not have a function to support the staff members' work.

[0005] The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a work support system that can record the actions of a person in charge and provide work support to the person in charge. Furthermore, this business support system can be used for providing education on how to carry out the business, securing and recording the knowledge (information) necessary to carry out the business, protecting the personnel carrying out the business and securing evidence, and improving the governance of the business. [Means for solving the problem]

[0006] A business support system according to an embodiment of the present invention comprises: A business support system that monitors and manages the actions of personnel engaged in a business and supports the personnel in carrying out the business, a portable device worn by the person in charge; an administrator terminal that displays images captured by the portable device and is capable of communicating with the portable device; a business support server that configures and stores staff action information based on data acquired by the portable device and controls communication between the portable device and the manager terminal; the portable device and the administrator terminal are connected to the business support server via a communication line; the portable device includes a headset unit, a battery unit, and a communication power cable connecting the headset unit and the battery unit; The headset unit includes: A camera unit that captures images of the person in charge and acquires the captured images; a microphone unit that receives an input of a voice around the person in charge and generates a voice data of the voice around the person in charge; an earphone unit that generates a voice to be transmitted to the person in charge based on the voice data; a voice communication control operation unit that is operated by the person in charge to control the operation of the microphone unit and the earphone unit to communicate voice data, The battery unit includes: a captured image sending unit that sends an image captured by the camera unit to the business support server via the communication line; a person in charge surrounding voice sending unit that sends the person in charge surrounding voice acquired by the microphone unit to the business support server via the communication line; an operation input and image display unit that receives operation input from the person in charge and displays an image; a person in charge voice receiving unit that receives a person in charge voice to be generated from the earphone unit, the person in charge voice being voice data transmitted from the business support server; a person in charge image receiving unit that receives a person in charge image to be displayed on the operation input and video display unit, the person in charge image receiving unit being image data sent from the business support server; a battery section that supplies driving power to operate the headset section via the communication power cable to the headset section and also supplies driving power to operate the battery unit section; A control unit for controlling an operation of the portable device, The control unit controls the captured image sending unit, the person in charge surroundings voice sending unit, the person in charge voice transmission receiving unit, the person in charge transmission image receiving unit, the microphone unit, the earphone unit, the camera unit, and the operation input and video display unit so that the portable device and the manager terminal transmit voice and video in both directions via the business support server and the communication line, thereby causing the person in charge transmission image sent from the manager terminal to be displayed on the operation input and video display unit and sending out the captured image taken by the camera unit, and generating the person in charge transmission voice sent from the manager terminal from the earphone unit and sending out the person in charge surroundings voice acquired by the microphone unit. It is characterized by:

[0007] In the business support system, The person in charge transmission image includes an image including information necessary for the person in charge to perform the task. It is characterized by:

[0008] In the business support system, The person in charge transmission voice includes a voice indicating information necessary for the person in charge to perform the task. It is characterized by:

[0009] In the business support system, Video, photograph, and / or memo information related to training and / or technology transfer for carrying out the business is stored in the portable device storage unit of the battery unit of the portable device. It is characterized by:

[0010] In the business support system, The administrator terminal, a display screen receiving unit that receives an image display screen on which the captured image is displayed, the image display screen being sent from the business support server; a manager terminal display unit that displays the received image display screen; an output voice receiving unit that receives the surrounding voice of the person in charge sent from the business support server; a voice generating unit that generates the received surrounding voice of the person in charge; a transmission voice input unit to which the person in charge's transmission voice is input; a voice transmission unit that transmits the input person-in-charge voice to the business support server; a behavior information provision request unit that transmits a behavior information provision request for requesting the person in charge behavior information to the business support server; a behavior information receiving unit that receives the person in charge behavior information provided from the business support server in response to the behavior information provision request; a transmission image input unit to which the person in charge transmission image is input or which configures the person in charge transmission image; a transmission image sending unit that sends the person-in-charge transmission image input or configured to the business support server; have It is characterized by:

[0011] In the business support system, The business support server includes: a photographed image receiving unit that receives the photographed image sent from the photographed image sending unit; a person in charge surrounding voice receiving unit that receives the person in charge surrounding voice transmitted from the person in charge surrounding voice transmitting unit; an action information configuration unit that configures the person in charge action information including the received photographed image and the person in charge surrounding sound; An action information storage unit that stores the configured person in charge action information; a display screen configuration unit that processes the photographed image accepted by the photographed image acceptance unit and configures the image display screen; a display screen providing unit that provides the configured image display screen to the administrator terminal; a voice transmission receiving unit that receives the person-in-charge voice transmitted from the voice transmission sending unit; a transmission image receiving unit that receives the person-in-charge transmission image sent from the transmission image sending unit K; a call establishment unit that establishes a call by voice and video between the portable device and the manager terminal by sending the person in charge surrounding voice received by the person in charge surrounding voice receiving unit to the manager terminal, sending the person in charge transmission voice received by the transmission voice receiving unit to the portable device, sending an image display screen including the photographed image provided by the display screen providing unit to the manager terminal, and sending the person in charge transmission image received by the transmission image receiving unit to the portable device; a behavior information request receiving unit that receives the behavior information request sent from the behavior information request unit; and a behavior information providing unit that provides the person in charge behavior information read from the behavior information storage unit in response to the received behavior information provision request to the manager terminal, When the action information receiving unit receives the person in charge action information, the received person in charge action information is displayed on the manager terminal display unit. It is characterized by:

[0012] In the business support system, The earphone unit is characterized in that it is configured by a bone conduction type earphone, a canal type earphone, or an inner-ear type earphone.

[0013] In the business support system, The camera unit, the microphone unit, and the earphone unit are integrated as a headset, and the headset is configured to be worn on the head of the person in charge. It is characterized by:

[0014] In the business support system, the mobile device has a camera off mode; In the camera stop mode, The image capturing by the camera unit and the acquisition of sound by the microphone unit are stopped, and a notification sound is generated from the earphone unit to notify that the camera is in the camera stop mode. It is characterized by:

[0015] In the business support system, the mobile device has a talk mode; In the call mode, A call can be made between the portable device and the administrator terminal, and the image captured by the camera unit is highlighted on the display unit of the administrator terminal. It is characterized by:

[0016] In the business support system, The mobile device comprises: a location information detection unit that receives radio waves from a satellite positioning system and calculates location information; a location information sending unit that sends the calculated location information to the business support server; The business support server includes: a location information receiving unit that receives the location information sent from the location information sending unit; a map information requesting unit that transmits a map information request for requesting map information of a periphery of the received position information; a map information receiving unit that receives the map information provided in response to the map information request, The behavioral information configuration unit The staff member behavior information is configured so as to link the location information received by the location information receiving unit to the photographed image received by the photographed image receiving unit and the staff member surrounding sound received by the staff member surrounding sound receiving unit; The display screen configuration unit includes: creating a position information image that displays the position information received by the position information receiving unit as the position of the portable device on the map of the map information received by the map information receiving unit; The image display screen is configured so that the photographed image and the position information image accepted by the photographed image accepting unit are displayed on the same screen. It is characterized by:

[0017] In the business support system, The business support server includes: A voice-to-text generator that generates text from voice, The voice-to-text generation unit generating the person in charge surrounding voice received by the person in charge surrounding voice receiving unit as a surrounding voice character string; The behavioral information configuration unit The staff action information is configured so as to link the surrounding voice character string generated by the voice character generation unit to the photographed image received by the photographed image receiving unit and the staff surrounding voice received by the staff surrounding voice receiving unit. It is characterized by:

[0018] In the business support system, The business support server includes: An abusive language determination unit that determines abusive language, The abusive language determination unit determining whether or not the surrounding voice character string generated by the voice character generation unit contains abusive language words registered in advance, and generating abusive language information based on the determination result; The behavioral information configuration unit The staff member action information is configured to include the abusive language information generated by the abusive language determination unit. It is characterized by:

[0019] In the business support system, The business support server includes: A voice analysis unit that analyzes voice and estimates a mental state, The voice analysis unit is an analysis unit that analyzes the person in charge's surrounding voice received by the person in charge surrounding voice receiving unit and the surrounding voice character string generated by the voice character generating unit to estimate the mental state of the person in charge, and generates person in charge mental state information based on the estimated mental state of the person in charge; The behavior information configuration unit The staff member behavior information is configured to include the staff member mental state information generated by the voice analysis unit. It is characterized by:

[0020] In the business support system, The business support server includes: a face authentication unit that authenticates a face of the care recipient appearing in the photographed image received by the photographed image receiving unit and identifies a name of the care recipient; an expression analysis unit that analyzes the facial expression of the care recipient shown in the captured image received by the captured image receiving unit, estimates at least one of emotions such as joy, anger, sadness, and happiness, vitality, and satisfaction, and calculates an estimated expression value by quantifying the estimation result; a care recipient information configuration unit that configures care recipient information to include the name of the care recipient identified by the face authentication unit and the estimated facial expression value calculated by the facial expression analysis unit; a care recipient information storage unit configured to store the care recipient information; a care recipient information provision request receiving unit that receives a care recipient information provision request that requests the care recipient information transmitted from the manager terminal; a care receiver information providing unit that provides the care receiver information read from the care receiver information storage unit to the manager terminal in response to the received care receiver information provision request, The administrator terminal, a care recipient information provision request unit configured to send the care recipient information provision request to the business support server; a care receiver information receiving unit that receives the care receiver information provided from the business support server in response to the care receiver information provision request, When the care recipient information receiving unit receives the care recipient information, the received care recipient information is displayed on the manager terminal display unit. It is characterized by:

[0021] In the business support system, The business support server includes: a contact time measurement unit that measures a contact time during which the care recipient appears in the photographed image received by the photographed image receiving unit, The care recipient information configuration unit, The care receiver information is configured to include the face-to-face time measured by the face-to-face time measuring unit. It is characterized by:

[0022] In the business support system, The business support server includes: an image and audio analysis unit that performs analysis based on the captured image and the sound around the person; The image and sound analysis unit an action of the person in charge is estimated from the photographed image received by the photographed image receiving unit, a statement of the person in charge is extracted from the voice around the person in charge received by the voice around the person in charge receiving unit, and a work evaluation of the person in charge is determined based on the action and the voice of the person in charge; The behavioral information configuration unit The staff action information is configured to include the work evaluation determined by the image and voice analysis unit. It is characterized by:

[0023] In the business support system, The business support server includes: an image and audio analysis unit that performs analysis based on the captured image and the sound around the person; The image and sound analysis unit an action of the person in charge is estimated from the photographed image received by the photographed image receiving unit, a statement of the person in charge is extracted from the voice around the person in charge received by the voice around the person in charge receiving unit, and a work evaluation of the person in charge is determined based on the action and the voice of the person in charge; The behavioral information configuration unit The person in charge action information is configured to include the work evaluation determined by the image and voice analysis unit, The image and sound analysis unit extracting a questioning statement for the care recipient from the person in charge surrounding voice received by the person in charge surrounding voice receiving unit, estimating the reaction of the care recipient to the questioning statement from the movement of the care recipient shown in the captured image received by the captured image receiving unit, and determining a risk of hearing loss or dementia of the care recipient; The care recipient information configuration unit, The care recipient information is configured to include the risk of hearing loss or dementia of the care recipient determined by the image and sound analysis unit. It is characterized by:

[0024] In the business support system, a plurality of said mobile devices; The business support server includes: The system further includes an administrator absence determination unit that determines the absence of the administrator who operates the administrator terminal, and when one of the plurality of portable devices transitions to the call mode, if the administrator absence determination unit determines that the administrator is absent, communication between the plurality of portable devices becomes possible. It is characterized by:

[0025] A portable device that is applied to a business support system that monitors and manages the actions of a person engaged in a business and supports the person in carrying out the business, the portable device being worn by the person, The business support system includes a manager terminal on which images captured by the portable device are displayed and which can communicate with the portable device, and a business support server that configures and stores staff action information based on data acquired by the portable device and controls communication between the portable device and the manager terminal; The mobile device comprises: the portable device and the administrator terminal are connected to the business support server via a communication line; the portable device includes a headset unit, a battery unit, and a communication power cable connecting the headset unit and the battery unit; The headset unit includes: A camera unit that captures images of the person in charge and acquires the captured images; a microphone unit that receives an input of a voice around the person in charge and generates a voice data of the voice around the person in charge; an earphone unit that generates a voice to be transmitted to the person in charge based on the voice data; a voice communication control operation unit that is operated by the person in charge to control the operation of the microphone unit and the earphone unit to communicate voice data, The battery unit includes: a captured image sending unit that sends an image captured by the camera unit to the business support server via the communication line; a person in charge surrounding voice sending unit that sends the person in charge surrounding voice acquired by the microphone unit to the business support server via the communication line; an operation input and image display unit that receives operation input from the person in charge and displays an image; a person in charge voice receiving unit that receives a person in charge voice to be generated from the earphone unit, the person in charge voice being voice data transmitted from the business support server; a person in charge image receiving unit that receives a person in charge image to be displayed on the operation input and video display unit, the person in charge image receiving unit being image data sent from the business support server; a battery section that supplies driving power to operate the headset section via the communication power cable to the headset section and also supplies driving power to operate the battery unit section; A control unit for controlling an operation of the portable device, The control unit controls the captured image sending unit, the person in charge surroundings voice sending unit, the person in charge voice transmission receiving unit, the person in charge transmission image receiving unit, the microphone unit, the earphone unit, the camera unit, and the operation input and video display unit so that the portable device and the manager terminal transmit voice and video in both directions via the business support server and the communication line, thereby causing the person in charge transmission image sent from the manager terminal to be displayed on the operation input and video display unit and sending out the captured image taken by the camera unit, and generating the person in charge transmission voice sent from the manager terminal from the earphone unit and sending out the person in charge surroundings voice acquired by the microphone unit. A mobile device characterized by: [Brief description of the drawings]

[0026] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of a task support system according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of functional blocks of a mobile device constituting the task support system illustrated in FIG. [Figure 3A] FIG. 3A is a diagram illustrating an example of a three-dimensional appearance of a headset unit of a portable device according to an embodiment, as viewed from a diagonal direction to the right of the front. [Figure 3B] FIG. 3B is a diagram illustrating an example of a three-dimensional appearance of a headset unit of a portable device according to an embodiment, as viewed from a diagonal left direction from the front. [Figure 3C] FIG. 3C is a diagram illustrating an example of a three-dimensional appearance of a headset portion of a portable device according to an embodiment as viewed from the side. [Figure 3D] FIG. 3D is a diagram illustrating an example of a three-dimensional appearance of a headset unit of a portable device according to an embodiment as viewed from above. [Figure 4A] FIG. 4A is a diagram illustrating an example of a three-dimensional appearance of a battery unit section of a portable device according to an embodiment, as viewed from an oblique right direction. [Figure 4B] FIG. 4B is a diagram illustrating an example of a three-dimensional appearance of a battery unit section of a portable device according to an embodiment, as viewed from an oblique left direction. [Diagram 5] FIG. 5 is a diagram illustrating an example of functional blocks of the manager terminal constituting the business support system illustrated in FIG. [Figure 6] FIG. 6 is a diagram illustrating an example of functional blocks of the business support server constituting the business support system illustrated in FIG. [Figure 7A] FIG. 7A is a diagram illustrating an example of the configuration of person in charge action information according to the embodiment. [Figure 7B] FIG. 7B is a diagram showing an example of the display of person in charge mental state information included in person in charge action information according to the embodiment. [Figure 8] FIG. 8 is a schematic diagram showing an example of the configuration of an image display screen displayed on the administrator terminal according to the embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of the configuration of care recipient information according to the embodiment. [Figure 10] FIG. 10 is a diagram for explaining an example of the flow of operations of the task support system performed during the task of a person in charge in the embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of a flow in which an analysis is performed based on the configured person in charge behavior information in the business support system according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] [Embodiment] An embodiment of the present invention will be described with reference to Figs. 1 to 11. In the following embodiment, a nursing care business will be described as an example of a business to which the business support system is applied. However, the business to which the business support system according to the present invention is applied is not limited to the nursing care business. In other words, the business support system according to the present invention can be applied to other businesses, such as construction businesses, childcare and education businesses, security businesses, maintenance businesses, police, administrative businesses such as ambulances and fire departments, trading businesses, manufacturing businesses, and medical businesses.

[0028] [Configuration of business support system] FIG. 1 is a diagram showing an example of the configuration of a business support system according to an embodiment. As described above, this embodiment will be described by taking a nursing care business as an example of business. The form of nursing care includes both nursing care in a facility and visiting nursing care. In addition, a caregiver (worker) engaged in nursing care work will be called a person in charge.

[0029] This business support system 1 includes portable devices 61, 62, . . . , 6 n and mobile devices 61, 62, . . . , 6 n An administrator terminal 8 that displays images taken by the portable devices 61, 62, . . . , 6n and that can communicate with the portable devices 61, 62, . . . , 6n; n Based on the data acquired by the mobile devices 61, 62, . . . , 6, the person in charge action information 200 described later is constructed and stored. n and a manager terminal 8, and a map information providing server 10 that provides map information. These mobile devices 61, 62, . . . , 6 n The administrator terminal 8, the business support server 2, and the map information providing server 10 are connected via a communication line 11, and are capable of transmitting and receiving information to and from each other. The communication line 11 may be the Internet or the like. This business support system 1 records the actions of staff engaged in business and provides business support to the staff, thereby monitoring and managing the actions of the staff and supporting the performance of business by staff outside Tokyo.

[0030] In the following description, unless it is necessary to specify that there are multiple mobile devices 61, 62, . . . , 6 n The portable device 6 is used to represent the above.

[0031] [Mobile devices] 2 is a schematic functional block diagram of the portable device 6. The portable device 6 is configured to include a control unit 60, a camera unit 61, a captured image transmission unit 62, a microphone unit 63, a person in charge surroundings voice transmission unit 64, a person in charge transmission voice reception unit 65, an earphone unit 66, a position information detection unit 70, a position information transmission unit 71, an operation input and video display unit T, a battery unit E, a voice communication control operation unit G, a power button B1, operation buttons B2, B3, a person in charge transmission image reception unit H, a portable device storage unit 73, and a communication unit 74.

[0032] For example, in this embodiment, the camera unit 61, microphone unit 63, earphone unit 66, and voice communication control operation unit G are mounted on a headset unit 600 of the portable device 6 shown in Figures 3A to 3D, as described below.

[0033] In addition, in this embodiment, the control unit 60, the captured image sending unit 62, the person in charge surrounding voice sending unit 64, the person in charge transmission voice receiving unit 65, the position information detection unit 70, the position information sending unit 71, the operation input and video display unit T, the battery unit E, the voice communication control operation unit G, the power button B1, the operation buttons B2, B3, the person in charge transmission image receiving unit H, the portable device memory unit 73 and the communication unit 74 are mounted in the battery unit 620Z of the portable device 6 shown in Figures 4A and 4B, as described below.

[0034] The control unit 60 controls the operation of each element of the portable device 6. For example, the control unit 60 is configured to include a CPU (Central Processing Unit), a flash memory which is a non-volatile memory, and a RAM (Random Access Memory) which is a volatile memory (not shown). The CPU executes programs, performs arithmetic processing, and controls each element constituting the portable device 6. The flash memory stores programs executed by the CPU, registered data, and the like, and the RAM is used as a work area for the CPU to execute programs and perform arithmetic processing.

[0035] The camera unit 61 is adapted to capture images of the surroundings of the person in charge. The imaging element of the camera unit 61 may be a complementary metal oxide semiconductor (CMOS) area sensor, a charged coupled device (CCD) area sensor, or the like. The captured image sending unit 62 sends (transmits) the captured image 802 captured by the camera unit 61 as image data to the business support server 2 via the communication line 11. The captured image 802 often constitutes a moving image, but may also be a still image.

[0036] The microphone unit 63 acquires the voice around the person in charge. That is, the microphone unit 63 is configured to generate the voice around the person in charge, which is voice data, by inputting the voice around the person in charge. In concrete terms, the microphone unit 63 detects the voice and converts it into an electrical signal.

[0037] The person in charge surrounding voice sending unit 64 is configured to send the person in charge surrounding voice 208 acquired by the microphone unit 63 to the business support server 2 as voice data via the communication line 11 .

[0038] The earphone unit 66 generates a voice to be transmitted to the person in charge based on the voice data. The person in charge voice receiving unit 65 receives the person in charge voice as voice data to be generated from the earphone unit 66. This person in charge voice is sent from the business support server 2 via the communication line 11 as voice data. The earphone unit 66 generates the voice to be transmitted to the person in charge. In concrete terms, the earphone unit 66 converts an electrical signal containing voice information (voice data) into a sound wave.

[0039] The position information detection unit 70 receives radio waves from a satellite positioning system and calculates position information 212. A GPS (Global Positioning System) module can be used for this position information detection unit 70. The GPS module receives a signal from the GPS system and notifies the current position on the earth. By acquiring the position information 212 from the GPS module, the current position of the portable device 6 can be obtained. The position information sending unit 71 sends the position information 212 calculated by the position information detection unit 70 to the business support server 2 as data via the communication line 11 .

[0040] The person in charge transmission image receiving unit H receives a person in charge transmission image to be displayed on the operation input and video display unit T, which is image data sent from the business support server 2 via the communication line 11.

[0041] The operation input and image display unit T receives operation input from the person in charge and displays images.

[0042] The portable device storage unit 73 is composed of a non-volatile memory. For example, an SD memory card can be used. The portable device storage unit 73 stores the captured image 802, the voice around the person in charge 208, the position information 212, and the like. When the communication line 11 is disconnected, the captured image 802 and the like are temporarily stored in the mobile device storage unit 73, and after the connection of the communication line 11 is restored, the temporarily stored captured image 802 and the like are transmitted to the business support server 2.

[0043] The communication unit 74 of the battery unit 620Z transmits and receives data to and from the outside. The communication unit 74 has a function of connecting to the communication line 11 by Wi-Fi (registered trademark) communication and a function of connecting to the communication line 11 via a mobile communication line by LTE (Long Term Evolution) communication. In an environment where a connection is possible by Wi-Fi communication, the communication unit 74 is controlled to connect to the communication line 11 via Wi-Fi communication, and in an environment where a connection by Wi-Fi communication is not possible, the communication unit 74 is controlled to connect to the communication line 11 via LTE communication.

[0044] Battery section E supplies driving power (power source) for operating headset section 600 via communication power cable 650 to headset section 600, and also supplies driving power (power source) for operating battery unit section 620Z.

[0045] The portable device 6 includes a headset unit 600 (see FIGS. 3A to 3D), a battery unit 620Z (see FIGS. 4A to 4B), and a communication power cable 650 (see FIGS. 3C to 3D) that connects the headset unit 600 and the battery unit 620Z. The portable device 6 is configured such that the headset unit 600, in which a camera unit 61, a microphone unit 63, and an earphone unit 66 are integrated, and the battery unit 620Z, which includes a battery unit E serving as a power source and a communication unit 74 having a communication function, are connected by the communication power cable 650.

[0046] Here, Fig. 3A is a diagram showing an example of a three-dimensional appearance of a headset unit of a portable device according to an embodiment when viewed from a diagonal right direction from the front. Fig. 3B is a diagram showing an example of a three-dimensional appearance of a headset unit of a portable device according to an embodiment when viewed from a diagonal left direction from the front. Fig. 3C is a diagram showing an example of a three-dimensional appearance of a headset unit of a portable device according to an embodiment when viewed from the side. Fig. 3D is a diagram showing an example of a three-dimensional appearance of a headset unit of a portable device according to an embodiment when viewed from above. It should be noted that communication power cable 650 is not shown in FIGS. 3A and 3B, but is shown in FIGS. 3C and 3D.

[0047] The headset unit 600 shown in Figs. 3A to 3D is worn on the head of a person in charge (not shown). 3A to 3D, widened portions 614, which are generally rectangular in side view and wide in the vertical direction, are formed on both ends of thin-line connecting portion 612 made of an elastic material and attached along the back of the head of headset portion 600. An electric board on which electronic components are mounted is embedded inside widened portions 614.

[0048] 4A and 4B, a video recording button and a still image recording button may be displayed on the operation input and video display section T of the battery unit section 620Z, or the operation buttons B2 and B3 may function as the video recording button and the still image recording button. Note that the video recording button and the still image recording button may be provided on the headset section 600, for example, on the widening section 614. For example, when the person in charge operates the video capture button, a video is captured by the camera unit 61, and the captured video is stored in the portable device storage unit 73 while also being transmitted as video data to the business support server 2 via the communication line 11 and stored therein. When the person in charge operates the still image capture button, a still image is captured by the camera unit 61, and the captured still image is stored in the portable device storage unit 73 while also being transmitted as image data to the business support server 2 via the communication line 11 and stored therein.

[0049] 4A and 4B, a volume button may be displayed on the operation input and video display section T of the battery unit section 620Z, or the operation buttons B2 and B3 may function as the volume button. The volume button may be provided on the headset section 600, for example, on the widening section 614. For example, when the volume button is operated by a person in charge, the volume generated from the earphone unit 66 of the headset unit 600 is adjusted.

[0050] Also, for example, as shown in Figures 3C to 3D, a communication power cable connection terminal N1 is provided on the back side of the widening portion 614 of the headset section 600, and a communication power cable 650 is inserted into this communication power cable connection terminal N1 to connect to the battery unit section 620Z.

[0051] 3A to 3D, a head portion 619 incorporating a microphone unit 63 and an earphone unit 66 is formed at the tip of each of the ear hook units 616 on both sides. In this embodiment, a bone conduction earphone is used for the earphone unit 66. Note that the earphone unit 66 is not limited to a bone conduction earphone, and may be another type of earphone, such as a canal type earphone or an inner ear type earphone.

[0052] 3A to 3D, the camera unit 61 is provided on the outer side surface of the left front part 619 so as to be rotatable in the vertical direction. The vertical rotation angle of the camera unit 61 is fixed at a plurality of predetermined positions. When performing care tasks such as bathing or changing clothes, the camera unit 61 can be directed upward so as not to capture the image of the care recipient.

[0053] 3B, for example, a voice communication control operation unit (operation button) G is provided on the outer side surface of the right front part 619 of the headset unit 600. This voice communication control operation unit G is a button that is operated by the person in charge to control the operation of the microphone unit 63 and the earphone unit 66 and communicate voice data. When this voice communication control operation unit G is operated (pressed) by the person in charge, the portable device 6 transitions to a call mode, and for example, a call can be made between the portable device 6 and the administrator terminal 8 (or another portable device 6).

[0054] For example, when the person in charge presses the voice communication control operation unit G once, voice communication between the portable device 6 and the administrator terminal 8 is started as a function of a so-called intercommunication system. Then, for example, when the person in charge presses the voice communication control operation unit G again, the voice communication is terminated. Also, for example, when the person in charge presses and holds the voice communication control operation unit G, voice communication between the portable device 6 and the administrator terminal 8 may be executed for the period during which the voice communication control operation unit G is pressed and held.

[0055] Also, for example, when the voice communication control operation unit G is operated to switch to a call mode, the captured image 206 captured by the camera unit 61 is highlighted on the manager terminal 8. Furthermore, the video (image) transmitted from the manager terminal 8 to the portable device 6 may be displayed on the operation input and video display unit T of the battery unit 620Z.

[0056] Also, for example, as shown in Fig. 4A, a power button B1 is provided on the outer side surface of the battery unit section 620Z. For example, the power button B1 is operated (e.g., pressed and held) by a person in charge to switch between power on and off. Also, for example, the power button B1 of the battery unit section 620Z is operated (pressed down for a short time) by a person in charge to switch between a camera stop mode and a camera operation mode. The power button B1 may be disposed on the headset unit 600, for example, on the widened portion 614 or the like.

[0057] The mobile device 6 has a camera stop mode and a camera operation mode. In the camera stop mode, image capture by the camera unit 61 and acquisition of audio by the microphone unit 63 are stopped, and a notification sound notifying that the camera stop mode is set is emitted from the earphone unit 66. On the other hand, in the camera operation mode, image capture by the camera unit 61 and acquisition of audio by the microphone unit 63 are performed, and the notification sound notifying that the camera stop mode is set is stopped.

[0058] When this headset unit 600 is connected to a commercially available PC (Personal Computer) or smartphone via a communication power cable 650, the camera unit 61, microphone unit 63 and earphone unit 66 will operate, and images and audio can be sent to the PC or smartphone as image data or audio data via the communication line 11, and audio sent from the PC or the like as audio data via the communication line 11 can be heard through the earphone unit 66.

[0059] 3, the microphone unit 63 is incorporated in the front part 619, but the present invention is not limited to this configuration and may be provided at the tip of the camera unit 61. When placed in this manner, the microphone unit 63 is positioned in front of the mouth of the person in charge, making it easier to detect the voice.

[0060] Here, Fig. 4A is a diagram showing an example of a three-dimensional appearance of a battery unit part of a portable device according to an embodiment as viewed from a right oblique direction. Also, Fig. 4B is a diagram showing an example of a three-dimensional appearance of a battery unit part of a portable device according to an embodiment as viewed from a left oblique direction. This battery unit part 620Z includes the above-mentioned battery part E, the captured image sending part 62, the person in charge surrounding voice sending part 64, the person in charge transmission voice receiving part 65, the position information detection part 70, the position information sending part 71, and the communication part 74.

[0061] In addition, the battery unit 620Z can download various applications and use the applications. This allows the mobile device 6 to work with various applications, such as a nurse call application and a web conference application, to improve the efficiency of the business in which the business is carried out.

[0062] As shown in Figures 4A and 4B, a communication power cable connection terminal N2 is provided at the lower end of the battery unit 620Z. By inserting the other end of the communication power cable 650 shown in Figures 3C and 3D into this communication power cable connection terminal N2, the headset unit 600 and the battery unit 620Z can be connected by the communication power cable 650.

[0063] In addition, the operation input and image display unit T is provided on the surface of the battery unit 620Z. As described above, the operation input and image display unit T receives operation input from the person in charge and displays images. Note that the operation input and image display unit T may be configured to display the operating state of the portable device 6.

[0064] This operation input and image display unit T is configured with a display device, and for example, a display device such as a touch panel LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) is used.

[0065] The operation input and image display section T of the battery unit section 620Z may display, for example, a Wi-Fi communication display, an LTE display, and a GPS display. This indicates that these functions are operating.

[0066] Here, for example, the control unit 60 of the portable device 6 controls the captured image sending unit 62, the person in charge surroundings voice sending unit 64, the person in charge transmission voice receiving unit 65, the person in charge transmission image receiving unit H, the microphone unit 63, the earphone unit 66, the camera unit 61, and the operation input and video display unit T so that the portable device 6 and the manager terminal 8 transmit voice and video bidirectionally via the business support server 2 and the communication line 11. As a result, the control unit 60 causes the person in charge transmission image sent from the manager terminal 8 to be displayed on the operation input and video display unit T and sends out the captured image taken by the camera unit 61, while generating the person in charge transmission voice sent from the manager terminal 8 from the earphone unit 66 and sending out the person in charge surroundings voice acquired by the microphone unit 63.

[0067] The battery unit 620Z may have a locking portion (not shown) on the back surface thereof so that the battery unit 620Z can be locked to the clothing of the person in charge. As described above, the power of the portable device 6 is supplied from the battery E built in the battery unit 620Z via the communication power cable 650.

[0068] [Administrator terminal] 5 is a schematic functional block diagram of the administrator terminal 8. The administrator terminal 8 is configured to include a control unit 80, a display screen receiving unit 81, an administrator terminal display unit 82, an output voice receiving unit 83, a voice generating unit 84, a transmission voice input unit 85, a transmission voice sending unit 86, a transmission image input unit J, a transmission image sending unit K, a behavior information provision request unit 90, a behavior information receiving unit 91, an administrator terminal storage unit 93, a care receiver information provision request unit 94, a care receiver information receiving unit 95, an input unit 98, and a communication unit 99. The administrator terminal 8 can be an information terminal such as a PC, a smartphone, or a tablet.

[0069] The control unit 80 is configured to include a CPU, an auxiliary storage device which is a non-volatile storage device, and a RAM (not shown). The CPU executes programs and controls each element constituting the administrator terminal 8. The auxiliary storage device stores the programs executed by the CPU, and may be an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0070] The display screen receiving unit 81 receives, as image data, an image display screen 800 on which a captured image 802 is displayed. The image display screen 800 is sent as image data from the business support server 2 via the communication line 11. The image display screen 800 received by the display screen receiving unit 81 is displayed on the manager terminal display unit 82. The manager terminal display unit 82 is composed of a display device, and a display device such as a touch panel LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) is used.

[0071] The output voice receiving unit 83 receives the person in charge surrounding voice 208 sent as voice data from the business support server 2 via the communication line 11. The voice generating unit 84 generates the person in charge surrounding voice 208 received by the output voice receiving unit 83. The voice generating unit 84 may be a speaker or a headphone.

[0072] The person in charge transmission voice to be transmitted to the portable device 6 is input to the transmission voice input unit 85 by the manager operating the manager terminal 8. A microphone is used for the transmission voice input unit 85. The transmission voice sending unit 86 sends the person in charge transmission voice input to the transmission voice input unit 85 as voice data to the business support server 2 via the communication line 11. The person in charge transmission voice may include voice indicating information necessary for the person in charge to perform the business (such as the voice of the manager explaining how to perform the business or voice explaining past cases).

[0073] The transmission image input unit J is configured to input a person in charge transmission image or to compose a person in charge transmission image in response to an operation input to the manager terminal 8 by the manager. The transmission image input unit J may include a camera unit (not shown). In this case, the person in charge transmission image may include video captured by the camera unit. The person in charge transmission image may include video including information necessary for the person in charge to perform the task (such as an image or video showing how to perform the task, an image or video showing past cases, or a video of the manager operating the manager terminal 8).

[0074] The transmission image sending unit K is configured to send the person in charge transmission image inputted or constructed by the transmission image input unit J to the business support server 2.

[0075] The behavior information provision request unit 90 sends a behavior information provision request for requesting person in charge behavior information 200 as data to the business support server 2 via the communication line 11. This behavior information provision request is executed by specifying the object of the request. For example, the behavior information provision request unit 90 specifies the person in charge ID 202 or the person in charge name 204, specifies the person in charge ID 202 in combination with the acquisition time 210, or specifies a combination of the person in charge ID 202, the acquisition time 210, and the fact that the requested object item is the captured image 206. The acquisition time 210 in the behavior information provision request unit 90 may be specified as a specific time, or as a start time and an end time. The staff member behavior information 200 includes a captured image 206 and staff member surrounding voice 208 acquired by the portable device 6. The behavior information receiving unit 91 receives staff member behavior information 200 provided from the business support server 2 in response to a behavior information provision request. When the behavior information receiving unit 91 receives the staff member behavior information 200, the received staff member behavior information 200 is displayed on the manager terminal display unit 82. Specifically, the photographed image 206 and surrounding voice character string 214 generated from the surrounding voice of the person in charge 208 included in the person in charge behavior information 200, as well as other components included in the person in charge behavior information 200, are displayed on the manager terminal display unit 82. In addition, the surrounding voice of the person in charge 208 included in the person in charge behavior information 200 is generated from the voice generating unit 84.

[0076] The manager terminal storage unit 93 uses a non-volatile storage device such as an HDD or SSD. The manager terminal storage unit 93 stores the person in charge behavior information 200 and the like received by the behavior information receiving unit 91. The manager terminal storage unit 93 also stores electronic files such as a care record and a care education training manual created using the captured image 206 and the person in charge surrounding voice 208 included in the person in charge behavior information 200. Then, for example, in response to an operation input by the manager, the transmission image input unit J of the manager terminal 8 may configure the person in charge transmission image as a video including information necessary for the person in charge to perform the task based on the electronic file such as the care record and the care education training manual.

[0077] The care receiver information request unit 94 transmits a request for care receiver information as image data to the business support server 2 via the communication line 11. This request for care receiver information is executed by specifying the object of the request. For example, the care receiver ID 302 or the care receiver's name 304 may be specified, or a combination of the care receiver ID 302 and the requested item being face-to-face time 308 may be specified.

[0078] The care receiver information receiving unit 95 receives the care receiver information 300 provided from the business support server 2 in response to the care receiver information provision request. When the care receiver information receiving unit 95 receives the care receiver information 300, the received care receiver information 300 is displayed on the manager terminal display unit 82.

[0079] The input unit 98 is composed of an input device for executing input, and uses a keyboard, a mouse, a touch panel, etc. The communication unit 99 executes transmission and reception of data to and from the outside.

[0080] [Business support server] 6 is a schematic diagram of the functional blocks of the business support server 2. The business support server 2 is configured to include a control unit 20, a photographed image receiving unit 21, a person in charge surrounding voice receiving unit 22, a behavior information composing unit 23, a behavior information storage unit 24, a display screen composing unit 25, a display screen providing unit 26, a transmission voice receiving unit 27, a call establishment unit 30, a manager absence determining unit 31, a behavior information request receiving unit 32, a behavior information providing unit 33, a position information receiving unit 34, a map information request unit 35, a map information receiving unit 36, a facility interior plan storing unit 37, a voice character generating unit 38, abusive language determining unit 40, a voice analysis unit 41, a face authentication unit 42, a facial expression analysis unit 43, a care receiver information composing unit 44, a care receiver information storage unit 45, a care receiver information provision request receiving unit 46, a care receiver information providing unit 47, a meeting time measuring unit 50, an image and voice analysis unit 51, a communication unit 52, and a transmission image receiving unit L. A server device, a PC, or the like is used for the business support server 2.

[0081] The control unit 20 is configured to include a CPU, a non-volatile storage device, and a RAM (not shown). The CPU executes programs and controls each element that constitutes the business support server 2.

[0082] The captured image receiving unit 21 receives a captured image 802 transmitted as image data from the captured image transmitting unit 62 of the portable device 6 via the communication line 11. The staff surrounding voice receiving unit 22 receives staff surrounding voice 208 transmitted as voice data from the staff surrounding voice transmitting unit 64 of the portable device 6 via the communication line 11.

[0083] The behavior information composition unit 23 composes staff member behavior information 200 including the photographed image 206 received by the photographed image reception unit 21 and the staff member surrounding voice 208 received by the staff member surrounding voice reception unit 22 .

[0084] The person in charge action information 200 is a collection of information that consolidates the records of the actions of the person in charge. By specifying an extraction condition, the record of the action of the person in charge can be extracted from the person in charge action information 200.

[0085] The behavior information configuration unit 23 divides the captured images 206 and the person in charge surrounding sounds 208 into predetermined time intervals, for example, 5 minutes or 10 minutes, and configures the person in charge behavior information 200 by associating the captured images 206, etc. with the acquisition times 210. By configuring the behavior information 200 in this way by the behavior information configuration unit 23, it is possible to extract the captured images 206, etc. corresponding to the acquisition time 210 from the person in charge behavior information 200 by specifying the acquisition time 210.

[0086] Here, Fig. 7A is a diagram showing an example of the configuration of the person in charge action information according to the embodiment. Also, Fig. 7B is a diagram showing an example of the display of the person in charge mental state information included in the person in charge action information according to the embodiment. 7A, the person in charge ID (identification) 202 and the person in charge name 204 included in the person in charge action information 200 are input when logging in to the business support server 2 from the portable device 6 at the start of business, etc. Alternatively, the person in charge ID 202 and the person in charge name 204 are registered in advance in the business support server 2 when the portable device 6 is assigned to a person in charge. The person in charge ID 202 and the person in charge name 204 are stored in the business support server 2 in association with the portable device ID that uniquely identifies the portable device 6. When a captured image 206 and the like are sent from the portable device 6 together with the portable device ID, the business support server 2 identifies the person in charge ID 202 and the person in charge name 204 from the received portable device ID, and configures the person in charge action information 200 so that the person in charge ID 202 and the received captured image 206 and the like are stored. By referencing this person in charge ID 202, etc., it is possible to identify which person in charge the person in charge action information 200 corresponds to. For example, by specifying the person in charge ID 202 or person in charge name 204, it is possible to extract the person in charge action information 200 corresponding to the specified person, and by specifying a combination of the person in charge ID 202 and the acquisition time 210, it is possible to extract the photographed image 206, the person in charge surrounding sound 208, etc. corresponding to that combination from the person in charge action information 200. The components of the person in charge action information 200 shown in FIG. 7A are just an example, and other elements may be included, or some of the elements may be deleted.

[0087] The action information storage unit 24 shown in FIG. 6 stores the person in charge action information 200 configured by the action information configuration unit 23.

[0088] The display screen composition unit 25 processes the photographed image 206 accepted by the photographed image acceptance unit 21 to compose an image display screen 800 .

[0089] Here, Fig. 8 is a schematic diagram showing an example of the configuration of an image display screen 800 displayed on the administrator terminal display unit 82. The image display screen 800 shown in Fig. 8 displays captured images 206 from four portable devices 6. Since one of the four portable devices 6 has its voice communication control operation unit G operated and is in a state of transitioning to a call mode, it is displayed highlighted as an enlarged captured image 804. A red frame is also displayed around the enlarged captured image 804 for emphasis. The three portable devices 6 that are not in the call mode are displayed as captured images 802 of normal size.

[0090] At the right end of this image display screen 800, there is a menu display section 810, which displays a screen selection button 812, a call selection button 814, and a mass call button 816. When the administrator selects the screen selection button 812 from this image display screen 800 displayed on the administrator terminal display section 82 by an operation of the administrator, the selected captured image 206 is enlarged and a call can be made between the portable device 6 and the administrator terminal 8.

[0091] Furthermore, by operation of the administrator, captured images 206 (video and still images) of the selected portable device 6 and surrounding voices 208 of the person in charge are stored in the business support server 2. When the call selection button 814 is selected, a call can be made to the selected portable device 6. When the mass call button 816 displayed on the administrator terminal display unit 82 is selected by operation of the administrator, a call can be made to all the displayed portable devices 6.

[0092] When there is no mobile device 6 in the call mode, for example, all are displayed as normal-sized captured images 802, and a maximum of 20 captured images 206 are displayed per image display screen 800.

[0093] 6 provides the image display screen 800 configured by the display screen configuration unit 25 to the administrator terminal 8. When the administrator terminal 8 accepts the image display screen 800, the image display screen 800 is displayed on the administrator terminal display unit 82 (see FIG. 8).

[0094] The voice transmission receiving unit 27 receives the person in charge voice transmission transmitted as voice data from the voice transmission transmitting unit 86 of the manager terminal 8 via the communication line 11 .

[0095] The call establishment unit 30 sends the person in charge surrounding voice 208 received by the person in charge surrounding voice receiving unit 22 to the manager terminal 8, and sends the person in charge transmission voice received by the transmission voice receiving unit 27 to the portable device 6, thereby establishing a call between the portable device 6 and the manager terminal 8. When the portable device 6 is in the call mode, a call can be made between the portable device 6 and the manager terminal 8.

[0096] The transmission image receiving unit L is adapted to receive a person-in-charge transmission image sent from the transmission image sending unit K of the manager terminal 8 via the communication line 11 as image data.

[0097] The call establishment unit 30 may then send the person in charge surrounding voice 208 received by the person in charge surrounding voice receiving unit 22 to the manager terminal 8, send the person in charge transmission voice received by the transmission voice receiving unit 27 to the mobile device 6, and send an image display screen including the captured image 206 provided by the display screen providing unit 26 to the manager terminal 8 and send the person in charge transmission image received by the transmission image receiving unit L to the mobile device 6, thereby establishing a call (such as a web conference) using audio and video between the mobile device 6 and the manager terminal 8.

[0098] The manager absence determination unit 31 determines whether the manager who operates the manager terminal 8 is absent when the portable device 6 transitions to a call mode. For example, if there is no response from the administrator terminal 8 even after a predetermined time has elapsed after switching to call mode, it may be determined that the person is absent, or if it is detected that an absence button (not shown) provided on the administrator terminal 8 is pressed, it may be determined that the person is absent. When the business support system 1 is made up of a plurality of portable devices 61, 62, ..., 6n, when one portable device 6i among the plurality of portable devices 61, 62, ..., 6n transitions to a call mode, if the manager absence determination unit 31 determines that the manager is absent, control is performed so that calls can be made between the plurality of portable devices 61, 62, ..., 6n.

[0099] The behavior information request receiving unit 32 receives a behavior information request sent as data from the behavior information request unit 90 of the manager terminal 8 via the communication line 11 . The behavior information providing unit 33 provides the person in charge behavior information 20 read from the behavior information storage unit 24 to the manager terminal 8 in response to the behavior information provision request received by the behavior information request receiving unit 32 . When a request for providing behavior information is made by specifying, for example, a person in charge ID, the person in charge behavior information 200 corresponding to the specified person in charge ID is provided. Furthermore, for example, when a combination of a person in charge ID 202 and an acquisition time 210 is specified, a photographed image 206 and person in charge surrounding voice 208 corresponding to the specification are extracted from the person in charge action information 200 and provided. Also, for example, if a combination of the person in charge ID 202, the acquisition time 210, and the requested item being the photographed image 206 is specified, the photographed image 206 corresponding to the specification is extracted from the person in charge action information 200 and provided.

[0100] The location information receiving unit 34 receives the location information 212 sent as data from the location information sending unit 71 of the portable device 6 via the communication line 11 .

[0101] The map information requesting unit 35 sends a map information request to the map information providing server 10 requesting map information about the area around the location information 212 received by the location information receiving unit 34 . The map information receiving unit 36 ​​receives map information provided from the map information providing server 10 in response to a map information request.

[0102] The behavior information configuration unit 23 configures the person in charge behavior information 200 so as to link the position information 212 received by the position information receiving unit 34 to the captured image 206 received by the captured image receiving unit 21 and the person in charge surrounding sound 208 received by the person in charge surrounding sound receiving unit 22 (see FIG. 7A). Since the position information 212 is linked to the captured image 206 and the person in charge surrounding sound 208, by specifying the position information 212, the captured image 206 and the person in charge surrounding sound 208 linked to the position information 212 can be extracted from the person in charge behavior information 200.

[0103] Furthermore, the display screen configuration unit 25 creates a position information image that displays the position information 212 accepted by the position information accepting unit 34 as the position of the portable device 6 on the map of the map information accepted by the map information accepting unit 36. Then, the display screen configuration unit 25 configures an image display screen 800 so that the captured image 206 accepted by the captured image accepting unit 21 and the position information image are displayed on the same screen. The location information image may display the route of the portable device 6 from the start of work through the day on a map. The locations of videos and still images captured by operating a video capture button or a still image capture button (or operation buttons B2, B3) displayed on the operation input and video display section T of the battery unit section 620Z of the portable device 6 may be displayed as thumbnails on a map, and the video or still image may be displayed when the thumbnail display is selected. The thumbnail display refers to the display of a sample image of the video or still image.

[0104] The facility interior plan storage unit 37 stores in advance facility interior plan information in which the floor plan of the facility and the like are recorded. When the position information 212 received by the position information receiving unit 34 is within the facility, the display screen configuration unit 25 creates a position information image so as to display the position of the mobile device 6 on the floor plan of the facility included in the facility interior plan information. Since the position information 212 also includes height information, the display screen configuration unit 25 determines which floor in the facility the position of the mobile device 6 is on, and displays the position of the mobile device 6 on the floor plan corresponding to that floor.

[0105] The voice-to-text generating unit 38 generates text from voice. The voice-to-text generating unit 38 generates the staff surrounding voice 208 received by the staff surrounding voice receiving unit 22 as a surrounding voice character string 214. For this voice-to-text conversion process, existing technology is used, and AI (Artificial Intelligence) technology can also be used.

[0106] The behavior information configuration unit 23 configures the staff behavior information 200 so as to link the surrounding voice character string 214 generated by the voice character generation unit 38 to the photographed image 206 received by the photographed image receiving unit 21 and the staff surrounding voice 208 received by the staff surrounding voice receiving unit 22 (see FIG. 7A). The generated surrounding voice character string 214 may be added to the photographed image 206 as a subtitle.

[0107] The abusive language determination unit 40 is adapted to determine abusive language. This abusive language determination unit 40 determines whether or not the surrounding voice character string 214 generated by the voice character generation unit 38 contains abusive language words registered in advance, and generates abusive language information 216 based on the determination result. The abusive language information 216 includes the abusive language words used and their frequency.

[0108] The behavior information configuration unit 23 configures the person in charge behavior information 200 so as to include the abusive language information 216 generated by the abusive language determination unit 40 (see FIG. 7A).

[0109] The voice analysis unit 41 estimates the mental state by analyzing the voice. The voice analysis unit 41 estimates the mental state of the person in charge by analyzing the person in charge surrounding voice 208 received by the person in charge surrounding voice receiving unit 22 and the surrounding voice character string 214 generated by the voice character generation unit 38, and generates person in charge mental state information 218 based on the estimated mental state of the person in charge.

[0110] This voice analysis unit 41 estimates the mental state of the person in question by comprehensively analyzing factors such as the volume of the person's voice, the intonation of the voice, the pitch of the voice, the number of sighs per given time, the words used, and the frequency of use of negative words. This analysis can also use AI technology, where the person in charge's surrounding voice 208 and the surrounding voice character string 214 are input to a trained generative model to calculate an estimate of the mental state. The voice analysis unit 41 also compares the past and present mental states to determine the stability of the mental state and the direction of change. Furthermore, the voice analysis unit 41 also judges whether the person is in a state where mental health care is necessary.

[0111] The staff mental state information 218 is configured to include vitality, voice clarity, accumulated anger level (accumulated anger value), mental state stability, direction (trend) of change in mental state, and degree of need for mental health care.

[0112] The behavior information configuration unit 23 configures the person in charge behavior information 200 so as to include the person in charge mental state information 218 generated by the voice analysis unit 41 (see FIG. 7A).

[0113] Here, Fig. 7B described above is a diagram showing an example of a display screen of the staff member mental state information 218 included in the staff member behavior information 200. In Fig. 7B, the staff member mental state information 218 for each staff member is displayed side by side. By displaying such a screen on the manager terminal 8, the manager can quickly find staff members who need mental health care.

[0114] 6 authenticates the face of the care recipient appearing in the captured image 206 received by the captured image receiving unit 21, and identifies the name 304 of the care recipient. The business support server 2 has registered features of the facial image of the care recipient in advance, and authenticates the care recipient by comparing the features calculated from the facial image appearing in the captured image 206. This face recognition process uses existing technologies including AI technology. The name 304 and the care recipient ID 302 of the care recipient are registered in the business support server 2 in association with the feature amount of the face image of the care recipient, and when the face recognition unit 42 executes authentication of the care recipient, the name 304 and the care recipient ID 302 of the care recipient are identified. When the face recognition unit 42 identifies the name 304 and the like of the care recipient, it generates the care recipient identification information 215 including the name 304 and the care recipient ID 302 of the care recipient.

[0115] Then, the behavior information configuration unit 23 configures the person in charge behavior information 200 so as to associate the captured image 206, the person in charge surrounding voice 208, and the surrounding voice character string 214 with the care recipient identification information 215 generated by the face authentication unit 42 (see FIG. 7A). With this configuration, for example, it becomes possible to specify the name 304 or the care recipient ID 302 of the care recipient included in the care recipient identification information 215, and extract the captured image 206 or the like in which the specified care recipient appears from the person in charge behavior information 200. In other words, when a combination of a person in charge ID 202 and a care recipient ID 302 is specified and a request to provide the person in charge behavior information 200 (request to provide behavior information) is made, a captured image 206 or the like showing the care recipient corresponding to the care recipient ID 302 is extracted from the person in charge behavior information 200 corresponding to that person in charge ID 202 and provided.

[0116] Then, the facial expression analysis unit 43 shown in FIG. 6 analyzes the facial expression of the care recipient shown in the captured image 206 received by the captured image receiving unit 21, estimates at least one of the emotions of joy, anger, sadness, and happiness, vitality, and satisfaction, and calculates an estimated facial expression value 306 by quantifying the estimation result. Existing technologies including AI technology are used for the facial expression analysis process by the facial expression analysis unit 43. For example, feature values ​​are calculated from the shape and changes of eyebrows, the shape and changes of eyes, cheeks, and lips, the direction of the chin, etc., and analysis is performed based on these feature values.

[0117] The care recipient information configuration unit 44 configures the care recipient information 300 so as to include the name 304 and the care recipient ID 302 of the care recipient identified by the face authentication unit 42 , and the estimated facial expression value 306 determined by the facial expression analysis unit 43 .

[0118] The care recipient information 300 is a collection of information that consolidates the observation records of the care recipients. By specifying a care recipient by name or the like, the observation records of that care recipient can be extracted from the care recipient information 300.

[0119] 9 is a diagram showing an example of the configuration of the care recipient information 300. When the name 304 and the care recipient ID 302 of the care recipient are identified by face recognition using the captured image 206, the care recipient information 300 is configured to store information that serves as an observation record of the care recipient together with the identified name 304 of the care recipient, etc. The components of the care receiver information 300 shown in this figure are only an example, and other elements may be included, or some of the elements may be deleted.

[0120] The care receiver information storage unit 45 shown in FIG. 6 stores the care receiver information 300 configured in the care receiver information configuration unit 44.

[0121] The care recipient information request receiving unit 46 receives a request for providing care recipient information, which is sent from the manager terminal 8, requesting the care recipient information 300. The care recipient information providing unit 47 provides the care recipient information 300, read out from the care recipient information storage unit 45, to the manager terminal 8 in response to the care recipient information request received by the care recipient information request receiving unit 46. When a request for care recipient information is made by designating, for example, the care recipient ID 302, the care recipient information 300 corresponding to the designated care recipient ID 302 is provided. Also, for example, if a combination of the care receiver ID 302 and the requested item being the meeting time 308 is specified, the meeting time 308 corresponding to that care receiver is extracted from the care receiver information 300 and provided.

[0122] The contact time measuring unit 50 measures a contact time 308 during which the care recipient appears in the photographed image 206 received by the photographed image receiving unit 21.

[0123] The care receiver information constructing unit 44 constructs the care receiver information 300 so as to include the contact time 308 measured by the contact time measuring unit 50 (see FIG. 9).

[0124] The image and audio analysis unit 51 performs analysis using AI technology based on the captured image 206 and the person in charge surrounding voice 208. This image and audio analysis unit 51 estimates the person in charge's actions from the captured image 206 received by the captured image receiving unit 21, extracts the person in charge's utterances from the person in charge surrounding voice 208 received by the person in charge surrounding voice receiving unit 22, and judges the person in charge's work evaluation 220 based on the person in charge's actions and utterances. For example, the image and audio analysis unit 51 will give a high rating if the caregiver smiles and speaks kindly to the person being cared for, but will give a low rating if the caregiver speaks little, has a quiet voice, or has a blank look on their face.

[0125] The behavior information configuration unit 23 configures the staff member behavior information 200 so as to include the staff member's work evaluation 220 determined by the image and voice analysis unit 51 (see FIG. 7A). The photographed images 206, the voices around the person in charge 208, the character strings of the voices around the person in charge 214, etc. of the person in charge whose work evaluation 220 is judged to be excellent and the person in charge whose work evaluation 220 is judged to be poor can be extracted from the person in charge behavior information 200 and used as training materials for the person in charge or for creating a manual.

[0126] In addition, the image and audio analysis unit 51 extracts a questioning comment for the care recipient from the staff member surrounding voice 208 received by the staff member surrounding voice receiving unit 22, estimates the care recipient's reaction to the questioning comment from the movements of the care recipient shown in the captured image 206 received by the captured image receiving unit 21, and determines the risk 310 of hearing loss or dementia of the care recipient. For example, if the caregiver repeatedly asks the care recipient the same question, or if the caregiver's voice gets louder with each question, the risk of hearing loss will be determined to be high. In addition, the risk of dementia will be determined by evaluating the time lag between the caregiver's question and the care recipient's response, as well as the consistency of the care recipient's answers to the questions.

[0127] The care receiver information constructing unit 44 constructs the care receiver information 300 so as to include the risk 310 of hearing loss or dementia of the care receiver determined by the image and sound analyzing unit 51 (see FIG. 9).

[0128] The communication unit 52 is adapted to transmit and receive data to and from the outside.

[0129] [Operation of the business support system] Next, the operation of the task support system 1 according to this embodiment will be described.

[0130] FIG. 10 is a diagram for explaining an example of the flow of operations of the task support system 1 performed while a person in charge is performing a task. As described above, the explanation of FIG. 10 is given by taking a nursing care business as an example of a business, as an example only. In other words, the business to which the flow shown in FIG. 10 is applied is not limited to the nursing care business. The flow shown in FIG. 10 can also be applied to, for example, construction businesses, childcare / education businesses, security businesses, maintenance businesses, administrative businesses such as police, ambulances, and fire departments, trading companies, manufacturing businesses, and medical businesses. For example, the positioning of a care recipient who receives a service in a nursing care business can correspond to a person who receives a service in a business to which the business support system according to the present invention is applied, or another person in charge who provides the service.

[0131] 10, a person in charge works while wearing a portable device 6. The portable device 6 captures images of the surroundings with the camera unit 61, acquires sounds of the surroundings with the microphone unit 63, and acquires position information with the position information detection unit 70 (step S10). The portable device 6 then transmits the captured image 206, the person in charge's surrounding voice 208, and the position information 212 to the task support server 2 together with a portable device ID that identifies the portable device 6 (step S11).

[0132] When the business support server 2 receives the photographed image 206, the person in charge surrounding voice 208, and the position information 212 (step S20), the voice character generation unit 38 generates the person in charge surrounding voice 208 as a surrounding voice character string 214 (step S21). Furthermore, the face authentication unit 42 authenticates the care recipient from the captured image 206, identifies the name 304 and the care recipient ID 302 of the care recipient, and generates the care recipient identification information 215 including the care recipient ID 302 and the like (step S22). Thereafter, the behavior information composition unit 23 composes the person in charge behavior information 200 so as to include the photographed image 206, the person in charge surrounding voice 208, the surrounding voice character string 214, the position information 212, and the care receiver specific information 215 (step S23). In this staff member behavior information 200, the photographed image 206, staff member surrounding voice 208, and surrounding voice character string 214 are divided into predetermined time intervals, for example, 5-minute intervals. Also, the photographed image 206, staff member surrounding voice 208, and surrounding voice character string 214 are configured to be associated with the acquisition time 210, position information 212, and care recipient specific information 215, respectively. The thus configured person in charge action information 200 is stored in the action information storage unit 24 (step S24).

[0133] The display screen composition unit 25 processes the photographed image 206 to compose the image display screen 800 (step S25), and provides the composed image display screen 800 to the administrator terminal 8 (step S26). When the administrator terminal 8 receives this image display screen 800 (step S30), this image display screen 800 is displayed on the administrator terminal display unit 82 (step S31).

[0134] The above operations are repeated while the person in charge is working.

[0135] When the person in charge talks to the manager, the voice communication control operation unit G of the headset unit 600 of the portable device 6 is operated, and the portable device 6 transitions to a call mode (step S12). When the business support server 2 detects a notification to transition to call mode (step S27), the call establishment unit 30 operates to enable a call between the mobile device 6 and the administrator terminal 8 (at this time, as described above, a call including video in addition to audio may be enabled). Then, when the voice of the person in charge is detected by the microphone unit 63 of the portable device 6, the voice of the person in charge is sent to the business support server 2 (step S13), and the business support server 2 receives this voice (step S28). At this time, when the surroundings of the person in charge are photographed by the camera unit 61 of the portable device 6, the photographed image sending unit 62 sends the captured image of the surroundings of the person in charge to the business support server 2, and the photographed image receiving unit 21 of the business support server 2 receives this captured image of the surroundings of the person in charge.

[0136] Then, the business support server 2 transmits the received voice and the captured image of the person in charge surroundings to the manager terminal 8 (step S29). In the manager terminal 8, the received voice is output from the voice generating unit 84 (step S32) and reaches the ears of the manager. At this time, the display screen receiving unit 81 of the manager terminal 8 receives an image display screen on which the captured image of the person in charge surroundings is displayed from the business support server 2. Then, the manager terminal display unit 82 displays an image display screen on which the captured image of the person in charge surroundings received by the display screen receiving unit 81 is displayed. This allows the manager to visually confirm the captured image of the person in charge surroundings using the manager terminal display unit 82. Note that the captured image of the person in charge surroundings may include not only the captured image captured by the camera unit 61 but also an image (photograph) edited by the person in charge on the portable device 6, etc.

[0137] When the manager utters a voice, the voice is input to the transmission voice input unit 85 of the manager terminal 8 and is sent from the transmission voice sending unit 86 to the business support server 2 (step S33). Furthermore, the manager terminal 8 may be configured so that the transmission image sending unit K sends the person in charge transmission image input or configured in the transmission image input unit J to the business support server 2 in response to an operational input by the manager. Furthermore, when the business support server 2 receives the voice of the manager (step S200), the voice is sent to the portable device 6 (step S201). At this time, when the business support server 2 receives a person in charge transmission image, the person in charge transmission image may be sent from the business support server 2 to the portable device 6. When the portable device 6 receives the voice of the manager, the voice is generated from the earphone unit 66 (step S14) and reaches the ears of the person in charge. The call establishment unit 30 of the business support server 2 controls the voice communication between the portable device 6 and the manager terminal 8 to establish a call between the person in charge and the manager (at this time, as described above, a call including video in addition to voice may be possible).

[0138] When the manager checks the record of the person in charge's actions, he / she specifies the person in charge ID 202 or person in charge name 204, as well as requested items such as captured image 206 and sound around the person in charge 208, acquisition time 210, location information 212, etc., and sends a request for providing action information from the manager terminal 8 to the business support server 2 (step S34).

[0139] When the business support server 2 receives this request for providing behavioral information (step S202), it retrieves the person in charge behavioral information 200 corresponding to this request from the behavioral information storage unit 24 and provides it to the manager terminal 8 (step S203).

[0140] When the manager terminal 8 receives the staff member behavior information 200 (step S35), the received staff member behavior information 200 is displayed on the manager terminal display unit 82 (step S36). In addition, when the staff member surrounding voice 208 included in the staff member behavior information 200 is received, it is generated from the voice generating unit 84.

[0141] 11 is a diagram for explaining an example of a flow in which analysis is performed based on the configured staff member behavior information 200 in the business support system 1. The example in this figure shows a flow in which staff member behavior information 200 obtained during a predetermined period, such as one day's worth, is accumulated in the business support server 2 through the work of the staff member, and then the staff member behavior information 200 obtained during the predetermined period is used for collective analysis. However, the present invention is not limited to this example, and real-time analysis may be performed using staff member action information 200 obtained during the staff member's work. Furthermore, as described above, the explanation of FIG. 11 is given by taking a nursing care business as an example of a business as only one example. In other words, the business to which the flow shown in FIG. 11 is applied is not limited to the nursing care business. The flow shown in FIG. 11 can also be applied to, for example, construction businesses, childcare / education businesses, security businesses, maintenance businesses, administrative businesses such as police, ambulances, and fire departments, trading companies, manufacturing businesses, and medical businesses. For example, the positioning of a care recipient who receives a service in a nursing care business can correspond to a person who receives the service in a business to which the business support system according to the present invention is applied, or another person in charge of providing the service.

[0142] 11, first, the business support server 2 reads out the already configured staff member behavior information 200 from the behavior information storage unit 24 (step S40). The business support server 2 judges abusive language using the abusive language judgment unit 40 based on the surrounding voice character string 214 included in the read staff member behavior information 200, and generates abusive language information 216 (step S41). Next, the business support server 2 generates the person in charge mental state information 218 by the voice analysis unit 41 based on the person in charge surrounding voice 208 and the surrounding voice character string 214 included in the person in charge behavior information 200 (step S42). Next, the task support server 2 determines the task evaluation 220 of the staff member by the image and audio analysis unit 51 based on the captured image 206 and the staff member surrounding audio 208 included in the staff member behavior information 200 (step S43). The business support server 2 constructs the staff member behavior information 200 by the behavior information construction unit 23 so as to include the obtained abusive language information 216, staff member mental state information 218, and staff member work evaluation 220, and stores and updates the constructed staff member behavior information 200 in the behavior information storage unit 24 (step S44).

[0143] Next, the business support server 2 identifies the name 304, etc. of the care recipient by the face authentication unit 42 based on the photographed image 206 included in the staff member behavior information 200 (step S45). In addition, the business support server 2 estimates at least one of the emotions of joy, anger, sadness, and happiness, vitality, and satisfaction using the facial expression analysis unit 43 based on the captured image 206 included in the staff member behavior information 200, and calculates an estimated facial expression value 306 that quantifies the estimation result (step S46). Next, the business support server 2 measures the contact time 308 by the contact time measuring unit 50 based on the photographed image 206 included in the person in charge behavior information 200 (step S47). Thereafter, the business support server 2 determines the risk 310 of hearing loss or dementia of the care recipient by the image and sound analysis unit 51 based on the captured image 206 and the person in charge's surrounding sound 208 included in the person in charge behavior information 200 (step S48). Finally, the business support server 2 constructs care recipient information 300 by the care recipient information construction unit 44 so as to include the obtained care recipient's name 304, estimated facial expression value 306, face-to-face time 308, and the care recipient's risk of hearing loss or dementia 310 (step S49), and stores the constructed care recipient information 300 in the care recipient information storage unit 45 (step S50).

[0144] When the manager checks the record of the person in charge's behavior, the manager specifies the required items such as the person in charge's mental state information 218 and the work evaluation 220 along with the person in charge's ID 202 or name 204, and sends a request for providing behavior information from the manager terminal 8 to the business support server 2 (step S60).

[0145] When the business support server 2 receives this request for providing behavior information (step S51), it retrieves the person in charge behavior information 200 corresponding to this request from the behavior information storage unit 24 and provides it to the manager terminal 8 (step S52).

[0146] When the manager terminal 8 receives the staff member action information 200 (step S61), the received staff member action information 200 is displayed on the manager terminal display unit 82 (step S62).

[0147] When the manager checks the observation records of the care recipient, the manager specifies the care recipient ID 302 or the care recipient's name 304, as well as requested items such as meeting time 308, and sends a request for care recipient information from the manager terminal 8 to the business support server 2 (step S63).

[0148] When the business support server 2 receives this request for providing care recipient information (step S53), it retrieves the care recipient information 300 corresponding to this request from the care recipient information storage unit 45 and provides it to the manager terminal 8 (step S54).

[0149] When the manager terminal 8 accepts the care receiver information 300 (step S64), the accepted care receiver information 300 is displayed on the manager terminal display unit 82 (step S65).

[0150] Next, the effects of this embodiment will be described.

[0151] According to this embodiment, the actions of the person in charge are recorded by constructing and storing person in charge action information 200, which includes images of the person in charge's surroundings captured by the portable device 6 and audio of the person in charge's surroundings acquired by the portable device 6.

[0152] In addition, since an image captured by the portable device 6 is displayed on the manager terminal 8 and a call can be made between the portable device 6 and the manager terminal 8, the manager who operates the manager terminal 8 can provide advice and instructions in real time to the person wearing the portable device 6, and can support the person in his / her work. In particular, by using the operation input and image display unit T of the portable device 6, it becomes possible to instantly exchange images in both directions between the person in charge who operates the portable device 6 and the manager who operates the manager terminal 8 and checks the manager terminal display unit 82. This allows the manager and the person in charge to exchange images and voices with each other, which can facilitate remote confirmation of the execution of the work by the manager and simplification of the person in charge's work report creation.

[0153] As described above, the operation input and video display unit T enables instant, two-way video and audio communication between the worker who operates the portable device 6 and the manager who operates the manager terminal 8 and checks the manager terminal display unit 82.

[0154] In addition, a person wearing a portable device 6 can remotely participate in an online conference using various applications via the portable device 6 with an administrator operating the administrator terminal 8 or other persons wearing other portable devices 6.

[0155] In particular, for example, a person in charge wearing the portable device 6 can check a photo displayed on the operation input and video display unit T and store the photo in the portable device storage unit 73. Furthermore, a person in charge wearing the portable device 6 can check a video displayed on the operation input and video display unit T and store the video in the portable device storage unit 73.

[0156] For example, the person in charge can create a folder in the portable device storage unit 73 by operating the operation input / video display unit T of the battery unit 620Z of the portable device 6, and store information such as videos, photos, and notes in the portable device storage unit 73 of the battery unit 620Z of the portable device 6. Note that the operation input / video display unit T of the battery unit 620Z of the portable device 6 may be configured to display a clock so that the person in charge can check the time.

[0157] In addition, in terms of training for the business, information such as videos, photos, and / or notes related to training and technology transfer for carrying out the business is stored in the portable device storage unit 73 of the battery unit 620Z of the portable device 6. This allows the person in charge to carry the portable device 6 and check this information at any time by operating the operation input and video display unit T to display it.

[0158] For example, information such as videos, photos, and / or notes related to the training and skill transfer required to carry out the work may be stored in the portable device storage unit 73 of the battery unit 620Z of the portable device 6.

[0159] Furthermore, the person in charge's action information 200 stored in the business support server 2 can be displayed on the manager's terminal 8 to confirm the past actions of the person in charge.

[0160] Furthermore, according to this embodiment, since the earphone unit 66 is configured as a bone conduction earphone, the ear of the person in charge who wears the portable device 6 is not blocked by the earphone. The person in charge can receive the sound transmitted from the manager terminal 8 through the bone conduction earphone and hear the surrounding sounds with his / her own ears. In this way, both the sound transmitted from the manager terminal 8 and the surrounding sounds can be clearly heard.

[0161] As described above, the earphone unit 66 is not limited to a bone conduction type earphone, but may be another type of earphone, such as a canal type earphone or an inner-ear type earphone.

[0162] According to this embodiment, the camera unit 61, microphone unit 63, and earphone unit 66 are integrated into the headset unit 600 and are configured to be worn on the head of the person in charge, so that the person in charge wearing the headset unit 600 can use both hands freely. Also, by aligning the shooting direction of the camera unit 61 with the line of sight of the person in charge, an image close to the field of vision of the person in charge can be captured.

[0163] According to this embodiment, in the camera stop mode, image capturing by the camera unit 61 and acquisition of audio by the microphone unit 63 are stopped. In situations where privacy must be respected, such as bathing care, the camera can be switched to the camera stop mode to stop image and audio acquisition. In addition, a notification sound notifying the user of the camera stop mode is emitted from the earphone unit 66, so that the person in charge can recognize that the camera stop mode is in effect.

[0164] Furthermore, according to this embodiment, when the portable device 6 switches to a call mode, a call can be made between the portable device 6 and the administrator terminal 8. Therefore, when the person in charge wants to receive advice or instructions, the person in charge can switch to the call mode and receive advice, etc. from the administrator operating the administrator terminal 8.

[0165] Furthermore, since the captured image 206 of the mobile device 6 that is making a call is highlighted, the manager operating the manager terminal 8 can easily recognize the captured image 206 and give advice or the like.

[0166] Here, as already described, the call establishment unit 30 of the business support server 2 may send the person in charge surrounding voice 208 received by the person in charge surrounding voice receiving unit 22 to the manager terminal 8 and send the person in charge transmission voice received by the transmission voice receiving unit 27 to the mobile device 6, and may also send an image display screen including the captured image 206 provided by the display screen providing unit 26 to the manager terminal 8 and send the person in charge transmission image received by the transmission image receiving unit L to the mobile device 6, thereby establishing a call (such as a web conference) using audio and video between the mobile device 6 and the manager terminal 8.

[0167] For example, so that the portable device 6 and the manager terminal 8 transmit audio and video bidirectionally via the business support server 2 and the communication line 11, the control unit 60 of the portable device 6 may control the captured image sending unit 62, the person in charge surrounding audio sending unit 64, the person in charge transmission audio receiving unit 65, the person in charge transmission image receiving unit H, the microphone unit 63, the earphone unit 66, the camera unit 61, and the operation input and video display unit T to display the person in charge transmission image sent from the manager terminal 8 on the operation input and video display unit T and send out the captured image taken by the camera unit 61, and to generate the person in charge transmission audio sent from the manager terminal 8 from the earphone unit 66 and send out the person in charge surrounding audio acquired by the microphone unit 63. This enables voice and video conversations (such as web conferences) between the person operating the mobile device 6 and the administrator operating the administrator terminal 8, enabling efficient support in carrying out the work.

[0168] In this way, the portable device 6 of the business support system 1 makes it possible to secure and record knowledge such as education, knowledge, experience, and cases necessary to carry out the business, protect employees, secure evidence, and improve governance.

[0169] Furthermore, according to this embodiment, the person in charge behavior information 200 is configured so that the captured image 206 and the person in charge surrounding sound 208 are linked to the position information 212, so that the position where the image was captured and the position where the sound was acquired can be identified.

[0170] In addition, since the position of the portable device 6 is displayed on a map, the position of the portable device 6 can be easily recognized.

[0171] Furthermore, according to this embodiment, the person in charge surrounding voice 208 is generated as a surrounding voice character string 214, and the person in charge action information 200 is configured so as to be linked to the photographed image 206 and the person in charge surrounding voice 208, so that the voice corresponding to the photographed image 206 and the person in charge surrounding voice 208 can be recognized as a character string. Furthermore, when creating a manual or record booklet based on the person in charge action information 200, there is no need to transcribe the text, which can save labor.

[0172] Furthermore, according to this embodiment, abusive language information 216 is created based on the result of judging whether abusive language has been used, and the staff member action information 200 is configured to include the abusive language information 216. By referring to this abusive language information 216, it is possible to prevent abusive language and improve business operations.

[0173] Furthermore, according to this embodiment, the mental state of the person in charge is estimated by analyzing the voice, which can be utilized for mental health care of the person in charge.

[0174] Furthermore, according to this embodiment, the face of the care recipient captured in the captured image 206 is authenticated to identify the name 304 of the care recipient, and the facial expression of the care recipient is analyzed to estimate at least one of the emotions of joy, anger, sadness, and happiness, vitality, and satisfaction to calculate an estimated facial expression value. By referring to the estimated facial expression value, the customer satisfaction level of each care recipient can be determined.

[0175] Furthermore, according to this embodiment, the contact time 308 for each care recipient is detected by the business support server 2. This makes it possible to check the bias in the provision of services and to utilize this for subsequent business improvements.

[0176] Furthermore, according to this embodiment, the business support server 2 performs an analysis based on the captured image 206 and the surrounding voice of the person in charge 208, thereby determining the business evaluation 220 of the person in charge. The comparison of the images and voices of the person in charge who received a good evaluation and a bad evaluation can be utilized for creating an education and training manual, etc.

[0177] Furthermore, according to this embodiment, the risk 310 of hearing loss or dementia of the care recipient can be determined by performing an analysis based on the captured image 206 and the surrounding sound 208 of the caregiver, which can be utilized for early detection and subsequent care.

[0178] Furthermore, according to this embodiment, when the portable device 6 transitions to a call mode, if it is determined that the manager operating the manager terminal 8 is absent, multiple personnel wearing the portable device 6 can make calls among themselves, so that advice can be obtained from other personnel. In this case, the business support server 2 may display images taken by the camera units 61 of the portable devices 6 of multiple personnel on the operation input and image display units T of the portable devices 6, so that the images can be shared.

[0179] Furthermore, according to this embodiment, the actions of the person in charge are recorded by constructing and storing person in charge action information 200, which includes images of the person in charge's surroundings captured by the portable device 6 and audio of the person in charge's surroundings acquired by the portable device 6.

[0180] The present invention is not limited to the above embodiment and can be modified as appropriate. For example, the functions of the business support server 2 may be integrated into the manager terminal 8. In addition, the behavior information storage unit 24 and the care recipient information storage unit 45 included in the business support server 2 may be constructed as an external server.

[0181] In addition, the business support system 1 according to the above embodiment has been described using a nursing care business as an example of business, but is not limited to the nursing care business and can be applied to other businesses. That is, as described above, the business support system according to the present invention can be applied to other businesses such as construction businesses, childcare and education businesses, security businesses, maintenance businesses, police, administrative businesses such as ambulances and fire departments, trading companies, manufacturing businesses, and medical businesses.

[0182] For example, when the business support system 1 is applied to a childcare / education business such as a nursery school or kindergarten, or a construction business, in the nursing care business, the captured image 206 taken by the portable device 6 will show the person being cared for, while in the childcare / education business, the captured image 206 will show nursery school or kindergarten children, and in the construction business, workers and the like will be shown. Then, in face recognition using the captured image 206, authentication is performed for the nursery school or kindergarten children and workers. When the face recognition unit 42 of the business support server 2 identifies the names and IDs of the nursery school or kindergarten children and workers, nursery school or kindergarten children identification information and worker identification information corresponding to the care recipient identification information 215 including the names and IDs are generated.

[0183] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are within the scope of the invention and its equivalents as described in the claims, as well as the scope and spirit of the invention. [Explanation of symbols]

[0184] 1. Business Support System 2. Business support server 61, 62, ..., 6 i , , 6 n , 6 mobile devices 8. Administrator terminal 10. Map information server 11. Communication Lines 20 Control section 21. Photographic Image Reception Section 22 Voice reception area around the person in charge 23 Behavioral Information Component 24 Behavior information storage unit 25 Display screen component 26 Display screen provision section 27 Voice transmission reception section 30 Call establishment unit 31 Administrator absence determination unit 32 Behavioral information request reception unit 33 Behavior Information Provision Department 34 Location information reception section 35 Map information request section 36 Map Information Reception Department 37 Facility Drawing Storage Section 38 Voice to Text Generation Unit 40. Abusive Language Judgment Department 41 Voice Analysis Section 42 Face Recognition Unit 43 Facial expression analysis department 44 Care recipient information configuration section 45 Care recipient information storage unit 46 Care recipient information request reception department 47 Care recipient information provision department 50 Face-to-face time measurement section 51 Image and audio analysis section 52 Communications Department L Transmission image reception section 200 Personnel Action Information 202 Person in charge ID 204 Name of person in charge 206 Images 208 Voices around the person in charge 210 Acquisition time 212 Location information 214 Ambient Audio String 215 Care recipient specific information 216 Abusive Information 218 Personnel Mental Status Information 220 Business Evaluation 300 Information on care recipients 302 Care recipient ID 304 Name of the person receiving care 306 Estimated Facial Expression Values 308 Face-to-face time 310 Risk of hearing loss or dementia 60 Control section 61 Camera Division 62 Photographed image transmission unit 63 Microphone section 64 Personnel Surrounding Audio Transmission Unit 65 Personnel Communication Voice Reception Section 66 Earphone section 70 Location information detection unit 71 Location information sending unit T Operation input and video display section H Person in charge transmission image reception section 73 Mobile device memory 74 Communications Department E Battery section B1 Power Button B2, B3 Operation buttons H Person in charge transmission image reception section 600 Headset section G Voice communication control unit 612 Connection section 614 Widening section 616 Ear hook N1 Communication power cable connection terminal 619 Head 620Z Battery unit N2 Communication power cable connection terminal 650 Communication Power Cable 80 Control section 81 Display screen reception section 82 Administrator terminal display unit 83 Output audio reception unit 84 Sound generation section 85 Transmission voice input section 86 Transmission voice transmission unit 90 Behavioral Information Request Department 91 Behavioral Information Reception Department 93 Administrator terminal memory unit 94 Care recipient information request department 95 Care Recipient Information Reception Department 98 Input section 99 Communications Department 800 image display screen 802 Normal size photo 804 Enlarged image 810 Menu display section 812 Screen selection button 814 Call selection button 816 Mass Call Button

Claims

1. A business support system that monitors and manages the actions of personnel engaged in a business and supports the personnel in carrying out the business, a portable device worn by the person in charge; an administrator terminal that displays images captured by the portable device and is capable of communicating with the portable device; a business support server that configures and stores staff action information based on data acquired by the portable device and controls communication between the portable device and the manager terminal; the portable device and the administrator terminal are connected to the business support server via a communication line; the portable device includes a headset unit, a battery unit, and a communication power cable connecting the headset unit and the battery unit; The headset unit includes: A camera unit that captures images of the person in charge and acquires the captured images; a microphone unit that receives an input of a voice around the person in charge and generates a voice data of the voice around the person in charge; an earphone unit that generates a voice to be transmitted to the person in charge based on the voice data; a voice communication control operation unit that is operated by the person in charge to control the operation of the microphone unit and the earphone unit to communicate voice data, The battery unit includes: a captured image sending unit that sends an image captured by the camera unit to the business support server via the communication line; a person in charge surrounding voice sending unit that sends the person in charge surrounding voice acquired by the microphone unit to the business support server via the communication line; an operation input and image display unit that receives operation input from the person in charge and displays an image; a person in charge voice receiving unit that receives a person in charge voice to be generated from the earphone unit, the person in charge voice being voice data transmitted from the business support server; a person in charge image receiving unit that receives a person in charge image to be displayed on the operation input and video display unit, the person in charge image receiving unit being image data sent from the business support server; a battery section that supplies driving power to operate the headset section via the communication power cable to the headset section and also supplies driving power to operate the battery unit section; A control unit for controlling an operation of the portable device, so that the portable device and the manager terminal transmit audio and video bidirectionally via the business support server and the communication line, the control unit controls the captured image sending unit, the person in charge surrounding audio sending unit, the person in charge transmission audio receiving unit, the person in charge transmission image receiving unit, the microphone unit, the earphone unit, the camera unit, and the operation input and video display unit to display the person in charge transmission image sent from the manager terminal on the operation input and video display unit and send out the captured image taken by the camera unit, generate the person in charge transmission audio sent from the manager terminal from the earphone unit, and send out the person in charge surrounding audio acquired by the microphone unit, The administrator terminal, a display screen receiving unit that receives an image display screen on which the captured image is displayed, the image display screen being sent from the business support server; a manager terminal display unit that displays the received image display screen; an output voice receiving unit that receives the surrounding voice of the person in charge sent from the business support server; a voice generating unit that generates the received surrounding voice of the person in charge; a transmission voice input unit to which the person in charge's transmission voice is input; a voice transmission unit that transmits the input person-in-charge voice to the business support server; a behavior information provision request unit that transmits a behavior information provision request for requesting the person in charge behavior information to the business support server; a behavior information receiving unit that receives the person in charge behavior information provided from the business support server in response to the behavior information provision request; a transmission image input unit to which the person in charge transmission image is input or which configures the person in charge transmission image; a transmission image sending unit that sends the person-in-charge transmission image input or configured to the business support server; and The business support server includes: a photographed image receiving unit that receives the photographed image sent from the photographed image sending unit; a person in charge surrounding voice receiving unit that receives the person in charge surrounding voice transmitted from the person in charge surrounding voice transmitting unit; an action information configuration unit that configures the person in charge action information including the received photographed image and the person in charge surrounding sound; An action information storage unit that stores the configured person in charge action information; a display screen configuration unit that processes the photographed image accepted by the photographed image acceptance unit and configures the image display screen; a display screen providing unit that provides the configured image display screen to the administrator terminal; a voice transmission receiving unit that receives the person-in-charge voice transmitted from the voice transmission sending unit; a transmission image receiving unit that receives the person-in-charge transmission image sent from the transmission image sending unit; a call establishment unit that establishes a call by voice and video between the portable device and the manager terminal by sending the person in charge surrounding voice received by the person in charge surrounding voice receiving unit to the manager terminal, sending the person in charge transmission voice received by the transmission voice receiving unit to the portable device, sending an image display screen including the photographed image provided by the display screen providing unit to the manager terminal, and sending the person in charge transmission image received by the transmission image receiving unit to the portable device; a behavior information request receiving unit that receives the behavior information request sent from the behavior information request unit; a behavior information providing unit that provides the person in charge behavior information read from the behavior information storage unit in response to the received behavior information provision request to the manager terminal, When the action information receiving unit receives the person in charge action information, the received person in charge action information is displayed on the manager terminal display unit, The mobile device comprises: a location information detection unit that receives radio waves from a satellite positioning system and calculates location information; a location information sending unit that sends the calculated location information to the business support server; The business support server includes: a location information receiving unit that receives the location information sent from the location information sending unit; a map information request unit that transmits a map information request for requesting map information of the surrounding area of ​​the received position information; a map information receiving unit that receives the map information provided in response to the map information request, The behavior information configuration unit The staff member behavior information is configured so as to link the location information received by the location information receiving unit to the photographed image received by the photographed image receiving unit and the staff member surrounding sound received by the staff member surrounding sound receiving unit; The display screen configuration unit includes: creating a position information image that displays the position information received by the position information receiving unit as the position of the portable device on the map of the map information received by the map information receiving unit; The image display screen is configured so that the photographed image and the position information image accepted by the photographed image accepting unit are displayed on the same screen. A business support system comprising:

2. The person in charge transmission image includes an image including information necessary for the person in charge to perform the task.

2. The business support system according to claim 1.

3. The person in charge transmission voice includes a voice indicating information necessary for the person in charge to perform the task.

3. The business support system according to claim 2.

4. Video, photograph, and / or memo information related to training and / or technology transfer for carrying out the business is stored in a portable device storage unit of the battery unit of the portable device.

2. The business support system according to claim 1.

5. 2. The task support system according to claim 1, wherein the earphone unit is configured by a bone conduction type earphone, a canal type earphone, or an inner-ear type earphone.

6. The camera unit, the microphone unit, and the earphone unit are integrated as a headset, and the headset is configured to be worn on the head of the person in charge.

2. The business support system according to claim 1.

7. the mobile device has a camera off mode; In the camera stop mode, The image capturing by the camera unit and the acquisition of sound by the microphone unit are stopped, and a notification sound is generated from the earphone unit to notify that the camera is in the camera stop mode.

2. The business support system according to claim 1.

8. the mobile device has a talk mode; In the call mode, A call can be made between the portable device and the administrator terminal, and the image captured by the camera unit is highlighted on the display unit of the administrator terminal.

8. The business support system according to claim 7.

9. The business support server includes: A voice-to-text generator that generates text from voice, The voice-to-text generation unit generating the person in charge surrounding voice received by the person in charge surrounding voice receiving unit as a surrounding voice character string; The behavior information configuration unit The staff action information is configured so as to link the surrounding voice character string generated by the voice character generation unit to the photographed image received by the photographed image receiving unit and the staff surrounding voice received by the staff surrounding voice receiving unit.

2. The business support system according to claim 1.

10. The business support server includes: An abusive language determination unit that determines abusive language, The abusive language determination unit is determining whether or not the surrounding voice character string generated by the voice character generation unit contains abusive language words registered in advance, and generating abusive language information based on the determination result; The behavior information configuration unit The staff member action information is configured to include the abusive language information generated by the abusive language determination unit.

10. The business support system according to claim 9.

11. The business support server includes: A voice analysis unit that analyzes voice and estimates a mental state, The voice analysis unit is an analysis unit that analyzes the person in charge's surrounding voice received by the person in charge surrounding voice receiving unit and the surrounding voice character string generated by the voice character generating unit to estimate the mental state of the person in charge, and generates person in charge mental state information based on the estimated mental state of the person in charge; The behavior information configuration unit The staff member behavior information is configured to include the staff member mental state information generated by the voice analysis unit.

11. The business support system according to claim 10.

12. The business support server includes: a face authentication unit that authenticates a face of the care recipient appearing in the photographed image received by the photographed image receiving unit and identifies a name of the care recipient; an expression analysis unit that analyzes the facial expression of the care recipient shown in the captured image received by the captured image receiving unit, estimates at least one of emotions such as joy, anger, sadness, and happiness, vitality, and satisfaction, and calculates an estimated expression value by quantifying the estimation result; a care recipient information configuration unit that configures care recipient information to include the name of the care recipient identified by the face authentication unit and the estimated facial expression value calculated by the facial expression analysis unit; a care recipient information storage unit configured to store the care recipient information; a care recipient information provision request receiving unit that receives a care recipient information provision request that requests the care recipient information transmitted from the manager terminal; a care receiver information providing unit that provides the care receiver information read from the care receiver information storage unit to the manager terminal in response to the received care receiver information provision request, The administrator terminal, a care recipient information provision request unit configured to send the care recipient information provision request to the business support server; a care receiver information receiving unit that receives the care receiver information provided from the business support server in response to the care receiver information provision request, When the care recipient information receiving unit receives the care recipient information, the received care recipient information is displayed on the manager terminal display unit.

12. The business support system according to claim 11.

13. The business support server includes: a contact time measurement unit that measures a contact time during which the care recipient appears in the photographed image received by the photographed image receiving unit, The care recipient information configuration unit, The care receiver information is configured to include the face-to-face time measured by the face-to-face time measuring unit.

13. The business support system according to claim 12.

14. The business support server includes: an image and audio analysis unit that performs analysis based on the captured image and the sound around the person; The image and sound analysis unit an action of the person in charge is estimated from the photographed image received by the photographed image receiving unit, a statement of the person in charge is extracted from the voice around the person in charge received by the voice around the person in charge receiving unit, and a work evaluation of the person in charge is determined based on the action and the voice of the person in charge; The behavior information configuration unit The staff action information is configured to include the work evaluation determined by the image and voice analysis unit.

12. The business support system according to claim 11.

15. The business support server includes: an image and audio analysis unit that performs analysis based on the captured image and the sound around the person; The image and sound analysis unit an action of the person in charge is estimated from the photographed image received by the photographed image receiving unit, a statement of the person in charge is extracted from the voice around the person in charge received by the voice around the person in charge receiving unit, and a work evaluation of the person in charge is determined based on the action and the voice of the person in charge; The behavior information configuration unit The person in charge action information is configured to include the work evaluation determined by the image and voice analysis unit, The image and sound analysis unit extracting a questioning statement for the care recipient from the person in charge surrounding voice received by the person in charge surrounding voice receiving unit, estimating the reaction of the care recipient to the questioning statement from the movement of the care recipient shown in the captured image received by the captured image receiving unit, and determining a risk of hearing loss or dementia of the care recipient; The care recipient information configuration unit, The care recipient information is configured to include the risk of hearing loss or dementia of the care recipient determined by the image and sound analysis unit.

14. The business support system according to claim 13.

16. a plurality of said mobile devices; The business support server includes: The system has an administrator absence determination unit that determines the absence of an administrator who operates the administrator terminal, and when one of the plurality of portable devices transitions to a call mode, if the administrator absence determination unit determines that the administrator is absent, calls can be made between the plurality of portable devices.

16. The business support system according to claim 15.

Images