Information processing method, information processing device, information processing system, and program

Abstract

This information processing method is performed by a computer and includes: a step for acquiring information pertaining to a person; a step for analyzing the information pertaining to the person; a step for detecting a trigger event for recording information on the basis of the analysis result of the information pertaining to the person; a step for acquiring site information when the trigger event has occurred; and a step for associating and recording the site information when the trigger event has occurred and the information pertaining to the person.

Description

Information processing method, information processing device, information processing system, and program

[0001] The present disclosure relates to an information processing method, an information processing device, an information processing system, and a program.

[0002] Patent Literature 1 discloses a device for managing workload information. The device includes a per-scene load calculation unit that calculates a workload value, which is an estimate of the worker's workload in an extracted behavior scene, based on time-series biometric information from the start time to the end time of the worker's behavior in the behavior scene. The device also includes a per-scene load accumulation unit that accumulates worker information, scene information for identifying the behavior scene, and the workload value of the behavior scene in a format in which the information is associated with each other, and a multi-person load comparison unit that compares the workload value of one behavior scene with the workload values ​​of multiple people that are accumulated in the per-scene load accumulation unit and associated with the same scene information as the scene information of the one behavior scene, to determine whether the workload value is higher than a predetermined condition.

[0003] International Publication No. 2017 / 187666

[0004] However, with the device disclosed in Patent Document 1, it is difficult to refer to important information at the work site later.

[0005] The present disclosure aims to provide an information processing method and the like that allows important information at a work site to be easily referenced later.

[0006] An information processing method according to one aspect of the present disclosure is an information processing method performed by a computer, and includes the steps of acquiring information about a person, analyzing the information about the person, detecting a trigger event that triggers information recording based on the analysis results of the information about the person, acquiring on-site information when the trigger event occurred, and linking and recording the on-site information when the trigger event occurred and the information about the person.

[0007] An information processing device according to one aspect of the present disclosure includes a person information acquisition unit that acquires information about a person, an analysis unit that analyzes the information about the person, an event detection unit that detects a trigger event that triggers information recording based on the analysis results by the analysis unit, a site information acquisition unit that acquires site information when the trigger event occurs, and a recording unit that links and records the site information when the trigger event occurs and information about the person.

[0008] An information processing system according to one aspect of the present disclosure is an information processing system comprising an information collecting robot that collects information about people, and an information processing device that processes the information obtained by the information collecting robot, wherein the information processing device comprises a person information acquisition unit that acquires information about the people sent from the information collecting robot, an analysis unit that analyzes the information about the people, an event detection unit that detects a trigger event that triggers information recording based on the analysis results by the analysis unit, a site information acquisition unit that acquires site information when the trigger event occurs, and a recording unit that links and records the site information when the trigger event occurs and information about the people.

[0009] A program according to one aspect of the present disclosure is a program for causing a computer to execute the above-described information processing method.

[0010] According to the information processing method and the like according to one aspect of the present disclosure, important information at a work site can be easily referenced later.

[0011] FIG. 1 is a schematic diagram showing an information processing system according to an embodiment. FIG. 2 is a diagram showing a block configuration of an information processing system according to an embodiment and an information processing device included in the information processing system. FIG. 3 is a diagram showing an overview of processing executed in the information processing system according to an embodiment. FIG. 4 is a diagram showing an example of emotion analysis performed by the information processing device. FIG. 5 is a diagram showing an example in which on-site information and human utterance information when a trigger event occurs are recorded in a linked state. FIG. 6 is a flowchart showing an information processing method according to an embodiment. FIG. 7 is a schematic diagram showing an information processing system according to a modified embodiment. FIG. 8 is a diagram showing a block configuration of an information processing system according to a modified embodiment and an information processing device included in the information processing system. FIG. 9 is a diagram showing an overview of processing executed in the information processing system according to a modified embodiment.

[0012] Building and construction sites are often dangerous places, and workers often have difficulty operating information terminals such as smartphones, so they communicate with each other through conversations as they work. Information given by people at work sites often contains information that is important for carrying out future tasks, and it would be convenient if this important information could be easily referenced later.

[0013] The information processing of the present disclosure includes the following processes and the like to enable easy later reference of important information at the work site.

[0014] The information processing method of Example 1 is an information processing method performed by a computer, and includes the steps of acquiring information about a person, analyzing the information about the person, detecting a trigger event that triggers information recording based on the analysis results of the information about the person, acquiring site information when the trigger event occurred, and linking and recording the site information when the trigger event occurred and the information about the person.

[0015] In this way, by analyzing information about people to detect trigger events and linking and recording the site information at the time the trigger event occurred with information about the people, important information from the work site can be easily referenced later.

[0016] An information processing method of Example 2 is the information processing method of Example 1, wherein the information about the person is speech information uttered by the person, and in the analyzing step, emotion analysis may be performed based on the speech information.

[0017] This technology analyzes human speech information to detect trigger events, and records the information on the workplace when the trigger event occurred in association with the human speech information, allowing for easy later reference of important information from the workplace.

[0018] An information processing method of Example 3 is the information processing method of Example 2, wherein when performing the emotion analysis in the analyzing step, the speech information is decomposed into a plurality of words and phrases by performing a morphological analysis on the speech information, and similarities between the plurality of words and a plurality of emotional expressions are calculated, and in the trigger event detecting step, a word among the plurality of words whose similarity is equal to or greater than a predetermined threshold is detected as the trigger event.

[0019] In this way, by detecting a phrase among multiple phrases in the speech information whose similarity is equal to or exceeds a predetermined threshold as a trigger event, it is possible to appropriately detect the trigger event and link and record the on-site information and the speech information of the person at the appropriate time, which allows for easy later reference of important information from the work site.

[0020] The information processing method of Example 4 is the information processing method according to Example 2 or 3, wherein the items relating to the plurality of emotional expressions used in performing the emotion analysis may include joy, anger, sadness, and happiness.

[0021] This allows for easy later reference to important information focused on emotional expressions such as joy, anger, sadness, and happiness.

[0022] The information processing method of Example 5 is the information processing method described in Example 2 or 3, and the items related to the plurality of emotional expressions used in the emotion analysis may include sadness, anxiety, anger, disgust, trust, surprise, and joy.

[0023] This allows users to easily refer to important information focused on emotional expressions such as sadness, anxiety, anger, disgust, trust, surprise, and joy later on.

[0024] The information processing method of Example 6 is the information processing method described in Example 2 or 3, and the items related to the multiple emotional expressions used in performing the emotion analysis may include joy, anger, excitement, sadness, like, fear, relief, disgust, surprise, and shame.

[0025] This allows for easy later reference of important information focusing on emotional expressions such as joy, anger, excitement, sadness, like, fear, relief, disgust, surprise, and shame.

[0026] An information processing method of Example 7 is the information processing method described in Example 1, wherein the information about the person is information about physical characteristics of the person's voice, and in the analyzing step, the physical characteristics of the person's voice may be analyzed, and in the detecting step, if an analysis value obtained by analyzing the physical characteristics of the person's voice exceeds a predetermined threshold, the physical characteristics of the person's voice may be detected as the trigger event.

[0027] This technology analyzes the physical characteristics of a person's voice to detect trigger events, and records the site information and information about the person when the trigger event occurred, allowing for easy later reference of important information from the workplace.

[0028] An information processing method of Example 8 is the information processing method described in Example 1, wherein the information about the person is information about the person's facial expression, and in the analyzing step, the display of the person's face is analyzed, and in the detecting step, if an analysis value obtained by analyzing the person's facial expression exceeds a predetermined threshold, the person's facial expression may be detected as the trigger event.

[0029] This technology analyzes people's facial expressions to detect trigger events, and records the site information and information about the person when the trigger event occurred, allowing important information from the work site to be easily referenced later.

[0030] An information processing method of Example 9 is the information processing method described in Example 1, wherein the information about the person is biometric information of the person, and in the analyzing step, the biometric information of the person is analyzed, and in the detecting step, if an analysis value obtained by analyzing the biometric information of the person exceeds a predetermined threshold, the biometric information of the person may be detected as the trigger event.

[0031] This technology analyzes a person's biometric information to detect a trigger event, and records the site information at the time the trigger event occurred in association with information about the person, allowing important information from the work site to be easily referenced later.

[0032] The information processing method of Example 10 is the information processing method described in any of Examples 1 to 9, and in the step of acquiring the on-site information, the on-site information may be acquired by controlling the attitude of a camera to capture an image of the site when the trigger event occurs.

[0033] In this way, by controlling the camera's attitude and capturing an image of the site when a trigger event occurs, it is possible to accurately obtain site information. This allows accurate site information to be linked to information about people and recorded. This makes it easy to refer to important information from the work site later.

[0034] The information processing method of Example 11 is the information processing method described in any of Examples 1 to 9, and in the step of acquiring the on-site information, the on-site information may be acquired by using an information collection robot to capture images and record audio of the scene when the trigger event occurs.

[0035] In this way, by using an information gathering robot to capture images and audio recordings of the work site, it is possible to accurately obtain on-site information. This allows accurate on-site information to be linked to information about people and recorded. This makes it easy to refer to important information from the work site later.

[0036] The information processing device described in Example 12 includes a person information acquisition unit that acquires information about people, an analysis unit that analyzes the information about the people, an event detection unit that detects a trigger event that triggers information recording based on the analysis results by the analysis unit, a site information acquisition unit that acquires site information when the trigger event occurs, and a recording unit that links and records the site information when the trigger event occurs and information about the people.

[0037] In this way, by analyzing information about people to detect trigger events and linking and recording the site information at the time the trigger event occurred with information about the people, important information from the work site can be easily referenced later.

[0038] The information processing system described in Example 13 is an information processing system comprising an information collecting robot that collects information about people, and an information processing device that processes the information obtained by the information collecting robot, wherein the information processing device comprises a person information acquisition unit that acquires information about the people sent from the information collecting robot, an analysis unit that analyzes the information about the people, an event detection unit that detects a trigger event that triggers information recording based on the analysis results by the analysis unit, a site information acquisition unit that acquires site information when the trigger event occurs, and a recording unit that links and records the site information when the trigger event occurs and information about the people.

[0039] In this way, by analyzing information about people to detect trigger events and linking and recording the site information at the time the trigger event occurred with information about the people, important information from the work site can be easily referenced later.

[0040] The program of Example 14 is a program for causing the computer to execute the information processing method according to any one of Examples 1 to 9.

[0041] This makes it possible to cause a computer to execute the above information processing method in accordance with the program.

[0042] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0043] Hereinafter, embodiments will be described with reference to the drawings. Note that the embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, the arrangement and connection of the components, steps, the order of steps, and the like shown in the following embodiments are merely examples and are not intended to limit the scope of the claims.

[0044] (Embodiment) [Configuration of Information Processing System] The configuration of an information processing system according to an embodiment will be described with reference to FIGS. 1 to 5. FIG.

[0045] Fig. 1 is a schematic diagram showing an information processing system 1 according to an embodiment. Fig. 2 is a block diagram showing the configuration of the information processing system 1 and an information processing device 10 included in the information processing system 1.

[0046] 1 and 2, the information processing system 1 includes an information processing device 10 and an information collecting robot 20. The information processing device 10 and the information collecting robot 20 are communicatively connected to each other via a communication network.

[0047] The information collection robot 20 is a robot deployed at a work site and is capable of moving within the work site. The information collection robot 20 is, for example, a drone 21, which is an unmanned aerial vehicle, or a mobile robot 22, such as a quadruped robot. The information collection robot 20 is controlled by the information processing device 10, but may also be autonomously controllable. The information collection robot 20 may also be forcibly controlled by a dedicated controller.

[0048] The information collection robot 20 has a movement mechanism 26 for moving itself and a detection unit for collecting information about people and site information. The detection unit is, for example, a microphone 25a for detecting sounds generated at the site and a camera 25b for capturing images of the site conditions. The camera 25b may be attached to an articulated arm so that its attitude can be controlled, or may be a PTZ (pan, tilt, zoom) camera. The camera 25b may also be an infrared camera capable of detecting temperature, thermal radiation, and the like.

[0049] The information collection robot 20 collects information about people at the work site and on-site information about events occurring at the work site. The information collected by the information collection robot 20 is transmitted to the information processing device 10 via a communication network.

[0050] The information processing device 10 is, for example, a computer device that inputs and outputs various types of information and performs arithmetic processing. The information processing device 10 executes various processes based on the information output from the information collecting robot 20. In this embodiment, an example will be described in which the information about a person is speech information i1 uttered by a person at a work site.

[0051] FIG. 3 is a diagram showing an outline of the processing executed by the information processing system 1. As shown in FIG.

[0052] The information processing system 1 acquires speech information i1 of a person at a work site, and automatically records important site information including noteworthy situations and non-routine situations at the work site in response to a trigger event te based on the speech information i1 of the person.

[0053] Fig. 3A shows utterance information i1 of a person at a work site as text information. Fig. 3B shows an example in which multiple words w (w1, w2, w3, ...) are extracted from the utterance information i1. Fig. 3C shows the analysis results of emotion analysis of the multiple words w.

[0054] The information processing system 1 extracts a plurality of words w from the speech information i1 and performs emotion analysis on the plurality of words w. The information processing system 1 detects a trigger event te through this emotion analysis (see (d) in FIG. 3). The trigger event te is an event that triggers the start of recording important on-site information.

[0055] When a trigger event te is detected, the information processing system 1 uses the information collection robot 20 to acquire on-site information at the time the trigger event te occurred (see (e) of FIG. 3). This on-site information is audio and video that shows the on-site situation at the location where the trigger event te occurred. The information processing system 1 records the on-site information at the time the trigger event te occurred in association with the speech information i1 (see (f) of FIG. 3). This information processing system 1 allows important information from the work site to be easily referenced later.

[0056] Hereinafter, each component of the information processing device 10 included in the information processing system 1 will be described.

[0057] 2, the information processing device 10 includes a human information acquisition unit 11, an analysis unit 12, an event detection unit 13, a site information acquisition unit 14, and a recording unit 15. The human information acquisition unit 11, the analysis unit 12, the event detection unit 13, the site information acquisition unit 14, and the recording unit 15 are processing units realized by a program and a processor that executes the program. Also shown in FIG. 2 is a display unit 16 that is a monitor.

[0058] The human information acquisition unit 11 is a processing circuit that acquires utterance information i1 uttered by a person. The human information acquisition unit 11 acquires utterance information i1 of a person output from the information collecting robot 20.

[0059] Human speech information i1 is information uttered by a worker at a work site. Human speech information i1 includes information uttered by a worker when he or she is alone, and information uttered by a person when multiple people communicate with each other or when a person communicates with an artifact. Communication between a person and an artifact includes dialogue (interaction) between a person and a machine (artifact) such as an interactive AI like ChatGPT (Chat Generative Pre-trained Transformer) or an artificial intelligence chatbot. The "artifact" referred to here is an interactive AI like ChatGPT, an artificial intelligence chatbot, etc.

[0060] The speech information i1 acquired by the person information acquisition unit 11 is output to the analysis unit 12. The speech information i1 output to the analysis unit 12 is composed of text data.

[0061] The analysis unit 12 is a processing circuit that performs emotion analysis based on the utterance information i1. The analysis unit 12 first performs morphological analysis on the utterance information i1 to decompose the utterance information i1 into multiple words and phrases w and acquires the multiple words and phrases w (e.g., w1, w2, w3, ...). The multiple words and phrases w here refer to all words and phrases included in the utterance information i1.

[0062] The multiple words w can also be considered one or more morphemes. A morpheme is the smallest unit of an expression that has meaning. Morphological analysis is part of natural language processing, and is a process of breaking down a sentence written in a natural language into the smallest units that have meaning in the language and determining the part of speech and conjugation of each one. This morphological analysis extracts multiple words w from the utterance information i1.

[0063] The analysis unit 12 performs emotion analysis on the multiple words w extracted by the morphological analysis. For example, the analysis unit 12 calculates similarities between the multiple words w and multiple emotional expressions that represent human emotions, and derives the results of the emotion analysis based on the similarities. The multiple emotional expressions are, for example, "joy, anger, sadness, and happiness."

[0064] The similarity between multiple phrases w and multiple emotional expressions is calculated using a cosine (cos) similarity method. Cosine similarity is a numerical representation of how closely two vectors point in the same direction. Specifically, cosine similarity is the angle between two vectors expressed as a cosine value, and is obtained by calculating the inner product of the two vectors as the numerator and the magnitude of the two vectors as the denominator. For example, a cosine similarity of 0 indicates low similarity, and a cosine similarity of 1 indicates high similarity.

[0065] Before calculating the similarity, the analysis unit 12 performs language conversion using Word2vec (word to vector). Word2vec is a natural language processing method that converts words in a sentence into numerical vectors to understand their meaning. For example, the words used when the analysis unit 12 performs emotion analysis are multiple phrases w extracted from the utterance information i1 and words indicating multiple emotional expressions, such as joy, anger, sadness, and happiness. Performing the Word2vec conversion makes it possible to perform an operation between two words, namely, the words in the utterance information i1 and the words indicating emotional expressions, specifically, to calculate the similarity.

[0066] 4 is a diagram showing an example of emotion analysis performed by the information processing device 10. Hereinafter, an example of emotion analysis performed on words uttered by people at a work site will be described.

[0067] The analysis unit 12 classifies each of the multiple words w into a plurality of emotional expressions (joy, anger, sadness, and happiness). Specifically, the analysis unit 12 derives the similarity (s1, s2, s3, ...) between each of the multiple words w (w1, w2, w3, ...) and the emotional expression "joy." Similarly, the analysis unit 12 derives the similarity between each of the multiple words w (w1, w2, w3, ...) and the emotional expression "anger," the similarity between each of the multiple words w (w1, w2, w3, ...) and the emotional expression "sad," and the similarity between each of the multiple words w (w1, w2, w3, ...) and the emotional expression "happy."

[0068] The analysis unit 12 then lists multiple words and phrases with high similarity for each of the multiple emotional expressions. For example, words and phrases with high similarity to emotional expressions include words expressing a call such as "Hey, come here for a moment," words expressing replies such as "I understand, I'm sorry, I don't understand," and interjections such as "Huh, why, ok, ok, what's this?" For example, the word "Hey" is considered to have a high similarity to the emotion of "anger," and the word "Huh" is considered to have a high similarity to the emotion of "sadness."

[0069] When listing multiple highly similar words and phrases, the analysis unit 12 lists words and phrases having a similarity equal to or greater than a predetermined threshold. The predetermined threshold is set appropriately, for example, from a range of 0.6 to 0.8. The number of words and phrases to be listed is set in advance, for example, to 100 or less. As a result, words and phrases that strongly express emotions among the multiple words and phrases w are classified into each emotional expression. In this way, the analysis unit 12 derives the analysis results of the emotional analysis by calculating the similarities between the multiple words and phrases w and the multiple emotional expressions.

[0070] The multiple emotional expressions used in emotion analysis are not limited to "joy, anger, sadness, and pleasure," but may be ten types of emotions: "joy, anger, excitement, sadness, like, fear, relief, disgust, surprise, and shame." The multiple emotional expressions may be the seven emotions listed in the Japanese IWC Dictionary of Emotion Expressions (JIWC-Dictionary): "sadness, anxiety, anger, disgust, trust, surprise, and joy." The multiple emotional expressions may be expressions based on Russell's emotional circumplex model.

[0071] The analysis result obtained by the analysis unit 12 is output to the event detection unit 13 .

[0072] The event detection unit 13 is a processing circuit that detects a trigger event te that triggers information recording based on the analysis result of the analysis unit 12.

[0073] The event detection unit 13 detects, as a trigger event te, a phrase for which the similarity between the multiple phrases w obtained by the analysis unit 12 and multiple emotional expressions is equal to or greater than a predetermined threshold. For example, if the predetermined threshold is set to 0.8, the event detection unit 13 detects, as a trigger event te, a phrase for which the similarity is equal to or greater than 0.8. The trigger event te is an occurrence that triggers the start of recording important information. The event detection unit 13 outputs information related to the trigger event te to the site information acquisition unit 14.

[0074] When the site information acquisition unit 14 receives information related to the trigger event te, it acquires site information at the time the trigger event te occurred, using the information collection robot 20. The site information acquisition unit 14 moves the information collection robot 20 to the location where the trigger event te occurred, and acquires audio and video that shows the situation at the site where the trigger event te occurred.

[0075] For example, the information collection robot 20 acquires on-site information of a worker at the location where the trigger event te occurred. If the on-site information relates to the worker's work procedures, the information collection robot 20 acquires conversations between the worker, such as pre-work checks, work samples, and post-work checks. The information collection robot 20 also acquires information about the actual work being performed at the location where the trigger event te occurred. The information about the work includes information about the work object, tools, and materials used. For example, if the work is painting, the work object is a wall or iron material, the tools used are a brush, roller, spray, and the like, and the materials used are ink. For example, if the on-site information acquired by the information collection robot 20 is painting, the information may include video information showing the pre-painting preparation, the start of painting, the drying status, and the status of any paint defects.

[0076] In this way, the on-site information at the time of the occurrence of the trigger event te includes important information, including notable situations and unsteady situations. The on-site information acquisition unit 14 acquires this on-site information and outputs it to the recording unit 15.

[0077] The recording unit 15 records the on-site information and the person's speech information i1 when the trigger event te occurs in a memory (storage unit) in a linked state.

[0078] FIG. 5 is a diagram showing an example in which on-site information and human utterance information i1 at the time when a trigger event te occurs are recorded in a linked state.

[0079] In the example shown in (a) of Figure 5, for a trigger event te of "Hey!", which indicates the emotion of anger, audio and video of the scene at the time when this trigger event te occurred and utterance information i1 are recorded. Note that the utterance information i1 may be recorded as summarized text information. In the example shown in (b) of Figure 5, for a trigger event te of "Oh!", which indicates the emotion of surprise, audio and video of the scene at the time when this trigger event te occurred and utterance information i1 are recorded. Note that the utterance information i1 may be recorded as summarized text information.

[0080] The information processing device 10 presents the recorded information recorded by the recording unit 15 by outputting it to the display unit 16. The information processing device 10 may output the recorded information to a computer or a mobile terminal used by a user. The information processing device 10 may use application software to distribute the recorded information to pre-registered users.

[0081] As described above, the information processing device 10 of this embodiment comprises a person information acquisition unit 11 that acquires information about people, an analysis unit 12 that analyzes information about people, an event detection unit 13 that detects a trigger event te that triggers information recording based on the analysis results by the analysis unit 12, a site information acquisition unit 14 that acquires site information when the trigger event te occurs, and a recording unit 15 that links and records the site information and information about people when the trigger event te occurs.

[0082] In this way, by analyzing information about people to detect trigger event te, and linking and recording the site information and information about people at the time the trigger event te occurred, important information at the work site can be easily referenced later.

[0083] [Information Processing Method] The information processing method according to the embodiment will be described with reference to FIG.

[0084] FIG. 6 is a flowchart showing an information processing method according to an embodiment.

[0085] First, the information processing device 10 acquires information about a person (step S10). The information about a person is, for example, speech information i1 uttered by a person at a work site.

[0086] Next, the information processing device 10 analyzes information about a person (step S20). For example, the information processing device 10 extracts multiple words and phrases w from utterance information i1, which is an example of information about a person, and performs emotion analysis on the multiple words and phrases w. Specifically, the information processing device 10 performs morphological analysis on the utterance information i1 to decompose the utterance information i1 into multiple words and phrases w and extract the multiple words and phrases w. Furthermore, the information processing device 10 calculates similarities between the multiple words and phrases w and multiple emotional expressions, and derives emotion analysis results based on the similarities.

[0087] Next, the information processing device 10 detects a trigger event te that triggers information recording based on the analysis result of the information about the person (step S30). For example, the information processing device 10 detects, as the trigger event te, a word or phrase among a plurality of words or phrases w whose similarity is equal to or greater than a predetermined threshold.

[0088] Next, the information processing device 10 acquires on-site information when the trigger event te occurred (step S40). For example, when the information processing device 10 receives information about the trigger event te, it uses the information collection robot 20 to acquire on-site information when the trigger event te occurred.

[0089] Next, the information processing device 10 records the site information and the information about the person when the trigger event te occurs in association with each other (step S50).

[0090] By executing steps S10 to S50, the site information and the information about the person at the time the trigger event te occurred can be linked and recorded. This information processing method allows important information from the work site to be easily referenced later.

[0091] [Modification of the embodiment] The configuration of an information processing system 1A according to a modification of the embodiment will be described with reference to Figures 7 to 9. In the modification, an example will be described in which the information processing system 1A includes a wearable terminal 30, and information obtained from the wearable terminal 30 is used as a trigger to record information.

[0092] Fig. 7 is a schematic diagram showing an information processing system 1A according to a modified example of the embodiment, and Fig. 8 is a diagram showing a block configuration of the information processing system 1A and an information processing device 10A included in the information processing system 1A.

[0093] 7 and 8, the information processing system 1A includes an information processing device 10A, an information collecting robot 20, and a wearable terminal 30. The information processing device 10A, the information collecting robot 20, and the wearable terminal 30 are communicatively connected to one another via a communication network.

[0094] The information collection robot 20 is a robot that is deployed at a work site and is capable of moving within the work site. The information collection robot 20 is, for example, a drone 21, which is an unmanned aerial vehicle, or a mobile robot 22, such as a quadruped robot.

[0095] The information collecting robot 20 acquires information about people at the work site. In this modification, an example will be described in which the information about the people is sensing information i2 about the people. The sensing information i2 about the people includes, for example, physical characteristics of the people's voices, facial expressions, and biometric information about the people.

[0096] The information gathering robot 20 acquires information about the physical characteristics of the voices of people at the work site. The physical characteristics of people's voices include the volume and intonation of the voices of people. The information gathering robot 20 also acquires information about the facial expressions of people at the work site. The facial expressions of people include information such as the person's gaze and pupil dilation. The information gathering robot 20 also acquires biometric information of people at the work site, such as the skin temperature and the amount of heat radiation (heat dissipation) from the human body.

[0097] The information collection robot 20 of the modified example collects the sensing information i2 and on-site information related to events occurring at the work site. The information collected by the information collection robot 20 is transmitted to the information processing device 10A via a communication network.

[0098] The wearable device 30 is a terminal device worn by a person at a work site. The wearable device 30 is, for example, a smart watch worn on the person's wrist or smart glasses placed in front of the person's eyes. The wearable device 30 acquires biometric information of the person, such as pulse (heart rate), blood pressure, blood lactate level, skin temperature, heat radiation (heat dissipation amount) from the human body, and sweat rate. The wearable device 30 also acquires information such as the person's line of sight and pupil dilation. The information collected by the wearable device 30 is transmitted to the information processing device 10A via a communication network.

[0099] The information processing device 10A is, for example, a computer device that inputs and outputs various types of information and performs arithmetic processing. The information processing device 10A executes various processes based on information output from the information collecting robot 20 and the wearable terminal 30.

[0100] FIG. 9 is a diagram showing an outline of the processing executed in the information processing system 1A.

[0101] The information processing system 1A acquires sensing information i2 of people at a work site, and automatically records important site information including noteworthy situations and unsteady situations at the work site in response to a trigger event te based on the sensing information i2.

[0102] 9A shows sensing information i2 of people at a work site, and FIG. 9B shows the analysis results obtained by performing an analysis based on the sensing information i2 of people.

[0103] The information processing system 1A detects a trigger event te by analyzing the sensing information i2 of the person (see (c) of FIG. 9).

[0104] When a trigger event te is detected, the information processing system 1A uses the information collection robot 20 to acquire on-site information at the time the trigger event te occurred (see (d) of FIG. 9). The information processing system 1A links the on-site information at the time the trigger event te occurred with the human sensing information i2 and records them (see (e) of FIG. 9). According to the information processing system 1A of the modified example, important information from the work site can be easily referenced later.

[0105] Hereinafter, each component of the information processing device 10A included in the information processing system 1A will be described.

[0106] As shown in FIG. 8, the information processing device 10A includes a person information acquisition unit 11, an analysis unit 12, an event detection unit 13, a site information acquisition unit 14, and a recording unit 15.

[0107] The human information acquisition unit 11 is a processing circuit that acquires human sensing information i2. The human information acquisition unit 11 acquires human sensing information i2 output from the information collection robot 20 and the wearable terminal 30. The sensing information i2 acquired by the human information acquisition unit 11 is output to the analysis unit 12.

[0108] The analysis unit 12 is a processing circuit that performs analysis based on the human sensing information i2. The analysis result obtained by the analysis unit 12 is output to the event detection unit 13.

[0109] The event detection unit 13 is a processing circuit that detects a trigger event te that triggers information recording based on the analysis result of the analysis unit 12.

[0110] The event detection unit 13 detects a trigger event te based on the analysis results obtained by the analysis unit 12. For example, when an analysis value (sound pressure or frequency difference) obtained by analyzing the physical characteristics of a person's voice exceeds a predetermined threshold, the event detection unit 13 detects the physical characteristics of the person's voice as the trigger event te. For example, when analysis values ​​(amount of change in gaze, pupil dilation, pupil movement speed, number of blinks) obtained by analyzing the person's facial expression exceed a predetermined threshold, the event detection unit 13 detects the person's facial expression as the trigger event te. For example, when analysis values ​​(pulse rate, blood pressure, blood lactate level, skin temperature, amount of heat radiation from the human body, amount of sweat) obtained by analyzing the person's biometric information exceed a predetermined threshold, the event detection unit 13 detects the person's biometric information as the trigger event te. The event detection unit 13 outputs information related to the trigger event te to the site information acquisition unit 14.

[0111] When the site information acquisition unit 14 receives information related to the trigger event te, it acquires site information at the time the trigger event te occurred using the information collection robot 20. The site information acquisition unit 14 acquires the site information and outputs it to the recording unit 15.

[0112] The recording unit 15 records the on-site information at the time when the trigger event te occurs and the human sensing information i2 in a linked state in a memory (storage unit).

[0113] In the modified information processing device 10A, information about people is analyzed to detect a trigger event te, and the site information at the time the trigger event te occurs is linked to information about the people and recorded, making it possible to easily refer to important information at the work site later.

[0114] (Other Embodiments) Although aspects of the information processing system etc. have been described above according to the embodiments, the aspects of the information processing system etc. are not limited to the embodiments. Modifications conceivable by those skilled in the art may be made to the embodiments, and multiple components in the embodiments may be combined in any manner.

[0115] For example, in FIG. 5 , a portion of the speech information i1 is shown as text information. However, this text information may be extracted by analyzing speech at a workplace using topic model analysis, which is an example of latent semantic analysis. Topic model analysis is a natural language processing technique that categorizes multiple pieces of text information based on the content contained in each piece of text information, interprets the meanings of words present in the text information by applying them to the classified categories, and extracts topics from the text information. A topic is a category of latent meaning, and is called a "latent topic" or simply "topic." It is a term that concisely and comprehensively represents the semantic content indicated by multiple words. A topic may be a word present in the text information, or may be a term formed by combining and processing words present in the text information. By performing topic model analysis, multiple words that have the same meaning can be grouped together as a single topic, making it possible to understand the latent meaning of a document.

[0116] Furthermore, for example, a process performed by a specific component in the embodiments may be performed by another component instead of the specific component. The order of multiple processes may be changed, or multiple processes may be performed in parallel. Furthermore, ordinal numbers such as "first" and "second" used in the description may be changed, removed, or newly assigned as appropriate. These ordinal numbers do not necessarily correspond to a meaningful order, and may be used to identify elements.

[0117] The information processing method may be executed by any system or device, i.e., the information processing method may be executed by the information processing system described above or by another system or device.

[0118] For example, a part or all of the information processing method may be executed by a computer including a processor, a memory, an input / output circuit, etc. In this case, the information processing method may be executed by the computer executing a program for causing the computer to execute the information processing method.

[0119] The above program may also be recorded on a non-transitory computer-readable recording medium such as a CD-ROM.

[0120] Furthermore, each component of an information processing system or the like may be configured with dedicated hardware, general-purpose hardware that executes the above-mentioned programs, or a combination of these. The general-purpose hardware may be configured with a memory in which the programs are recorded and a general-purpose processor that reads and executes the programs from the memory. Here, the memory may be a semiconductor memory or a hard disk, and the general-purpose processor may be a CPU or the like.

[0121] Furthermore, the dedicated hardware may be configured with a memory and a dedicated processor, etc. For example, the dedicated processor may refer to the memory and execute the information processing method described above.

[0122] Furthermore, each component of the information processing system may be an electric circuit. These electric circuits may form a single electric circuit as a whole, or may be separate electric circuits. Furthermore, these electric circuits may correspond to dedicated hardware, or may correspond to general-purpose hardware that executes the above-mentioned programs, etc.

[0123] The present disclosure is applicable to an information processing system that allows important information at a work site to be easily referenced later.

[0124] 1, 1A Information processing system 10, 10A Information processing device 11 Person information acquisition unit 12 Analysis unit 13 Event detection unit 14 Site information acquisition unit 15 Recording unit 16 Display unit 20 Information collection robot 21 Drone 22 Mobile robot 25a Microphone 25b Camera 26 Mobile mechanism 30 Wearable terminal i1 Speech information i2 Sensing information s1, s2, s3 Similarity te Trigger event w, w1, w2, w3 Phrase

Claims

1. An information processing method performed by a computer, comprising the steps of: acquiring information about a person; analyzing the information about the person; detecting a trigger event that triggers information recording based on the results of analyzing the information about the person; acquiring site information when the trigger event occurred; and linking and recording the site information and information about the person when the trigger event occurred.

2. The information processing method according to claim 1, wherein the information relating to the person is speech information uttered by the person, and in the analyzing step, emotion analysis is performed based on the speech information.

3. The information processing method according to claim 2, wherein when performing the emotion analysis in the analyzing step, a morphological analysis is performed on the speech information to decompose the speech information into a plurality of words and phrases, and similarities between the plurality of words and a plurality of emotional expressions are obtained; and in the trigger event detection step, a word among the plurality of words whose similarity is equal to or greater than a predetermined threshold is detected as the trigger event.

4. The information processing method according to claim 3, wherein the items relating to a plurality of emotional expressions used in carrying out the emotion analysis include joy, anger, sadness, and happiness.

5. The information processing method according to claim 3, wherein the items relating to a plurality of emotional expressions used in carrying out the emotion analysis include sadness, anxiety, anger, disgust, trust, surprise, and joy.

6. The information processing method according to claim 3, wherein the items relating to a plurality of emotional expressions used in carrying out the emotion analysis include joy, anger, excitement, sadness, like, fear, relief, disgust, surprise, and shame.

7. The information processing method of claim 1, wherein the information about the person is information about the physical characteristics of the person's voice, and in the analyzing step, the physical characteristics of the person's voice are analyzed, and in the trigger event detecting step, if an analysis value obtained by analyzing the physical characteristics of the person's voice exceeds a predetermined threshold, the physical characteristics of the person's voice are detected as the trigger event.

8. The information processing method of claim 1, wherein the information about the person is information about facial expressions of the person, the analyzing step analyzes the display of the person's face, and the trigger event detection step detects the person's facial expression as the trigger event if an analysis value obtained by analyzing the person's facial expression exceeds a predetermined threshold.

9. The information processing method of claim 1, wherein the information about the person is biometric information of the person, the analyzing step analyzes the biometric information of the person, and the trigger event detection step detects the biometric information of the person as the trigger event when an analysis value obtained by analyzing the biometric information of the person exceeds a predetermined threshold value.

10. An information processing method according to any one of claims 1 to 9, wherein in the step of acquiring the on-site information, the on-site information is acquired by controlling the attitude of a camera to capture an image of the site when the trigger event occurs.

11. An information processing method according to any one of claims 1 to 9, wherein in the step of acquiring the on-site information, the on-site information is acquired by using an information collecting robot to capture images and audio of the scene when the trigger event occurs.

12. An information processing device comprising: a human information acquisition unit that acquires information about a person; an analysis unit that analyzes the information about the person; an event detection unit that detects a trigger event that triggers information recording based on the analysis results by the analysis unit; a site information acquisition unit that acquires site information when the trigger event occurs; and a recording unit that links and records the site information and information about the person when the trigger event occurs.

13. An information processing system comprising an information collecting robot that collects information about people, and an information processing device that processes information obtained by the information collecting robot, wherein the information processing device comprises: a person information acquisition unit that acquires information about the person transmitted from the information collecting robot; an analysis unit that analyzes the information about the person; an event detection unit that detects a trigger event that triggers information recording based on the analysis results by the analysis unit; a site information acquisition unit that acquires site information when the trigger event occurred; and a recording unit that links and records the site information and information about the person when the trigger event occurred.

14. A program for causing the computer to execute the information processing method according to any one of claims 1 to 9.

Images