Information processing device, method, program and system

Abstract

To provide a system capable of quantifying excitement in a prescribed space while maintaining security.SOLUTION: A system includes means installed in a prescribed space and collecting sounds generated in the space; means installed in the space, classifying types of the sounds collected by the means for collecting the sounds, and determining a state relating to excitement and a degree of the state on the basis of the classified types and volumes; means for aggregating the state and the degree and calculating time transition of the excitement in the space; and means for presenting information based on the time transition of the excitement in the space to a user.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, a method, a program, and a system. [Background technology]

[0002] As an indicator of whether a meeting was meaningful or not, whether the meeting was lively or energetic is an important factor. In recent years, a technique for measuring the degree of liveliness of a meeting has been proposed (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-012216 Summary of the Invention [Problem to be solved by the invention]

[0004] Patent Document 1, for example, describes a meeting analysis device that calculates the level of excitement of a meeting from recorded meeting data including audio and / or video. The meeting analysis device extracts feature amounts as a time series along the progress of the meeting from the meeting data, and calculates the level of excitement as a time series along the progress of the meeting as an absolute evaluation value based on the feature amounts.

[0005] However, in Patent Document 1, the meeting data is transmitted to a meeting analysis device, and the information discussed in the meeting is stored in the meeting analysis device. Therefore, there is a possibility that information that should be kept within the meeting may be leaked to the outside.

[0006] The objective of this disclosure is to quantify the rise in a given space while maintaining security. [Means for solving the problem]

[0007] The system comprises: means for being installed in a specified space and collecting sounds generated in the space; means for being installed in the space and classifying the types of sounds collected by the means for collecting sounds, and determining the state and degree of excitement of the state based on the classified type and volume; means for aggregating the state and degree and calculating the time progression of excitement in the space; and means for presenting information based on the time progression of excitement in the space to a user. [Effects of the Invention]

[0008] According to the present disclosure, it is possible to quantify the excitement in a given space while maintaining security. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram showing the overall configuration of a system 1. FIG. [Figure 2] 2 is a block diagram showing an example of the functional configuration of a terminal device 10. FIG. [Figure 3] 2 is a block diagram showing an example of the functional configuration of a server 20. FIG. [Figure 4] FIG. 2 is a block diagram showing an example of the functional configuration of an analysis device 30. [Figure 5] FIG. 2 is a diagram illustrating a data structure of a table. [Figure 6] FIG. 2 is a diagram illustrating a data structure of a table. [Figure 7] FIG. 2 is a diagram illustrating a data structure of a table. [Figure 8] FIG. 2 is a diagram illustrating a data structure of a table. [Figure 9] FIG. 2 is a diagram illustrating an example of a processing flow in the system 1. [Figure 10] FIG. 2 is a diagram illustrating an example of a processing flow in the system 1. [Figure 11] FIG. 10 is a diagram illustrating an example screen of the present disclosure. [Figure 12] FIG. 10 is a diagram illustrating an example screen of the present disclosure. [Figure 13] FIG. 10 is a diagram illustrating an example screen of the present disclosure. [Figure 14]FIG. 10 is a diagram illustrating an example screen of the present disclosure. [Figure 15] FIG. 9 is a block diagram showing the basic hardware configuration of a computer 90. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In all drawings describing the embodiments, common components are designated by the same reference numerals, and repeated description will be omitted. Note that the following embodiments do not unduly limit the content of the present disclosure described in the claims. Furthermore, not all components shown in the embodiments are necessarily essential components of the present disclosure. Furthermore, each drawing is a schematic diagram and is not necessarily a precise illustration.

[0011] In the following description, a "processor" refers to one or more processors. The at least one processor is typically a microprocessor such as a CPU (Central Processing Unit), but may also be another type of processor such as a GPU (Graphics Processing Unit). The at least one processor may be single-core or multi-core.

[0012] Furthermore, the at least one processor may be a processor in the broad sense, such as a hardware circuit (for example, a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC)) that performs part or all of the processing.

[0013] In the following explanation, information that produces an output for an input may be described using expressions such as "xxx table," but this information may be data of any structure, or may be a learning model such as a neural network that produces an output for an input. Therefore, an "xxx table" may be referred to as "xxx information."

[0014] Furthermore, in the following description, the configuration of each table is an example, and one table may be divided into two or more tables, or all or part of two or more tables may be one table.

[0015] In addition, in the following explanation, processing may be described using the "program" as the subject, but since a program is executed by a processor to perform specified processing while appropriately using a memory unit and / or an interface unit, etc., the subject of the processing may also be the processor (or a device such as a controller that has that processor).

[0016] The program may be installed in a device such as a computer, or may be stored in, for example, a program distribution server or a computer-readable (e.g., non-transitory) recording medium. Also, in the following description, two or more programs may be realized as one program, or one program may be realized as two or more programs.

[0017] Furthermore, in the following description, identification numbers are used as identification information for various objects, but other types of identification information (for example, identifiers including alphabetic characters or symbols) may also be used.

[0018] In addition, in the following description, when describing elements of the same type without distinguishing between them, reference symbols (or common symbols among the reference symbols) may be used, and when describing elements of the same type with distinction between them, the identification numbers (or reference symbols) of the elements may be used.

[0019] In the following description, the control lines and information lines are those that are considered necessary for the description, and do not necessarily represent all the control lines and information lines in the product. All components may be interconnected.

[0020] Each information processing device is configured by a computer equipped with an arithmetic unit and a storage device. The basic hardware configuration of the computer and the basic functional configuration of the computer realized by the hardware configuration will be described later. For each of the terminal device 10 and the server 20, descriptions that overlap with the basic hardware configuration and basic functional configuration of the computer will be omitted.

[0021] <Summary> The system according to this embodiment uses edge AI to identify the type of sound, and then determines a numerical value related to excitement based on the identified type and volume. The determined numerical value is used to estimate the excitement level of the space.

[0022] The following describes an example of estimating the level of excitement in a conference room. However, the space in which this embodiment is implemented is not limited to a closed space such as a conference room, but may be an open space within an office. The open space within an office may be, for example, an area within the office such as a booth, salon, or cafeteria. Hereinafter, the open space within an office will be referred to as an area.

[0023] <1 Overall system configuration> Fig. 1 is a block diagram showing an example of the overall configuration of a system 1. The system 1 shown in Fig. 1 includes, for example, a terminal device 10, a server 20, an analysis device 30, a microphone 31, an image capture device 32, and a speaker 33. The terminal device 10, the server 20, the analysis device 30, and the speaker 33 are connected for communication via, for example, a network 40.

[0024] 1 shows an example in which the system 1 includes two terminal devices 10, but the number of terminal devices 10 included in the system 1 is not limited to two. The number of terminal devices 10 included in the system 1 may be less than three, or may be three or more.

[0025] In this embodiment, a collection of multiple devices may be considered as one server. The allocation of multiple functions required to realize the server 20 according to this embodiment to one or more pieces of hardware can be determined appropriately in consideration of the processing capacity of each piece of hardware and / or the specifications required for the server 20.

[0026] The terminal device 10 shown in Fig. 1 is, for example, an information processing device used by a user who wants to grasp the excitement in a space. The terminal device 10 may be, for example, a mobile terminal such as a smartphone or a tablet, or may be a stationary personal computer (PC) or a laptop PC. It may also be a wearable terminal such as a head mounted display (HMD) or a wristwatch terminal. The terminal device 10 may be, for example, owned and carried by an individual, or may be shared among employees and fixed to the wall of a conference room.

[0027] The terminal device 10 includes a communication IF (Interface) 12, an input device 13, an output device 14, a memory 15, a storage 16, and a processor 19.

[0028] The communication IF 12 is an interface for inputting and outputting signals so that the terminal device 10 can communicate with devices in the system 1, such as the server 20, for example.

[0029] The input device 13 is a device for receiving input operations from a user (for example, a touch panel, a touch pad, a pointing device such as a mouse, a keyboard, etc.).

[0030] The output device 14 is a device (such as a display or speaker) for presenting information to the user.

[0031] The memory 15 is for temporarily storing programs and data to be processed by the programs, and is a volatile memory such as a DRAM (Dynamic Random Access Memory).

[0032] The storage 16 is for storing data, and is, for example, a flash memory or a hard disk drive (HDD).

[0033] The processor 19 is hardware for executing an instruction set written in a program, and is composed of an arithmetic unit, a register, a peripheral circuit, and the like.

[0034] The server 20 is, for example, an information processing device that analyzes excitement based on information acquired from the analysis device 30. The server 20 is realized by, for example, a computer connected to a network 40. As shown in FIG. 1 , the server 20 includes a communication IF 22, an input / output IF 23, a memory 25, a storage 26, and a processor 29.

[0035] The communication IF 22 is an interface for inputting and outputting signals so that the server 20 can communicate with devices in the system 1, such as the terminal device 10, for example.

[0036] The input / output IF 23 functions as an interface with an input device for receiving input operations from the user and an output device for presenting information to the user.

[0037] The memory 25 is for temporarily storing programs and data to be processed by the programs, and is a volatile memory such as a DRAM.

[0038] The storage 26 is for storing data, and is, for example, a flash memory or a HDD.

[0039] The processor 29 is hardware for executing an instruction set written in a program, and is composed of an arithmetic unit, registers, peripheral circuits, and the like.

[0040] The analysis device 30 is, for example, an information processing device that analyzes audio and images from the viewpoint of excitement.

[0041] One analysis device 30 may cover one space, or may cover multiple spaces. Specifically, one analysis device 30 may analyze audio and images obtained in one conference room, or may analyze audio and images obtained in multiple conference rooms. The analysis device 30 is installed in a location where information analysis is possible. For example, the analysis device 30 is placed in a corner of a conference room. Also, specifically, one analysis device 30 may analyze audio and images obtained in one area, or may analyze audio and images obtained in multiple areas. The analysis device 30 is installed in a location where information analysis is possible. For example, the analysis device 30 is placed in a corner of an area.

[0042] The analysis device 30 receives information transmitted from the microphone 31 and the photographing device 32, analyzes the received information, and transmits the analyzed information to the server 20. The analysis device 30 also transmits information acquired from the server 20 to the microphone 31 and the photographing device 32. The information acquired from the server 20 includes, for example, information for updating the settings of the microphone 31 or the photographing device 32.

[0043] The microphone 31 is installed so as to collect sound within the space. For example, the microphone 31 is installed so as to collect sound from the entire conference room. Specifically, the microphone 31 is, for example, a disk-shaped sound-collecting microphone installed on a desk in the center of the conference room. Furthermore, for example, the microphone 31 is installed so as to collect sound from the entire area. Specifically, the microphone 31 is, for example, a disk-shaped sound-collecting microphone installed on the ceiling in the center of the area. The microphone 31 outputs the acquired sound signal to the analysis device 30. If one microphone 31 cannot capture the sound from the entire space, multiple microphones 31 are installed. In this case, the microphone 31 may be, for example, a microphone with preset directivity.

[0044] The camera device 32 is installed so as to capture the situation within the space. For example, the camera device 32 is installed so as to capture the situation of the entire conference room. Specifically, the camera device 32 is, for example, an omnidirectional camera installed on a desk in the center of the conference room. Furthermore, for example, the camera device 32 is installed so as to capture the situation of the entire area. Specifically, the camera device 32 is, for example, an omnidirectional camera installed on the ceiling in the center of the area. The camera device 32 outputs the acquired image signal to the analysis device 30. If one camera device 32 cannot capture the situation of the entire space, multiple camera devices 32 are installed. In this case, the camera device 32 may be, for example, a camera whose shooting direction is set in advance.

[0045] The speaker 33 is a speaker that is installed in a space and that plays music within the space. For example, the speaker 33 is installed so as to deliver music to the entire conference room. Specifically, the speaker 33 is, for example, an omnidirectional speaker installed on a desk in the center of the conference room. Also, for example, the speaker 33 is installed so as to deliver music to the entire area. Specifically, the speaker 33 is, for example, an omnidirectional speaker installed on the ceiling in the center of the area. The speaker 33 converts audio signals provided by the server 20 into music and outputs the music into the space. If one speaker 33 cannot deliver music to the entire space, multiple speakers 33 are installed.

[0046] <2. Functional configuration of terminal device> Fig. 2 is a block diagram showing an example of the functional configuration of the terminal device 10 shown in Fig. 1. The terminal device 10 shown in Fig. 2 is realized by a mobile terminal, a PC, or a wearable terminal. As shown in Fig. 2, the terminal device 10 includes a communication unit 120, an input device 13, an output device 14, an audio processing unit 17, a microphone 171, a speaker 172, a camera 161, a position information sensor 150, a storage unit 180, and a control unit 190. The blocks included in the terminal device 10 are electrically connected by, for example, a bus or the like.

[0047] The communication unit 120 performs processing such as modulation and demodulation for the terminal device 10 to communicate with other devices. The communication unit 120 performs transmission processing on the signal generated by the control unit 190 and transmits it to the outside (for example, the server 20). The communication unit 120 performs reception processing on the signal received from the outside and outputs it to the control unit 190.

[0048] The input device 13 is a device for inputting instructions or information by a user operating the terminal device 10. The input device 13 is realized, for example, by a touch-sensitive device 131 or the like, which inputs instructions by touching an operation surface. When the terminal device 10 is a PC or the like, the input device 13 may be realized by a reader, keyboard, mouse, or the like. The input device 13 converts instructions input by the user into electrical signals and outputs the electrical signals to the control unit 190. The input device 13 may include, for example, a receiving port that receives electrical signals input from an external input device.

[0049] The output device 14 is a device for presenting information to a user operating the terminal device 10. The output device 14 is realized, for example, by a display 141 or the like. The display 141 displays data according to the control of the control unit 190. The display 141 is realized, for example, by an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display or the like.

[0050] The audio processing unit 17 performs, for example, digital-to-analog conversion processing of an audio signal. The audio processing unit 17 converts a signal provided from the microphone 171 into a digital signal and provides the converted signal to the control unit 190. The audio processing unit 17 also provides the audio signal to the speaker 172. The audio processing unit 17 is realized, for example, by a processor for audio processing. The microphone 171 receives audio input and provides an audio signal corresponding to the audio input to the audio processing unit 17. The speaker 172 converts the audio signal provided from the audio processing unit 17 into audio and outputs the audio to the outside of the terminal device 10.

[0051] The camera 161 is a device that receives light with a light receiving element and outputs the light as an image signal.

[0052] The position information sensor 150 is a sensor that detects the position of the terminal device 10, and is, for example, a GPS (Global Positioning System) module. The GPS module is a receiving device used in a satellite positioning system. In the satellite positioning system, signals are received from at least three or four satellites, and the current position of the terminal device 10 equipped with the GPS module is detected based on the received signals. The position information sensor 150 may detect the current position of the terminal device 10 from the position of the wireless base station to which the terminal device 10 is connected.

[0053] The storage unit 180 is realized by, for example, the memory 15, the storage 16, etc., and stores data and programs used by the terminal device 10.

[0054] The control unit 190 is realized by the processor 19 reading a program stored in the storage unit 180 and executing instructions included in the program. The control unit 190 controls the operation of the terminal device 10. The control unit 190 functions as an operation reception unit 191, a transmission / reception unit 192, and a presentation control unit 193 by operating in accordance with the program.

[0055] Operation reception unit 191 performs processing for receiving instructions or information input from input device 13. Specifically, for example, operation reception unit 191 receives information based on instructions input from touch-sensitive device 131 or the like. The instructions input via touch-sensitive device 131 or the like are, for example, editing instructions from the user.

[0056] Furthermore, the operation reception unit 191 receives voice instructions input from the microphone 171. Specifically, for example, the operation reception unit 191 receives a voice signal that is input from the microphone 171 and converted into a digital signal by the voice processing unit 17. For example, the operation reception unit 191 analyzes the received voice signal and extracts a predetermined noun, thereby acquiring an instruction from the user.

[0057] The transmitting / receiving unit 192 performs processing for the terminal device 10 to transmit and receive data to and from an external device such as the server 20 in accordance with a communication protocol. Specifically, for example, the transmitting / receiving unit 192 transmits an editing instruction input by a user to the server 20. In addition, the transmitting / receiving unit 192 receives information about the user from the server 20.

[0058] The presentation control unit 193 controls the output device 14 to present information provided from the server 20 to the user. Specifically, for example, the presentation control unit 193 causes the information transmitted from the server 20 to be displayed on the display 141. In addition, the presentation control unit 193 causes the information transmitted from the server 20 to be output from the speaker 172.

[0059] <3. Functional configuration of the server> 3 is a block diagram showing an example of the functional configuration of the server 20. As shown in FIG.

[0060] The communication unit 201 performs processing for the server 20 to communicate with external devices.

[0061] The storage unit 202 includes, for example, a voice analysis log database (DB) 2021, an image analysis log database (DB) 2022, etc. The tables stored in the storage unit 202 are not limited to these.

[0062] The voice analysis log DB 2021 accumulates the contents of the voice analysis log DB 3021 of each analysis device 30. Details will be described later.

[0063] The image analysis log DB 2022 accumulates the contents of the image analysis log DB 3022 of each analysis device 30. This will be explained later.

[0064] The control unit 203 is realized by the processor 29 reading a program stored in the storage unit 202 and executing instructions included in the program. By operating in accordance with the program, the control unit 203 performs functions shown as a reception control module 2031, a transmission control module 2032, an analysis control module 2033, and a presentation control module 2034.

[0065] The reception control module 2031 controls the process by which the server 20 receives signals from external devices in accordance with a communication protocol.

[0066] The transmission control module 2032 controls the process in which the server 20 transmits signals to external devices in accordance with a communication protocol.

[0067] The analysis control module 2033 controls the process of analyzing the information received from the analysis device 30. Specifically, the analysis control module 2033 controls, for example, the process of analyzing information relating to the excitement in the space.

[0068] The presentation control module 2034 controls the process of presenting information to the user.

[0069] <4. Functional configuration of the analysis device> 4 is a block diagram showing an example of the functional configuration of analysis device 30. As shown in FIG. 4, analysis device 30 functions as a communication unit 301, a storage unit 302, and a control unit 303.

[0070] The communication unit 301 performs processing for the analysis device 30 to communicate with external devices.

[0071] The storage unit 302 includes, for example, an audio analysis log database (DB) 3021, an image analysis log database (DB) 3022, a first trained model 3023, and a second trained model 3024. The tables stored in the storage unit 302 are not limited to these.

[0072] The audio analysis log DB 3021 is a database for holding information about audio within a space collected by the microphone 31. Details will be described later.

[0073] The image analysis log DB 3022 is a database for holding information about images of the situation within a space captured by the image capturing device 32. Details will be described later.

[0074] The first trained model 3023 is a model that, when a voice is input, learns to output information for evaluating excitement based on the input voice. For example, when a voice is input, the first trained model 3023 learns to distinguish the type of sound in a space, etc. The types of voice include, for example, conversation, cheers, laughter, belly laughs, angry shouts, silence, screams, crying, moans, sneezing, humming, typing, etc.

[0075] In this case, the learning data may be, for example, speech as input data, and the correct output data may be a judgment on the type of sound contained in the input data.

[0076] The second trained model 3024 is a model that, when an image is input, learns to output information for evaluating excitement based on the input image. For example, when an image is input, the second trained model 3024 learns to determine the number of people in a space, their facial expressions, and the like. Types of facial expressions include, for example, smiling, serious, crying, and the like. Types of facial expressions also include, for example, gestures such as nodding, shaking head, frowning, and looking down.

[0077] In this case, the learning data may be, for example, a human image as input data, and the judgment on the facial expression contained in this input data may be used as correct output data.

[0078] The first trained model 3023 and the second trained model 3024 are models generated by having a machine learning model perform machine learning in accordance with a model learning program. The first trained model 3023 and the second trained model 3024 are, for example, parameterized composite functions formed by combining multiple functions that perform predetermined inference based on input data. The parameterized composite function is defined by a combination of multiple adjustable functions and parameters. The trained model according to this embodiment may be any parameterized composite function that meets the above requirements.

[0079] For example, if the first trained model 3023 and the second trained model 3024 are generated using a forward propagation type multilayer network, the parameterized composite function is defined as a combination of, for example, a linear relationship between each layer using a weight matrix, a nonlinear relationship (or a linear relationship) using an activation function in each layer, and a bias. The weight matrix and bias are called parameters of the multilayer network. The form of the parameterized composite function as a function changes depending on how the parameters are selected. In a multilayer network, by appropriately setting the constituent parameters, it is possible to define a function that can output a desirable result from the output layer.

[0080] The multi-layered network according to the present embodiment may be, for example, a deep neural network (DNN), which is a multi-layered neural network that is the subject of deep learning. As the DNN, for example, a convolution neural network (CNN) that targets images may be used.

[0081] The control unit 303 is realized by the processor in the analysis device 30 reading a program stored in the storage unit 302 and executing instructions included in the program. By operating in accordance with the program, the control unit 303 performs functions shown as a reception control module 3031, a transmission control module 3032, an analysis control module 3033, and a presentation control module 3034.

[0082] The reception control module 3031 controls the process by which the analysis device 30 receives signals from external devices in accordance with a communication protocol.

[0083] The transmission control module 3032 controls the process in which the analysis device 30 transmits signals to external devices in accordance with a communication protocol.

[0084] The analysis control module 3033 controls the process of analyzing information received from the microphone 31 and the image capturing device 32. Specifically, the analysis control module 3033 analyzes audio and images from the perspective of excitement. More specifically, for example, the analysis control module 3033 analyzes audio using the first trained model 3023. Furthermore, for example, the analysis control module 3033 analyzes images using the second trained model 3024.

[0085] The presentation control module 3034 controls the process of presenting information to the user.

[0086] <5 Data Structure> The data structure of the table is explained below. Note that the data structure explained is an example and does not exclude data that is not listed. Furthermore, even data listed in the same table may be stored in separate storage areas.

[0087] 5 is a diagram showing the data structure of the voice analysis log DB 3021. As shown in FIG. 5, each record in the voice analysis log DB 3021 includes, for example, an item "location," an item "microphone ID," an item "time," an item "voice," an item "volume," an item "state," and an item "value." When voice is output by the first trained model 3023, a new record is created in the voice analysis log DB 3021, and each piece of information is stored in the corresponding item. The voice analysis log DB 3021 of each analysis device 30 records information about the space covered by that analysis device 30.

[0088] The item "location" indicates the space that the analysis device 30 covers.

[0089] The item "microphone ID" indicates identification information for identifying the microphone 31 installed in the space.

[0090] The item "time" indicates the time when the sound was collected by the microphone 31.

[0091] The item "audio" indicates the type of audio, for example, the type of audio detected by the first trained model 3023.

[0092] The item "volume" indicates the volume of the item "audio." The item "volume" is expressed, for example, in three levels. If the intensity of the sound acquired by the microphone 31 exceeds a set threshold, the server 20 sets a value corresponding to the exceeded threshold. Note that the item "volume" is not limited to three levels.

[0093] The item "state" indicates a state related to excitement. The item "state" includes, for example, "excitement", "decline", and "neutral". Positive sounds such as laughter are classified under "excitement". Negative sounds such as angry voices are classified under "decline". "Neutral" means a state that is neither "excitement" nor "decline". The state related to excitement is not limited to three types. For example, there may be two types: "excitement" and "decline".

[0094] The item "value" indicates the degree of excitement. The item "value" is expressed in four stages, for example. Note that the item "value" is not limited to four stages.

[0095] 7 is a diagram showing the data structure of the voice analysis log DB 2021. As shown in FIG. 7, each record of the voice analysis log DB 2021 includes, for example, an item "location," an item "microphone ID," an item "time," an item "voice," an item "volume," an item "state," and an item "value." As will be described later, the voice analysis log DB 2021 accumulates the contents of the voice analysis log DB 3021 of each analysis device 30.

[0096] Each item in the voice analysis log DB 2021 is the same as each item in the voice analysis log DB 3021, and therefore a description of each item in the voice analysis log DB 2021 will be omitted.

[0097] FIG. 6 is a diagram showing the data structure of the image analysis log DB 3022. As shown in FIG. 6, each record in the image analysis log DB 3022 includes, for example, an item "location," an item "camera ID," an item "time," an item "facial expression," an item "number of people," an item "status," and an item "value." When a facial expression is output by the second trained model 3024, a new record is created in the image analysis log DB 3022, and each piece of information is stored in the corresponding item. The image analysis log DB 3022 of each analysis device 30 records information about the space covered by that analysis device 30.

[0098] The item "location" indicates the space that the analysis device 30 covers.

[0099] The item "camera ID" indicates identification information for identifying the image capturing device 32 installed in the space.

[0100] The item “time” indicates the time when the space was photographed by the photographing device 32 .

[0101] The item "Facial Expression" indicates the type of facial expression. For example, the item "Facial Expression" indicates the type of facial expression detected by the second trained model 3024. The item "Facial Expression" may include information related to gestures such as nodding.

[0102] The item "number of people" indicates the number of people who have taken the facial expression. For example, the item "number of people" indicates the number of people who have taken the facial expression detected by the second trained model 3024.

[0103] The item "state" indicates a state related to excitement. The item "state" includes, for example, "excitement", "decline", and "neutral". Positive expressions such as smiles are classified under "excitement". Negative expressions such as frowning are classified under "decline". "Neutral" means a state that is neither "excitement" nor "decline". The state related to excitement is not limited to three types. For example, there may be two types: "excitement" and "decline".

[0104] The item "value" indicates the degree of excitement. The item "value" is expressed in four stages, for example. Note that the item "value" is not limited to four stages.

[0105] 8 is a diagram showing the data structure of the image analysis log DB 2022. As shown in Fig. 8, each record of the image analysis log DB 2022 includes, for example, an item "location," an item "camera ID," an item "time," an item "facial expression," an item "number of people," an item "status," and an item "value." As will be described later, the image analysis log DB 2022 accumulates the contents of the image analysis log DB 3022 of each analysis device 30.

[0106] Each item in the image analysis log DB 2022 is the same as each item in the image analysis log DB 3022, and therefore a description of each item in the image analysis log DB 2022 will be omitted.

[0107] <6 operations> An example of the processing flow in the system 1 will be described.

[0108] FIG. 9 is a flowchart showing an example of the operation of analysis device 30 when analyzing audio and images.

[0109] In step S701, the analysis device 30 receives audio and images. Specifically, for example, the reception control module 3031 receives, from the microphone 31, audio data of the space collected by the microphone 31, and receives, from the image capturing device 32, image data of the space captured by the image capturing device 32.

[0110] In step S702, the control unit 303 analyzes the audio data and image data.

[0111] Specifically, the analysis control module 3033 periodically inputs, for example, received audio data, i.e., audio data collected by the microphone 31, to the first trained model 3023. The first trained model 3023 outputs information for evaluating the excitement level based on the input audio. Specifically, when audio is input, the first trained model 3023 outputs the type of audio. Specifically, the first trained model 3023 outputs, for example, laughter from the audio at 10:30:10 on April 1, 2024 in the second conference room.

[0112] The input period may be based on time, for example. This period may be set based on the approximate time it takes for a person to perform one action. Specifically, this period may be set based on the time it takes for a person to laugh out loud. For example, the analysis control module 3033 inputs voice data to the first trained model 3023 every 2 to 5 seconds.

[0113] Specifically, the analysis control module 3033 periodically inputs received image data, i.e., image data captured by the image capture device 32, to the second trained model 3024. The second trained model 3024 outputs information for evaluating excitement based on the input image. Specifically, when an image is input, the second trained model 3024 outputs the type of facial expression. When an image is input, the second trained model 3024 outputs the number of people who exhibit that type of facial expression. For example, the second trained model 3024 outputs two people smiling from an image taken at 10:30:10 on April 1, 2024 in the second conference room.

[0114] The input cycle is based on time, for example. This time may be set based on the approximate time it takes for a person to perform one action. Specifically, this time may be set based on the time it takes for a person to perform one nodding motion, for example. For example, the analysis control module 3033 inputs image data to the second trained model 3024 every 2 to 5 seconds. The image data input to the second trained model 3024 may be a single still image or multiple still images for multiple frames.

[0115] Next, specifically, the analysis control module 3033 evaluates the excitement level of the space based on the output information. More specifically, the analysis control module 3033 determines the excitement level and its degree based on the type and volume of the sound. For example, the analysis control module 3033 evaluates laughter at a volume of 2 as a value of "3" for the "excited" state. Also, more specifically, the analysis control module 3033 determines the excitement level and its degree based on facial expressions and the number of people who exhibit the same expression. For example, the analysis control module 3033 evaluates two people frowning as a value of "2" for the "decreased" state.

[0116] In this embodiment, for example, types of sound are associated in advance with excitement states. Specifically, for example, positive sound types such as cheers, laughter, and belly laughs are associated with an excitement state of "excitement." Furthermore, negative sound types such as angry shouts, silence, screams, crying, groans, and humming are associated with an excitement state of "decline." Furthermore, for example, commonly occurring sound types such as conversation, sneezing, and typing are associated with an excitement state of "neutral." Note that the relationship between sound types and excitement states is not limited to this. Furthermore, for example, volume and degree of excitement (decline) are associated in advance. Specifically, for example, volume 1 is associated with excitement (decline) degree 1, volume 2 is associated with excitement (decline) degree 2, and volume 3 is associated with excitement (decline) degree 3.

[0117] In this embodiment, for example, facial expressions are associated in advance with excitement states. Specifically, for example, positive facial expressions such as smiling and nodding are associated with an excitement state of "excited." Furthermore, for example, negative facial expressions such as crying, frowning, and shaking one's head are associated with an excitement state of "depressed." Furthermore, for example, commonly occurring facial expressions such as a serious face are associated with an excitement state of "neutral." Note that the relationship between facial expressions and excitement states is not limited to this. Furthermore, for example, the number of people and the degree of excitement (depression) are associated in advance. Specifically, for example, the number of people 1 is associated with excitement (depression) degree 1, the number of people 2 is associated with excitement (depression) degree 2, and the number of people 3 is associated with excitement (depression) degree 3.

[0118] In step S703, the control unit 303 saves information related to the analysis in step S702. Specifically, for example, the analysis control module 3033 records information corresponding to the items "microphone ID," "location," "time," "audio," "volume," "status," and "value" in the audio analysis log DB 3021. Furthermore, specifically, for example, the analysis control module 3033 records information corresponding to the items "camera ID," "location," "time," "facial expression," "number of people," "status," and "value" in the image analysis log DB 3022.

[0119] In step S704, the control unit 303 transmits information related to the analysis in step S702. Specifically, for example, the transmission control module 3032 transmits the contents of the audio analysis log DB 3021 and the contents of the image analysis log DB 3022 recorded in step S703 to the server 20.

[0120] The transmission control module 3032 continuously transmits the information recorded in step S703 to the server 20. The transmission control module 3032 may also transmit at a fixed cycle based on the volume of unsent data. Specifically, the transmission control module 3032 transmits the unsent data to the server 20, for example, every time the volume of the unsent data reaches a predetermined volume. The transmission control module 3032 may also transmit at a fixed cycle based on the time of the unsent data. Specifically, the transmission control module 3032 transmits the data to the server 20, for example, every time a predetermined time has elapsed since the most recent transmission.

[0121] FIG. 10 is a flowchart showing an example of the operation of the server 20 when analyzing the excitement in a space.

[0122] In step S801, the control unit 203 receives information related to the excitement in the space from each analysis device 30. Specifically, for example, the reception control module 2031 receives the contents of the audio analysis log DB 3021 and the contents of the image analysis log DB 3022 of each analysis device 30 transmitted in step S704.

[0123] For example, the control unit 203 records the contents of the audio analysis log DB 3021 in the audio analysis log DB 2021, and the contents of the image analysis log DB 3022 in the image analysis log DB 2022. In other words, the audio analysis log DB 2021 accumulates the contents of the audio analysis log DB 3021, and the image analysis log DB 2022 accumulates the contents of the image analysis log DB 3022.

[0124] In step S802, the control unit 203 analyzes the received information relating to the excitement in the space.

[0125] (Step S802A Evaluate collection of 1s in progress) Specifically, for example, the analysis control module 2033 evaluates the excitement of a gathering in progress in a space. For example, the analysis control module 2033 calculates an index value indicating the excitement of the gathering in terms of voice or facial expression based on information related to the excitement of the gathering in the space. Specifically, the index value increases as the "state" becomes "excited" and its "value" increases, and the index value decreases as the "state" becomes "decreased" and its "value" increases. The analysis control module 2033 may also calculate an index value comprehensively from the perspectives of both voice and facial expression. The comprehensive calculation is, for example, the sum of the voice value and the facial expression value. For example, the analysis control module 2033 calculates an index value of "4" based on a value of "3" for the "excitement" state related to voice and a value of "1" for the "excitement" state related to facial expression. For example, the analysis control module 2033 calculates an index value of "2" based on a value of "3" for the "excitement" state related to voice and a value of "1" for the "decreased" state related to facial expression. Instead of simply adding them up, it is also possible to weight at least one of them and then add them up.

[0126] Furthermore, the analysis control module 2033 continuously evaluates the excitement of one gathering, thereby calculating the time transition of the index value of the excitement of that gathering.

[0127] (Step S802B: Obtaining trends related to excitement) Specifically, for example, the analysis control module 2033 acquires trends related to excitement. For example, the analysis control module 2033 acquires trends related to excitement by comparing the temporal changes in excitement at multiple past gatherings. Specifically, for example, by comparing multiple gatherings in terms of time, place, organizer, participants, and theme, the analysis control module 2033 acquires that Monday from 10:00 to 11:00, Conference Room 3, the sales manager as the organizer, the sales department managers as participants, and sales reports are elements that liven up a gathering. Furthermore, specifically, for example, by comparing multiple gatherings in terms of speech, the analysis control module 2033 acquires that icebreakers and closing talks at the beginning of a gathering are elements that liven up a gathering.

[0128] The collections to be compared may be collections at the same time but in different spaces, collections at different times but in the same space, or collections at different times but in different spaces.

[0129] (Step S802C: Obtain and utilize exemplary transitions) Specifically, for example, the analysis control module 2033 acquires an exemplary transition of excitement. For example, the analysis control module 2033 acquires a predetermined exemplary transition trajectory by referring to past gatherings that generally exhibited high excitement. The predetermined exemplary transition trajectory is, for example, a generally upward trajectory with periodic peaks of excitement.

[0130] Specifically, for example, the analysis control module 2033 utilizes an exemplary transition of excitement. For example, the analysis control module 2033 scores the excitement of the ongoing gathering by comparing the transition of excitement of the ongoing gathering evaluated in step S802A with the exemplary transition. The more closely the transition of excitement of the ongoing gathering follows the exemplary transition, the higher the score calculated for the transition of excitement of the ongoing gathering. The gathering may also be scored after completion. The score may be, for example, a numerical score out of 100 points or an alphabetical score from A to F.

[0131] (Step S802D Evaluate the establishment) Specifically, for example, the analysis control module 2033 evaluates the activity of the entire establishment. For example, the analysis control module 2033 calculates an index representing the activity of gatherings in all spaces that make up one establishment based on the index values ​​indicating the activity of gatherings in each space calculated in step 802A. For example, the index is the sum of the index values ​​in each space. Note that the index may be calculated using a predetermined statistical method rather than just the sum. The index may also be the sum of index values ​​that are scaled according to the size of each space. For example, even if the original index values ​​of two spaces are the same, if one space is twice the size of the other space, the index value of one space will be twice the index value of the other space.

[0132] Specifically, for example, the analysis control module 2033 calculates statistical data regarding the activity of the establishment. For example, the analysis control module 2033 calculates the average value of the index for a divided time period. The divided time period may be, for example, a morning, a day of the week, a week, a month, a year, etc. Furthermore, for example, the analysis control module 2033 may calculate the median, maximum, and minimum values ​​of the index for the divided time period.

[0133] (Step S802E: Verify the effect of the action) Specifically, for example, the analysis control module 2033 verifies the effectiveness of an action to stimulate excitement in generating excitement. For example, the analysis control module 2033 verifies the effectiveness of the action in generating excitement by observing the transition of the excitement index value calculated in step S802A for a certain period of time from the time the action was performed. If the index value increases for a certain period of time from the time the action was performed, the analysis control module 2033 determines that the action has increased, and thereby determines that the action is effective in generating excitement.

[0134] Actions to stimulate excitement are activities related to human activity, such as radio calisthenics, serving coffee, and giving a speech at the kickoff of a project.

[0135] (Step S802F Select music and volume) Specifically, for example, the analysis control module 2033 selects the music and volume to be played in a space according to the index value of the excitement of the gathering in that space calculated in step 802A. For example, the analysis control module 2033 selects an up-tempo song and a loud volume to promote excitement in a space with a low current index value. Also, for example, the analysis control module 2033 lowers the volume in a space with a high current index value to maintain excitement.

[0136] The songs may be downloaded from the Internet each time they are selected, or may be stored in the storage unit 202, for example.

[0137] In step S803, the control unit 203 provides feedback to the space based on the analysis in step S802.

[0138] (Step S803A: Display on the terminal in the space) Specifically, for example, the transmission control module 3032 transmits the analysis results related to the space to the terminal device 10. For example, the transmission control module 3032 transmits the transition of the index value of the excitement of the gathering in one space calculated in step S802A, and the exemplary transition of the excitement and the score for the excitement acquired in step S802C, to the terminal device 10 installed in the space. The transmission control module 3032 may transmit the data continuously in real time while the gathering is in progress, or may transmit the data once when the gathering is completed.

[0139] (Step S803B Display on business terminal) Specifically, for example, the transmission control module 3032 transmits the analysis results related to the establishment to the terminal device 10. For example, the transmission control module 3032 transmits the index representing the activity of the entire establishment calculated in step S802D to the terminal device 10 installed in the establishment.

[0140] (Step S803C Speaker) Specifically, for example, the transmission control module 3032 transmits the music to be played in the space to the speaker 33. For example, the transmission control module 3032 transmits the music and volume selected in step S802F to the speaker 33 installed in the space.

[0141] <7 Screen example> An example of a screen according to the present disclosure will be described.

[0142] Fig. 11 is a schematic diagram showing an example of a display screen of the display 141 of the terminal device 10. Fig. 11 illustrates an example in which the excitement of a gathering in progress in the conference room 201 is displayed.

[0143] Text 9001 indicates the space where the gathering is taking place.

[0144] Text 9002 is text that indicates the current time.

[0145] Graph 9003 is a graph for expressing the transition of the index value of the excitement of the gathering. Graph 9003 is composed of a vertical axis showing the index value of the excitement and a horizontal axis showing the transition over time. The larger the value on the vertical axis, the higher the index value of the excitement. The origin of the horizontal axis may indicate, for example, the start time of the ongoing gathering or the start time of the business.

[0146] The solid line 9004 depicts the progression of the ongoing excitement index calculated in step S802A. The solid line 9004 may depict index values ​​based on real-time processing, batch processing, or a collection of index values ​​after the completion of a gathering. Batch processing may be performed, for example, every 30 seconds.

[0147] Dashed line 9005 is a dashed line for comparison with solid line 9004. Specifically, for example, dashed line 9005 depicts the transition of the average value of the index value for the same time period in the past. Also, specifically, for example, dashed line 9005 may depict the exemplary transition obtained in step S802C.

[0148] The dashed line 9005 may depict only the transition up to the current time, or may depict the transition beyond the current time.

[0149] Fig. 12 is a schematic diagram showing an example of a display screen of the display 141 of the terminal device 10. Fig. 12 illustrates an example in which the scores of a completed meeting in conference room 201 are displayed.

[0150] Text 1001 is text that indicates the space where the gathering took place.

[0151] Text 1002 is text indicating the time the gathering took place.

[0152] Icon 1003 is an icon that describes the excitement of the gathering. The analysis control module 2033 divides the current gathering into time segments and compares the trend of the gathering at each segment with the average trend of the past. The analysis control module 2033 determines that the gathering is excited if the overall trend of the gathering is higher than the average trend, and determines that the gathering is declining if the overall trend of the gathering is lower than the average trend. For example, the analysis control module 2033 outputs a sunny, then cloudy icon to a gathering that was excited in the first half of the gathering but became less exciting in the second half. The presentation control module 2034 presents the output icon to the user.

[0153] Text 1004 is text that indicates the score for the excitement of the gathering. For example, text 1004 indicates the score calculated in step S802C.

[0154] Text 1005 is text that represents advice on how to liven up the gathering. The analysis control module 2033 compares the progress of the current gathering with the model progress and analyzes how much the excitement of the gathering falls short of the model progress. The analysis control module 2033 outputs advice based on the timing of the excitement. For example, the analysis control module 2033 outputs advice such as "Next time, try to liven up the gathering with a closing talk" to a gathering that was lively in the first half but waned in the second half compared to the model progress. The presentation control module 2034 presents the output advice to the user.

[0155] Fig. 13 is a schematic diagram showing an example of a display screen of the display 141 of the terminal device 10. Fig. 13 illustrates an example in which the activity of the Azabudai office is displayed.

[0156] Text 1101 is text that indicates the business establishment and the number of people who came to the business establishment today.

[0157] Graph 1102 is a graph for expressing emotional beats, i.e., the excitement at work. Graph 1102 is composed of a vertical axis showing an excitement index and a horizontal axis showing time progression. The larger the value on the vertical axis, the higher the excitement index. The horizontal axis shows time, such as a day, a week, a month, three months, six months, or a year. The time on the horizontal axis may be switched by the user via an interface operation.

[0158] Icons representing emotions may be added to the graph 1102. For example, an icon representing a smile may be added to a scene in the graph 1102 where a predetermined index is exceeded.

[0159] A solid line 1103 depicts the transition of the excitement index of the first business establishment calculated in step S802D. The solid line may be plotted by real-time processing or by batch processing. Batch processing is performed, for example, every 30 seconds.

[0160] The dashed line 1104 is a dashed line for comparison with the solid line 1103. Specifically, for example, the dashed line 1104 depicts the transition of the average value of the index for the establishment at the same time in the past.

[0161] The dashed line 1104 may depict only the progression up to the current time, or may depict the progression beyond the current time.

[0162] The pie chart 1105 shows the emotional rate, i.e., the proportion of facial expressions associated with the workplace. Specifically, for example, the pie chart 1105 is a pie chart showing the proportion of people with specific facial expressions among the number of people attending work. Specifically, the analysis control module 2033 calculates the sum of the number of people whose "status" is "excited," the sum of the number of people whose "status" is "depressed," and the sum of the number of people whose "status" is "neutral" from among the records in the image analysis log DB 2022 whose "time" item is the current time. The analysis control module 2033 also calculates the number of "others" by subtracting the sum of the above three sums from the current number of people attending work. Next, specifically, the analysis control module 2033 calculates the proportion of the number of people whose "excited" status is the proportion of the number of people who attended work, the proportion of the number of people whose "depressed" status is the proportion of the number of people who attended work, the proportion of the number of people whose "neutral" status is the proportion of the number of people who attended work, and the proportion of the number of people whose "others" status is the proportion of the number of people who attended work. Specifically, the analysis control module 2033 then outputs the percentage of "excited" people with a smiley face icon, the percentage of "depressed" people with an anxious face icon, the percentage of "neutral" people with a serious face icon, and the percentage of "other" people with a document icon. The presentation control module 2034 presents the output percentages and icons to the user.

[0163] A list 1106 shows actions to promote excitement in the business establishment and the participants involved.

[0164] Fig. 14 is a schematic diagram showing an example of a display screen of the display 141 of the terminal device 10. Fig. 14 illustrates an example in which the real-time excitement of gatherings in each space in a business establishment is displayed.

[0165] Map 1201 is a map showing the entire business establishment and the spaces that make up the establishment. A space is a place where gatherings take place and the atmosphere is evaluated, and text indicating the name of the space is attached. The name of the space may be, for example, Conference Room A, Salon Table 1, etc. Map 1201 may be drawn in two dimensions or in three dimensions.

[0166] The volume meters 1202, exemplified by volume meters 1202-1 to 1202-3, are volume meters that indicate the real-time excitement of the crowd in the space based on the index value that indicates the real-time excitement calculated in step S802A. For example, the volume meters 1202 divide the possible index values ​​from the highest to the lowest into five stages, and express the real-time index value with five bars corresponding to the five stages. The higher the index value, i.e., the more excited the crowd, the more the bar of the volume meters 1202 fills up.

[0167] An icon indicating the trend of excitement may be added to the volume meter 1202. For example, an upward arrow is added to the volume meter of a space where excitement has been on an upward trend over the past five minutes.

[0168] <8 Summary> As described above, in the above embodiment, system 1 includes a means for collecting audio generated in a predetermined space (such as an office space or a conference room) and installed in the predetermined space; a means for classifying the types of audio collected by the audio collecting means and determining a state and a level of excitement in the state based on the classified type and volume; a means for aggregating the state and level and calculating a time transition of the excitement in the space; and a means for presenting information based on the time transition of the excitement in the space to a user. This makes it possible to calculate the excitement in the space without transmitting raw audio data and image data to the cloud. Therefore, the program, method, information processing device, and system according to this embodiment can automatically quantify the excitement in a predetermined space from the perspective of audio while maintaining security.

[0169] In the above embodiment, the system 1 includes a means for capturing images of the space, and the determining means classifies the types of facial expressions included in the images collected by the capturing means, and determines the state and degree of excitement based on the classified type and the number of people with that type of facial expression. This makes it possible to automatically quantify the excitement in a specified space in terms of the participants' facial expressions while maintaining security, without transmitting raw audio data and image data to the cloud.

[0170] In the above embodiment, the system 1 also includes a means for comparing the transitions in the excitement of multiple gatherings in a space and acquiring a predetermined trend related to the excitement. This allows the server 20 to acquire measures to promote excitement while comparing multiple gatherings. In the above embodiment, the system 1 also includes a means for presenting improvement suggestions related to the excitement based on the acquired trend. This allows users to know measures to promote excitement and improve the vitality of gatherings in which they participate.

[0171] Furthermore, in the above embodiment, the system 1 includes a means for verifying the effectiveness of an action on excitement based on an index value indicating the excitement resulting from the action. This allows users to recognize what actions contribute to excitement. Conventionally, it has been difficult for users to quantitatively verify measures to promote excitement, but this allows users to know measures that are backed up by quantitative verification.

[0172] Furthermore, in the above embodiment, the system 1 includes a means for statistically calculating an index representing the activity of the entire business entity based on index values ​​indicating the activity levels in multiple spaces. This allows the activity of the entire business entity to be automatically quantified. While it has traditionally been difficult to visualize the activity level of a business entity, this embodiment makes it possible to visualize the activity level of the business entity. This allows users to estimate the status of the business entity based on the visualized activity level. For example, if the activity level of a business entity is shown to be low, the business entity manager can decide to increase the activity level of the business entity by launching a new project.

[0173] In the above embodiment, the system 1 also includes a means for playing music in the space according to the excitement index value of the space, thereby promoting excitement in the space and maintaining a high level of excitement.

[0174] <9 Variations> In the above embodiment, a microphone is used to collect sounds in a space and a camera is used to capture images of the space to measure the level of excitement. However, a beacon may also be used to obtain the number of people in the space and their positions to measure the level of excitement.

[0175] For example, a user terminal on which a dedicated beacon app is installed receives a beacon signal transmitted from an analysis device 30 installed in a space. Next, for example, the user terminal transmits its location information to a nearby analysis device 30, and the analysis device 30 transmits the location information of the user terminal to the server 20. Alternatively, the user terminal may directly transmit the location information of the user terminal to the server 20. As a result, the server 20 obtains the number of people in the space and their locations. A user may have a device that transmits beacon signals, and the analysis device 30 may respond to the beacon signals transmitted from the device.

[0176] The analysis device 30 may have a function to analyze the number, position, etc. in the space. The analysis device 30 determines the state of excitement and the degree of the state based on the detected number. Specifically, the analysis device 30 determines the state of excitement and the degree of the state based on the number, density, amount of change, or a combination of at least two of the detected users. For example, if the number of detected users has increased by more than a predetermined number in a short period of time, the analysis device 30 evaluates the value of the state "excitement" as "3." The analysis device 30 transmits the analysis results to the server 20.

[0177] In this modification, the system 1 includes, for example, a means for detecting the positions of users installed in a predetermined space, and a determining means for determining the state and the degree of excitement based on the number of detected users, thereby automatically quantifying the excitement in the predetermined space in terms of the positions of users.

[0178] Furthermore, people within a space may be identified using identification numbers assigned to each person using beacons.

[0179] For example, a user terminal on which a dedicated beacon app is installed receives a beacon signal transmitted from an analysis device 30 installed in a space. Next, for example, the user terminal transmits location information of the user terminal to a nearby analysis device 30. The analysis device 30 transmits the location information of the user terminal to the server 20. Alternatively, the user terminal may transmit the location information of the user terminal directly to the server 20. The transmitted and received information includes an identification number assigned to each person's account for the app. The server 20 identifies people in the space based on the identification number. A user may have a device that transmits beacon signals, and the analysis device 30 may respond to the beacon signals transmitted from the device.

[0180] The server 20 analyzes the correlation between the excitement in the space and the person specified based on the identification number, thereby obtaining a correlation between the excitement in the space and the person in the space, such as, for example, the space becoming more exciting when a specific person is present in the space.

[0181] In the above embodiment, the analysis device 30 does not have a display. However, the analysis device 30 may have a display for showing a user information related to the excitement of the space in which the analysis device 30 is installed. For example, the presentation control module 3034 displays, on the display of the analysis device 30, the transition of the index value of the excitement of the gathering in one space calculated in step S802A, and the exemplary transition of the excitement and the score for the excitement acquired in step S802C.

[0182] <10 Basic computer hardware configuration> 15 is a block diagram showing the basic hardware configuration of a computer 90. The computer 90 includes at least a processor 901, a main memory device 902, an auxiliary memory device 903, and a communication IF 991 (interface), which are electrically connected to one another by a communication bus 921.

[0183] The processor 901 is hardware for executing an instruction set written in a program, and is composed of an arithmetic unit, registers, peripheral circuits, and the like.

[0184] The main memory device 902 is used to temporarily store programs, data to be processed by the programs, etc. For example, it is a volatile memory such as a DRAM (Dynamic Random Access Memory).

[0185] The auxiliary storage device 903 is a storage device for saving data and programs, such as a flash memory, a hard disk drive (HDD), a magneto-optical disk, a CD-ROM, a DVD-ROM, or a semiconductor memory.

[0186] The communication IF 991 is an interface for inputting and outputting signals for communicating with other computers via a network using wired or wireless communication standards. The network is composed of the Internet, a LAN, various mobile communication systems constructed by wireless base stations, etc. For example, the network includes 3G, 4G, and 5G mobile communication systems, LTE (Long Term Evolution), and wireless networks (e.g., Wi-Fi (registered trademark)) that can connect to the Internet via a predetermined access point. In the case of a wireless connection, communication protocols include, for example, Z-Wave (registered trademark), ZigBee (registered trademark), and Bluetooth (registered trademark). In the case of a wired connection, the network also includes a direct connection using a USB (Universal Serial Bus) cable, etc.

[0187] It should be noted that the computer 90 can be virtually realized by distributing all or part of each hardware configuration across multiple computers 90 and interconnecting them via a network. In this way, the computer 90 is a concept that includes not only a computer 90 housed in a single housing or case, but also a virtualized computer system.

[0188] <Basic functional configuration of computer 90> A description will now be given of the functional configuration of the computer realized by the basic hardware configuration of the computer 90. The computer comprises at least the functional units of a control unit, a storage unit, and a communication unit.

[0189] The functional units of the computer 90 can also be realized by distributing all or part of the functional units among multiple computers 90 interconnected via a network. The computer 90 is a concept that includes not only a single computer 90 but also a virtualized computer system.

[0190] The control unit is realized by the processor 901 reading out various programs stored in the auxiliary storage device 903, expanding them in the main storage device 902, and executing processing in accordance with the programs. The control unit can realize functional units that perform various types of information processing depending on the type of program. In this way, the computer is realized as an information processing device that performs information processing.

[0191] The storage unit is realized by a main storage device 902 and an auxiliary storage device 903. The storage unit stores data, various programs, and various databases. Furthermore, the processor 901 can allocate a storage area corresponding to the storage unit in the main storage device 902 or the auxiliary storage device 903 in accordance with the programs. Furthermore, the control unit can cause the processor 901 to execute processes for adding, updating, and deleting data stored in the storage unit in accordance with the various programs.

[0192] A database refers to a relational database, which manages data sets called masters and tables in a tabular format structurally defined by rows and columns, by relating them to each other. In a database, a table is called a table, a master, a column in a table is called a column, and a row in a table is called a record. In a relational database, relationships between tables and masters can be set and associated. Typically, each table and each master has a column set as a primary key to uniquely identify a record, but setting a primary key to a column is not essential. The control unit can cause the processor 901 to add, delete, or update records in specific tables and masters stored in the storage unit according to various programs. Furthermore, by storing data, various programs, and various databases in the storage unit, it can be considered that the information processing device and information processing system according to the present disclosure have been manufactured.

[0193] Note that the databases and masters in this disclosure may include any data structure in which information is structurally defined (such as a list, dictionary, associative array, or object). The data structure also includes data that can be considered as a data structure by combining data with functions, classes, methods, etc. written in any programming language.

[0194] The communication unit is realized by the communication IF 991. The communication unit realizes a function of communicating with other computers 90 via a network. The communication unit can receive information transmitted from other computers 90 and input the information to the control unit. The control unit can cause the processor 901 to execute information processing on the received information in accordance with various programs. In addition, the communication unit can transmit information output from the control unit to other computers 90.

[0195] Furthermore, the above-described configurations, functions, processing units, processing means, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. The present invention can also be realized by software program code that implements the functions of the embodiments. In this case, a storage medium on which the program code is recorded is provided to a computer, and a processor included in the computer reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium implements the functions of the above-described embodiments, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media for providing such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs, optical disks, magneto-optical disks, CD-Rs, magnetic tape, non-volatile memory cards, and ROMs.

[0196] Furthermore, the program code that realizes the functions described in this embodiment can be implemented in a wide range of program or script languages, such as assembler, C / C++, perl, Shell, PHP, and Java (registered trademark).

[0197] Furthermore, the program code of the software that realizes the functions of the embodiments may be distributed via a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the processor of the computer may read and execute the program code stored in the storage means or storage medium.

[0198] The functions performed by the components described herein may be implemented in circuitry or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), a CPU (a Central Processing Unit), conventional circuits, and / or combinations thereof, programmed to perform the described functions. A processor includes transistors and other circuits and is considered to be circuitry or processing circuitry. A processor may also be a programmed processor that executes programs stored in memory. In this specification, a circuitry, unit, or means is hardware that is programmed to realize or performs the described functions, which may be any hardware disclosed herein or any hardware known to be programmed to realize or perform the described functions. If the hardware is a processor considered to be a type of circuitry, the circuitry, means, or unit is a combination of the hardware and software used to configure the hardware and / or processor.

[0199] Although several embodiments of the present disclosure have been described above, these embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and modifications are intended to be included in the scope of the inventions and their equivalents as defined in the claims, as well as in the scope and spirit of the inventions.

[0200] <Additional Notes> The matters described in the above embodiments will be supplemented below. (Appendix 1) A system comprising: means for being installed in a specified space and collecting sounds generated in the space; means for being installed in the space and classifying the types of sounds collected by the means for collecting sounds, and determining a state and degree of excitement in the state based on the classified type and volume; means for aggregating the state and degree and calculating the time progression of excitement in the space; and means for presenting information based on the time progression of excitement in the space to a user. (Appendix 2) A system as described in (Appendix 1), which is installed in a space and includes a means for capturing images within the space, and the determining means classifies the types of facial expressions contained in the images collected by the capturing means, and determines the state of excitement and the degree of said state based on the classified type and the number of people with said type of facial expression. (Appendix 3) A system as described in (Appendix 1) or (Appendix 2), which is installed in a space and includes means for detecting the position of users within the space, and the determining means determines the excitement state and the degree of the state based on the detected number. (Appendix 4) A system described in any one of (Appendix 1) to (Appendix 3), wherein the specified space is a space within an office. (Appendix 5) A system according to any one of (Appendix 1) to (Appendix 3), wherein the specified space is a conference room. (Appendix 6) A system described in any one of (Appendix 1) to (Appendix 5), comprising means for comparing the trends in the excitement of multiple gatherings in a space and acquiring a predetermined trend regarding the excitement. (Appendix 7) The system described in (Appendix 6), wherein the means for acquiring trends presents improvement suggestions regarding excitement based on the acquired trends. (Appendix 8) A system described in any one of (Appendix 1) to (Appendix 7), comprising a means for verifying the effectiveness of an action in generating excitement based on an index value indicating the excitement that has been generated by the action to promote excitement. (Appendix 9) A system described in any one of (Appendix 1) to (Appendix 8), comprising a means for statistically calculating an index representing the activity of the entire business entity based on index values ​​indicating the activity in multiple spaces. (Appendix 10) The system according to any one of (Supplementary Note 1) to (Supplementary Note 9), comprising a means for playing music in the space according to the excitement index value of the space. (Appendix 11) A method executed in a system comprising a microphone, an analysis device, and a server, the method comprising the steps of: a) installing the microphone in a specified space and collecting sounds generated in the space; b) installing the analysis device in the space and classifying the types of sounds collected in the sound collecting step, and determining a state of excitement and the degree of said state based on the classified type and volume; c) aggregating the state and degree and calculating the time progression of excitement in the space; and d) presenting information based on the time progression of excitement in the space to a user. (Appendix 12) A program for operating a computer installed in a specified space, the program having a processor and a memory, the program causing the processor to execute the steps of acquiring sounds occurring in the space collected by a microphone installed in the space, and classifying the types of acquired sounds, and determining a state of excitement and the degree of said state based on the classified type and volume. (Appendix 13) An information processing device that is installed in a specified space and includes a control unit and a memory unit, wherein the control unit executes the steps of acquiring audio generated in the space collected by a microphone installed in the space, and classifying the type of acquired audio, and determining a state of excitement and the degree of that state based on the classified type and volume. [Explanation of symbols]

[0201] 1. System 10...Terminal device 12...Communication IF 13...Input device 14...Output device 15...Memory 16…Storage 19...Processor 20...Server 22...Communication IF 23...Input / output interface 25…Memory 26…Storage 29...Processor 30...Information processing device 31...Mike 32...Photographic equipment 33...Speaker 40…Network

Claims

1. a means for collecting sounds generated in a predetermined space; a means for classifying types of sounds collected by the means for collecting sounds, and determining a state of excitement and a level of the state based on the classified type and volume; a means for aggregating the state and degree of excitement and calculating the time transition of excitement in the space; means for presenting information based on the time transition of the excitement in the space to a user; A system comprising:

2. a means for capturing an image of the space, the means being installed in the space; The system described in claim 1, wherein the determining means classifies the types of facial expressions contained in the images collected by the photographing means, and determines the state of excitement and the degree of the state based on the classified type and the number of people with that type of facial expression.

3. means for detecting a position of a user within the space; The system according to claim 1 , wherein the determining means determines a state of excitement and a degree of the state based on the detected number.

4. The system of claim 1 , wherein the predetermined space is a space within an office.

5. The system of claim 1 , wherein the predetermined space is a conference room.

6. The system according to claim 1 , further comprising means for comparing the transition of the excitement of a plurality of gatherings in the space and acquiring a predetermined trend regarding the excitement.

7. The system of claim 6 , wherein the means for acquiring trends provides improvement suggestions for excitement based on the acquired trends.

8. The system according to claim 1 , further comprising means for verifying the effect of an action on excitement based on an index value indicating excitement received from the action for promoting excitement.

9. The system according to claim 1 , further comprising means for statistically calculating an index representing the activity of the entire business entity based on index values ​​indicating the activity in a plurality of the spaces.

10. The system according to claim 1 , further comprising means for playing music in the space according to the excitement index value of the space.

11. 1. A method performed on a system including a microphone, an analysis device, and a server, comprising: The microphone is installed in a predetermined space and collects sounds generated in the space; a step in which the analysis device is installed in the space, classifies the type of sound collected in the step of collecting the sound, and determines a state of excitement and a degree of the state based on the classified type and volume; the server tallying up the state and degree and calculating the transition of the excitement in the space over time; a step in which the server presents information based on a time transition of the excitement in the space to a user; How to do it.

12. A program for operating a computer that is installed in a predetermined space and that includes a processor and a memory, the program causing the processor to: acquiring sounds generated in the space, the sounds being collected by microphones installed in the space; classifying the acquired type of sound, and determining a state of excitement and a degree of the state based on the classified type and volume; A program that executes the following.

13. An information processing device installed in a predetermined space, comprising a control unit and a storage unit, wherein the control unit: acquiring sounds generated in the space, the sounds being collected by microphones installed in the space; classifying the acquired type of sound, and determining a state of excitement and a degree of the state based on the classified type and volume; An information processing device that executes the above.

Images