Information processing device, information processing method, and information processing program

Abstract

To provide an information processing device capable of detecting water wetting on a surface of a terminal device using sound output from a speaker and acquired by a microphone, an information processing method, and an information processing program.SOLUTION: In an information processing system 1, a server device 100 comprises: an instruction unit which issues an instruction to output sound from a speaker SK attached to a housing of a terminal device 10; a detection unit which detects a foreign matter adhering to the housing of the terminal device by using active acoustic sensing on the basis of sound acquired by a microphone MK attached to the housing of the terminal device; and a learning unit which learns characteristics of sound with the foreign matter adhering to the housing of the terminal device and constructs a model. The detection unit also detects the foreign matter adhering to the housing of the terminal device by using the model.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] An active acoustic sensing technology has been disclosed that recognizes hand gestures by attaching a speaker and microphone to the arm. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Study on hand gesture recognition method using active acoustic sensing, Information Processing Society of Japan Research Report, August 6, 2016, [Retrieved November 22, 2021], Internet <URL:https: / / ipsj.ixsq.nii.ac.jp / ej / ?action=repository_action_common_download&item_id=171767&item_no=1&attribute_id=1&file_no=1> Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned conventional technologies merely recognize gestures, and there is room for providing more services to users. For example, the recent emergence of waterproof smart devices has made it possible to easily operate devices near water such as the sea or pool, in wet areas such as kitchens and bathrooms, and even outdoors in the rain. However, with capacitive touch sensing, false touch detection (touch occurs at the location of the water droplets, or the touch position is shifted due to the water droplets) can occur when water droplets are present on the screen.

[0005] The present application has been made in view of the above, and aims to detect water wetting of the surface of a terminal device using sound output from a speaker and picked up by a microphone. [Means for solving the problem]

[0006] The information processing device of the present application is characterized by comprising a command unit that issues a command to output sound from a speaker attached to the housing of a terminal device, and a detection unit that detects the attachment of foreign matter to the housing of the terminal device using active acoustic sensing based on sound acquired by a microphone attached to the housing of the terminal device. [Effects of the Invention]

[0007] According to one aspect of the embodiment, water wetting of the surface of the terminal device can be detected by the sound output from the speaker and picked up by the microphone. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is an explanatory diagram showing an overview of an information processing method according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of the configuration of an information processing system according to the embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of the configuration of a terminal device according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of the configuration of a server device according to the embodiment. [Figure 5] FIG. 5 is a diagram illustrating an example of the user information database. [Figure 6] FIG. 6 is a diagram illustrating an example of the history information database. [Figure 7] FIG. 7 is a diagram illustrating an example of the detection information database. [Figure 8] FIG. 8 is a flowchart showing a processing procedure according to the embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of a hardware configuration. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an information processing device, an information processing method, and an information processing program according to the present application (hereinafter referred to as "embodiments") will be described in detail with reference to the drawings. Note that the information processing device, the information processing method, and the information processing program according to the present application are not limited to these embodiments. Furthermore, the same components in the following embodiments will be denoted by the same reference numerals, and duplicated descriptions will be omitted.

[0010] [1. Overview of information processing method] First, an overview of an information processing method performed by an information processing device according to an embodiment will be described with reference to Fig. 1. Fig. 1 is an explanatory diagram showing an overview of an information processing method according to an embodiment. Note that Fig. 1 explains an example in which water wetting of the surface of a terminal device is detected by sound output from a speaker and picked up by a microphone.

[0011] 1, the information processing system 1 includes a terminal device 10 and a server device 100. The terminal device 10 and the server device 100 are connected to each other via a network N (see FIG. 2) in a wired or wireless manner so as to be able to communicate with each other. In this embodiment, the terminal device 10 cooperates with the server device 100.

[0012] The terminal device 10 is a smart device such as a smartphone or tablet terminal used by a user U, and is a portable terminal device capable of communicating with any server device via a wireless communication network such as 4G (Generation (4G)) or LTE (Long Term Evolution) networks. The terminal device 10 has a screen such as a liquid crystal display with a touch panel function, and accepts various operations on displayed data such as content, such as tapping, sliding, and scrolling, performed by the user U with a finger or a stylus. An operation performed on an area of ​​the screen where content is displayed may be considered an operation on the content. The terminal device 10 may be not only a smart device, but also an information processing device such as a desktop PC (Personal Computer) or a notebook PC.

[0013] The server device 100 is an information processing device that works in conjunction with the terminal device 10 of each user U and provides API (Application Programming Interface) services for various applications (hereinafter referred to as apps) and various data to the terminal device 10 of each user U, and is realized by a computer, a cloud system, etc.

[0014] The server device 100 may also be an information processing device that provides some kind of online web service to the terminal device 10 of each user U. For example, the server device 100 may provide the following web services: internet connection, search service, social networking service (SNS), electronic commerce (EC), electronic payment, online games, online banking, online trading, hotel and ticket reservations, video and music distribution, news, maps, route search, route guidance, line information, operation information, and weather forecast. In practice, the server device 100 may cooperate with various servers that provide the above-mentioned web services and act as an intermediary for the web services or may be responsible for processing the web services.

[0015] The server device 100 can acquire user information about the user U. For example, the server device 100 acquires information about the attributes of the user U, such as the gender, age, and residential area of ​​the user U. The server device 100 then stores and manages the information about the attributes of the user U together with identification information (such as a user ID) that identifies the user U.

[0016] The server device 100 also acquires various types of history information (log data) indicating the behavior of the user U from the terminal device 10 of the user U or from various servers based on the user ID, etc. For example, the server device 100 acquires a location history, which is a history of the user U's location and date and time, from the terminal device 10. The server device 100 also acquires a search history, which is a history of search queries entered by the user U, from a search server (search engine). The server device 100 also acquires a browsing history, which is a history of content viewed by the user U, from a content server. The server device 100 also acquires a purchase history (payment history), which is a history of the user U's product purchases and payment processes, from an e-commerce server or a payment processing server. The server device 100 may also acquire a listing history and a sales history, which are a history of the user U's listings on the marketplace, from the e-commerce server or the payment processing server. The server device 100 also acquires a posting history, which is a history of the user U's posts, from a posting server or SNS server that provides a word-of-mouth posting service.

[0017] [1-1. Water detection using active acoustic sensing] Generally, active acoustic sensing technology generates an acoustic signal using a speaker and captures the reverberant sound and sound propagating through the enclosure as an acoustic signal using a microphone.The captured acoustic signal is then analyzed using FFT (Fast Fourier Transformation), and the signal strength for each frequency band is used as a feature for machine learning.

[0018] In this embodiment, the server device 100 detects water damage (water droplets) on the surface of the terminal device 10 using active acoustic sensing, which outputs sound from a speaker attached to the housing of the terminal device 10 of each target user U and captures the sound with a microphone attached to the housing of the terminal device 10.

[0019] As shown in Fig. 1, the server device 100 issues a command (instruction) via a network N (see Fig. 2) to output sound from a speaker SK attached to the housing of the terminal device 10 of each user U (step S1). The command (instruction) may be data or a signal. Note that if the speaker SK is independent of the terminal device 10, the server device 100 may directly issue a command to the speaker SK to output sound.

[0020] Next, the terminal device 10 of the user U receives a command from the server device 100 and outputs sound from a speaker SK attached to the housing of the terminal device 10 (step S2). For example, the speaker SK is a vibration speaker that can be used for active acoustic sensing. The speaker SK may be attached externally to the surface of the housing, or may be built into or mounted on the housing.

[0021] Next, the terminal device 10 of the user U acquires the sound using a microphone MK attached to the housing of the terminal device 10 (step S3). For example, the microphone MK is a piezo microphone that can be used for active acoustic sensing. The microphone MK may be attached externally to the surface of the housing, or may be built into or mounted in the housing.

[0022] Next, the terminal device 10 of the user U transmits information about the sound acquired by the microphone MK to the server device 100 via the network N (see FIG. 2) (step S4). If the microphone MK is independent from the terminal device 10, the microphone MK may directly transmit information about the acquired sound to the server device 100.

[0023] Next, the server device 100 receives information about the sound picked up by the microphone MK from the terminal device 10 of each user U, and detects whether the housing of the terminal device 10 has been wetted by water using active acoustic sensing (step S5). The housing of the terminal device 10 includes a screen DP. The server device 100 detects whether the screen DP of the housing has been wetted by water.

[0024] The server device 100 may detect whether the housing of the terminal device 10 is wet based on the sound characteristics (properties) when the housing of the terminal device 10 is wet and the sound characteristics when the housing is not wet. In this case, the server device 100 constructs (generates, updates) a model that has learned the sound characteristics when the housing is wet, and detects whether the housing of the terminal device 10 is wet using the model.

[0025] For example, the server device 100 constructs a model by learning the characteristics of the sound (output sound) output from the speaker SK and the characteristics of the sound (acquired sound) acquired by the microphone MK when the housing of the terminal device 10 is wet. In this case, the server device 100 constructs a model by learning the characteristics of the acquired sound (or both the characteristics of the output sound and the characteristics of the acquired sound). Alternatively, the server device 100 constructs a model by learning the difference between the output sound and the acquired sound.

[0026] The server device 100 may construct a model indicating which part is wet by learning the characteristics of the acquired sound (or the characteristics of the difference between the output sound and the acquired sound) when a predetermined part (place, part, portion) of the housing of the terminal device 10 is wet. For example, the server device 100 may construct a model that outputs information indicating which part of the housing of the terminal device 10 (rear face, screen, microphone part, speaker part, charging connector part, etc.) is wet when the acquired sound is input.

[0027] At this time, the server device 100 may build a model by learning for each type (model, size, etc.) of the terminal device 10. The server device 100 may also build a model by learning for each peripheral device, part, or accessory (cover, ring, etc.) of the terminal device 10.

[0028] Furthermore, the server device 100 may change a threshold for determining contact with a touch panel or the like of the housing of the terminal device 10 based on the result of detecting water damage to the housing of the terminal device 10. Furthermore, the server device 100 may determine whether an operation on the terminal device 10 is valid or invalid based on the result of detecting water damage to the housing of the terminal device 10. In this way, active acoustic sensing technology is used on the smart device to detect the presence of water droplets (a state in which water droplets are attached) and reduce erroneous touch operations. When water droplets are attached, the touch sensitivity threshold is changed so that touch input is not detected based on signals that may be generated by water droplets. Furthermore, because water droplets spread but a finger does not change shape as much as a water droplet, a threshold may be set for the contact area value in addition to the touch sensitivity.

[0029] In addition, the server device 100 may perform control such as "cancel the operation because it is wet" based on the results that "there is a mysterious touch" and "water damage has been detected (water droplet detection)" on the terminal device 10 of each user U.

[0030] In addition, the server device 100 provides information regarding the detection results of water damage to the housing of each user U's terminal device 10 via the network N (see Figure 2) to the terminal device 10 being detected (or the other terminal device 10 communicating with that terminal device).

[0031] Furthermore, the server device 100 may detect not only water damage (water droplets) on the surface of the terminal device 10 but also dirt (dust, oil film, air bubbles, etc.) on the surface of the terminal device 10. In this case, the server device 100 may construct a model that has learned the sound characteristics of a water-soaked state and the sound characteristics of a dirty state, and use the model to determine whether the terminal device is water-soaked or dirty.

[0032] Furthermore, sounds transmitted through the air also become characteristics of the right hand, left hand, etc. Therefore, the server device 100 may construct a model that learns characteristics of the right hand, left hand, etc. from sounds transmitted through the air, and use the model to determine whether the user U is touching the terminal device 10 with his / her right hand or left hand.

[0033] As described above, in this embodiment, a speaker SK capable of outputting sound containing a predetermined wavelength and a microphone MK capable of acquiring the sound output by the speaker SK are attached to the housing of the terminal device 10 of the target user U. The server device 100 then detects water damage to the housing according to the characteristics of the acquired sound acquired by the microphone MK. This allows the server device 100 to detect water damage to the surface of the terminal device 10 of the user U and determine whether a suspicious operation is caused by water damage, etc. Furthermore, the server device 100 can detect, for example, water droplets that the user U cannot see (visually recognize) with the naked eye or that the user U is unaware of.

[0034] In practice, in the above embodiment, the server device 100 may be replaced by another terminal device 10 or the terminal device 10 of the target user U. That is, the subject of the process is not limited to the server device 100, but the other terminal device 10 or the terminal device 10 of the target user U may use active acoustic sensing to detect water wetting of the surface of the terminal device 10 of the user U.

[0035] For example, the other terminal device 10 detects water damage to the surface of user U's terminal device 10 using active acoustic sensing, which outputs sound from a speaker SK attached to the housing of user U's terminal device 10 and acquires the sound with a microphone MK attached to the housing of user U's terminal device 10. This allows the other terminal device 10 to detect water damage to the surface of user U's terminal device 10, and to determine whether suspicious communication content from user U's terminal device 10 is due to water damage, etc.

[0036] Furthermore, the terminal device 10 of the user U detects water damage to the surface of the terminal device 10 using active acoustic sensing, which outputs sound from a speaker SK attached to the housing of the terminal device 10 and acquires the sound with a microphone MK attached to the housing of the terminal device 10. This enables the terminal device 10 of the user U to perform a self-diagnosis of water damage to the surface of the terminal device 10. Furthermore, the terminal device 10 of the user U can detect, for example, water droplets that cannot be seen (visually recognized) by the user U with the naked eye or water droplets that the user U is unaware of.

[0037] [2. Example of information processing system configuration] Next, a configuration of an information processing system 1 including a server device 100 according to an embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the configuration of the information processing system 1 according to an embodiment. As shown in Fig. 2, the information processing system 1 according to an embodiment includes a terminal device 10 and a server device 100. These various devices are connected to each other via a network N so as to be able to communicate with each other via a wired or wireless connection. The network N is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network) such as the Internet.

[0038] Furthermore, the number of devices included in the information processing system 1 shown in Fig. 2 is not limited to that shown in the figure. For example, in Fig. 2, for the sake of simplicity, only one terminal device 10 is shown, but this is merely an example and is not limiting, and two or more devices may be included.

[0039] The terminal device 10 is an information processing device used by a user U. For example, the terminal device 10 is a smart device such as a smartphone or a tablet terminal, a feature phone, a PC (Personal Computer), a PDA (Personal Digital Assistant), a game console or AV device with a communication function, a car navigation system, a wearable device such as a smart watch or a head-mounted display, smart glasses, etc.

[0040] In addition, the terminal device 10 can connect to the network N via a wireless communication network such as LTE (Long Term Evolution), 4G (4th Generation), or 5G (5th Generation: 5th generation mobile communication system), or via short-range wireless communication such as Bluetooth (registered trademark) or wireless LAN (Local Area Network), and communicate with the server device 100.

[0041] The server device 100 is, for example, a computer such as a PC or a blade server, or a mainframe or a workstation, etc. The server device 100 may be realized by cloud computing.

[0042] [3. Example of terminal device configuration] Next, the configuration of the terminal device 10 will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of the configuration of the terminal device 10. As shown in Fig. 3, the terminal device 10 includes a communication unit 11, a display unit 12, an input unit 13, a positioning unit 14, a sensor unit 20, a control unit 30 (controller), and a storage unit 40.

[0043] (Communications Department 11) The communication unit 11 is connected to a network N (see FIG. 2) by wire or wirelessly, and transmits and receives information to and from the server device 100 via the network N. For example, the communication unit 11 is realized by a NIC (Network Interface Card), an antenna, etc.

[0044] (Display section 12) Display unit 12 is a display device that displays various information such as position information. For example, display unit 12 is a liquid crystal display (LCD) or an organic electro-luminescent display (OLED). Display unit 12 is also a touch panel display, but is not limited to this.

[0045] (Input section 13) The input unit 13 is an input device that accepts various operations from the user U. For example, the input unit 13 has buttons for inputting characters, numbers, etc. The input unit 13 may be an input / output port (I / O port), a USB (Universal Serial Bus) port, etc. If the display unit 12 is a touch panel display, a part of the display unit 12 functions as the input unit 13. The input unit 13 may be a microphone that accepts voice input from the user U. The microphone may be wireless.

[0046] (Positioning unit 14) The positioning unit 14 receives signals (radio waves) transmitted from satellites of a GPS (Global Positioning System), and acquires position information (e.g., latitude and longitude) indicating the current position of the terminal device 10, which is the device itself, based on the received signals. That is, the positioning unit 14 positions the position of the terminal device 10. Note that GPS is merely an example of a GNSS (Global Navigation Satellite System).

[0047] The positioning unit 14 can also measure the position using various methods other than GPS. For example, the positioning unit 14 may measure the position by using various communication functions of the terminal device 10 as an auxiliary positioning means for position correction, etc., as described below.

[0048] (Wi-Fi positioning) For example, the positioning unit 14 uses a Wi-Fi (registered trademark) communication function of the terminal device 10 or a communication network provided by each communication company to measure the position of the terminal device 10. Specifically, the positioning unit 14 performs Wi-Fi communication or the like and measures the distance to a nearby base station or access point, thereby measuring the position of the terminal device 10.

[0049] (Beacon positioning) The positioning unit 14 may also measure the position by using a Bluetooth (registered trademark) function of the terminal device 10. For example, the positioning unit 14 measures the position of the terminal device 10 by connecting to a beacon transmitter connected by the Bluetooth (registered trademark) function.

[0050] (geomagnetic positioning) The positioning unit 14 also measures the position of the terminal device 10 based on a geomagnetic pattern of a structure that has been measured in advance and a geomagnetic sensor that the terminal device 10 has.

[0051] (RFID positioning) Furthermore, for example, if the terminal device 10 has a function of an RFID (Radio Frequency Identification) tag equivalent to a contactless IC card used at station ticket gates, in stores, etc., or has a function of reading an RFID tag, the location where the terminal device 10 was used is recorded together with information on the payment or the like made by the terminal device 10. The positioning unit 14 may obtain such information to determine the location of the terminal device 10. Alternatively, the location may be determined by an optical sensor, an infrared sensor, or the like provided in the terminal device 10.

[0052] The positioning unit 14 may measure the position of the terminal device 10 using one or a combination of the above-mentioned positioning means, as needed.

[0053] (Sensor unit 20) The sensor unit 20 includes various sensors mounted on or connected to the terminal device 10. The connection may be wired or wireless. For example, the sensors may be detection devices other than the terminal device 10, such as wearable devices or wireless devices. In the example shown in FIG. 3 , the sensor unit 20 includes an acceleration sensor 21, a gyro sensor 22, a barometric pressure sensor 23, a temperature sensor 24, a sound sensor 25, a light sensor 26, a magnetic sensor 27, and an image sensor (camera) 28.

[0054] The above-described sensors 21 to 28 are merely examples and are not intended to be limiting. That is, the sensor unit 20 may be configured to include some of the sensors 21 to 28, or may include other sensors such as a humidity sensor in addition to or instead of the sensors 21 to 28.

[0055] The acceleration sensor 21 is, for example, a three-axis acceleration sensor, and detects physical movements of the terminal device 10, such as the direction of movement, speed, and acceleration of the terminal device 10. The gyro sensor 22 detects physical movements of the terminal device 10, such as tilt in three axial directions, based on the angular velocity of the terminal device 10. The air pressure sensor 23 detects, for example, the air pressure around the terminal device 10.

[0056] Since the terminal device 10 includes the acceleration sensor 21, the gyro sensor 22, the atmospheric pressure sensor 23, etc., it is possible to measure the position of the terminal device 10 using a technique such as Pedestrian Dead-Reckoning (PDR) that uses these sensors 21 to 23. This makes it possible to obtain indoor position information that is difficult to obtain using a positioning system such as GPS.

[0057] For example, the number of steps, walking speed, and distance walked can be calculated using a pedometer that uses the acceleration sensor 21. In addition, the direction of travel, line of sight, and body tilt of the user U can be determined using the gyro sensor 22. In addition, the altitude and floor on which the terminal device 10 of the user U is located can be determined from the air pressure detected by the air pressure sensor 23.

[0058] The temperature sensor 24 detects, for example, the temperature around the terminal device 10. The sound sensor 25 detects, for example, the sound around the terminal device 10. The light sensor 26 detects the illuminance around the terminal device 10. The magnetic sensor 27 detects, for example, the geomagnetism around the terminal device 10. The image sensor 28 captures an image around the terminal device 10.

[0059] The above-mentioned air pressure sensor 23, temperature sensor 24, sound sensor 25, light sensor 26, and image sensor 28 can detect the air pressure, temperature, sound, and illuminance, respectively, and capture images of the surroundings, thereby detecting the environment and situation around the terminal device 10. Furthermore, the accuracy of the location information of the terminal device 10 can be improved based on the environment and situation around the terminal device 10.

[0060] (control unit 30) The control unit 30 includes, for example, a microcomputer having a CPU (Central Processing Unit), ROM (Read Only Memory), RAM, input / output ports, etc., and various other circuits. The control unit 30 may also be configured with hardware such as an integrated circuit, for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 30 includes a transmitting unit 31, a receiving unit 32, and a processing unit 33.

[0061] (Transmitter 31) The transmission unit 31 can transmit, for example, various information input by the user U using the input unit 13, various information detected by each sensor 21 to 28 mounted on or connected to the terminal device 10, and location information of the terminal device 10 measured by the positioning unit 14 to the server device 100 via the communication unit 11.

[0062] (Receiving unit 32) The receiving unit 32 can receive various types of information provided by the server device 100 and requests for various types of information from the server device 100 via the communication unit 11.

[0063] (Processing unit 33) The processing unit 33 controls the entire terminal device 10, including the display unit 12. For example, the processing unit 33 can output various information transmitted by the transmitting unit 31 and various information received from the server device 100 by the receiving unit 32 to the display unit 12 for display.

[0064] In this embodiment, the processing unit 33 outputs sound (output sound) from a speaker SK attached to the housing of the terminal device 10, and acquires sound (acquired sound) with a microphone MK attached to the housing of the terminal device 10. At this time, the processing unit 33 may output sound from the speaker SK in response to a command to output sound from the speaker SK received by the receiving unit 32 from the server device 100 or another terminal device 10. Furthermore, the processing unit 33 may transmit information related to the sound acquired by the microphone MK from the transmitting unit 31 to the server device 100 or another terminal device 10.

[0065] (Storage unit 40) The storage unit 40 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), an optical disk, etc. The storage unit 40 stores various programs, various data, etc.

[0066] [4. Server device configuration example] Next, the configuration of the server device 100 according to the embodiment will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of the configuration of the server device 100 according to the embodiment. As shown in Fig. 4, the server device 100 includes a communication unit 110, a storage unit 120, and a control unit 130.

[0067] (Communication unit 110) The communication unit 110 is realized by, for example, a network interface card (NIC), etc. The communication unit 110 is also connected to a network N (see FIG. 2) by wire or wirelessly.

[0068] (Storage unit 120) The storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as an HDD, an SSD, an optical disk, etc. As shown in FIG. 4 , the storage unit 120 has a user information database 121, a history information database 122, and a detection information database 123.

[0069] (User Information Database 121) The user information database 121 stores user information about the user U. For example, the user information database 121 stores various information such as the attributes of the user U. FIG. 5 is a diagram showing an example of the user information database 121. In the example shown in FIG. 5, the user information database 121 has items such as "User ID (Identifier)," "Age," "Gender," "Home," "Workplace," and "Interests."

[0070] The "user ID" indicates identification information for identifying the user U. The "user ID" may be the contact information of the user U (telephone number, email address, etc.), or may be identification information for identifying the terminal device 10 of the user U.

[0071] Furthermore, "age" indicates the age of user U identified by the user ID. Note that "age" may be information indicating the specific age of user U (e.g., 35 years old), or may be information indicating the generation of user U (e.g., 30s). Alternatively, "age" may be information indicating the date of birth of user U, or may be information indicating the generation of user U (e.g., born in the 1980s). Furthermore, "gender" indicates the gender of user U identified by the user ID.

[0072] Furthermore, "home" indicates the location information of the home of user U identified by the user ID. In the example shown in FIG. 5, "home" is illustrated as an abstract code such as "LC11," but it may also be latitude and longitude information, etc. Furthermore, for example, "home" may also be the name of an area or an address.

[0073] Furthermore, "workplace" indicates location information of the workplace (school in the case of a student) of user U identified by the user ID. In the example shown in FIG. 5, "workplace" is illustrated as an abstract code such as "LC12," but it may also be latitude and longitude information, etc. Furthermore, for example, "workplace" may also be the name of a region or an address.

[0074] Furthermore, "interests" indicate the interests of user U identified by the user ID. In other words, "interests" indicate subjects in which user U identified by the user ID is highly interested. For example, "interests" may be search queries (keywords) entered by user U into a search engine. In the example shown in FIG. 5, each user U is shown with one "interest," but there may be multiple "interests."

[0075] For example, in the example shown in FIG. 5, the age of user U identified by user ID "U1" is "20s" and the gender is "male." Furthermore, for example, the home address of user U identified by user ID "U1" is "LC11." Furthermore, for example, the workplace of user U identified by user ID "U1" is "LC12." Furthermore, for example, the user U identified by user ID "U1" is interested in "sports."

[0076] 5, abstract values ​​such as "U1", "LC11", and "LC12" are used for illustration, but "U1", "LC11", and "LC12" are assumed to store information such as specific character strings and numerical values. Below, abstract values ​​may also be illustrated in diagrams relating to other information.

[0077] The user information database 121 may store various types of information depending on the purpose, without being limited to the above. For example, the user information database 121 may store various types of information related to the terminal device 10 of the user U. The user information database 121 may also store information related to the user U's attributes, such as demographic attributes, psychographic attributes, geographic attributes, and behavioral attributes. For example, the user information database 121 may store information such as name, family structure, hometown (hometown), occupation, job title, income, qualifications, type of residence (detached house, apartment, etc.), whether or not the user has a car, commuting time, commuting route, commuter pass area (station, line, etc.), frequently used stations (other than the station nearest to home or workplace), extracurricular activities (location, time zone, etc.), hobbies, interests, lifestyle, etc.

[0078] (History Information Database 122) The history information database 122 stores various information related to history information (log data) that indicates the behavior of the user U. Fig. 6 is a diagram showing an example of the history information database 122. In the example shown in Fig. 6, the history information database 122 has items such as "user ID," "location history," "search history," "browsing history," "purchase history," and "posting history."

[0079] "User ID" indicates identification information for identifying user U. "Location history" indicates the location history, which is the history of user U's location and movements. "Search history" indicates the search history, which is the history of search queries entered by user U. "Browsing history" indicates the browsing history, which is the history of content viewed by user U. "Purchase history" indicates the purchase history, which is the history of purchases made by user U. "Posting history" indicates the posting history, which is the history of posts made by user U. "Posting history" may also include questions about user U's possessions.

[0080] For example, in the example shown in Figure 6, user U, identified by user ID "U1," moved as shown in "Location History #1," searched as shown in "Search History #1," viewed content as shown in "Viewing History #1," purchased specific products at specific stores as shown in "Purchase History #1," and posted as shown in "Posting History."

[0081] Here, in the example shown in Figure 6, abstract values ​​such as "U1", "Location History #1", "Search History #1", "Browsing History #1", "Purchase History #1", and "Post History #1" are used for the illustration, but "U1", "Location History #1", "Search History #1", "Browsing History #1", "Purchase History #1", and "Post History #1" are assumed to store specific information such as character strings and numbers.

[0082] The history information database 122 is not limited to the above and may store various types of information depending on the purpose. For example, the history information database 122 may store the user U's usage history of a predetermined service. The history information database 122 may also store the user U's store visit history or facility visit history. The history information database 122 may also store the user U's payment history (electronic payment) using the terminal device 10.

[0083] (Detection Information Database 123) The detection information database 123 stores various information related to the detection results of water damage, etc. Fig. 7 is a diagram showing an example of the detection information database 123. In the example shown in Fig. 7, the detection information database 123 has items such as "user ID," "terminal," "output sound," "acquired sound," "water damage," "dirt," and "location."

[0084] "User ID" indicates identification information for identifying user U. "Terminal" indicates the type (model, size, etc.) of user U's terminal device 10. Note that "terminal" may also include information regarding peripheral devices, parts, or accessories (covers, rings, etc.) of the terminal device 10.

[0085] Furthermore, "output sound" indicates the characteristics of the sound (output sound) output from a speaker SK attached to the housing of the terminal device 10. Furthermore, "acquired sound" indicates the characteristics of the sound (acquired sound) acquired by a microphone NK attached to the housing of the terminal device 10.

[0086] Also, "wet with water" indicates whether or not there is water damage such as water droplets adhering to the housing of the terminal device 10. Furthermore, "dirt" indicates whether or not there is dirt attached to the housing of the terminal device 10. Note that "dirt" may also be information indicating the type of dirt (dust, oil film, air bubbles, etc.).

[0087] Furthermore, the "location" indicates the location (place, part, portion) of the housing of the terminal device 10 where water damage or soiling has occurred. Note that the "location" may separately indicate the location where "water damage" has occurred and the location where "soiling" has occurred.

[0088] For example, in the example shown in Figure 7, user U, identified by user ID "U1," is using a terminal device 10 of the type indicated by "Terminal #1," and outputs a sound indicated by "Output Sound #1" from a speaker SK attached to the terminal device 10, and acquires a sound indicated by "Acquired Sound #1" with a microphone MK attached to the terminal device 10, and as a result of active acoustic sensing, it is detected that "wetness" and "dirt" have occurred on the "screen" of the terminal device 10.

[0089] Here, in the example shown in Figure 7, abstract values ​​such as "U1," "terminal #1," "output sound #1," and "acquired sound #1" are used for the illustration, but "U1," "terminal #1," "output sound #1," and "acquired sound #1" are assumed to store specific information such as character strings and numerical values.

[0090] The detection information database 123 is not limited to the above and may store various types of information depending on the purpose. For example, the detection information database 123 may store a model constructed by learning the characteristics of the output sound and the acquired sound (or the difference between them). The detection information database 123 may also store items to be adjusted based on the detection result of water damage to the housing of the terminal device 10. The detection information database 123 may also store information indicating whether the user U is touching the terminal device 10 with his / her right hand or his / her left hand.

[0091] (control unit 130) 4, the explanation will be continued. The control unit 130 is a controller, and is realized by, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or the like, executing various programs (corresponding to an example of an information processing program) stored in a storage device inside the server device 100 using a storage area such as a RAM as a working area. In the example shown in FIG. 4, the control unit 130 has an acquisition unit 131, a command unit 132, a detection unit 133, a learning unit 134, an adjustment unit 135, and a provision unit 136.

[0092] (Acquisition part 131) The acquisition unit 131 acquires a search query input by the user U. For example, when the user U inputs a search query into a search engine or the like to perform a keyword search, the acquisition unit 131 acquires the search query via the communication unit 110. That is, the acquisition unit 131 acquires, via the communication unit 110, the keywords input by the user U into the search box of a search engine, website, or app.

[0093] Furthermore, the acquisition unit 131 acquires user information about the user U via the communication unit 110. For example, the acquisition unit 131 acquires identification information (such as a user ID) indicating the user U, location information of the user U, attribute information of the user U, etc. from the terminal device 10 of the user U. Furthermore, the acquisition unit 131 may acquire the identification information indicating the user U, attribute information of the user U, etc. when registering the user U. Then, the acquisition unit 131 registers the user information in the user information database 121 of the storage unit 120.

[0094] Furthermore, the acquisition unit 131 acquires various types of history information (log data) indicating the behavior of the user U via the communication unit 110. For example, the acquisition unit 131 acquires various types of history information indicating the behavior of the user U from the terminal device 10 of the user U or from various servers based on the user ID or the like. Then, the acquisition unit 131 registers the various types of history information in the history information database 122 of the storage unit 120.

[0095] (Instruction part 132) The command unit 132 issues a command via the communication unit 110 to the terminal device 10 of each target user U to output sound from a speaker SK attached to the housing of the terminal device 10 .

[0096] (Detection unit 133) The detection unit 133 receives sound acquired by a microphone MK attached to the housing of the terminal device 10 via the communication unit 110, and detects the attachment of foreign matter (water damage, dirt, etc.) to the housing of the terminal device 10 using active acoustic sensing based on the sound acquired by the microphone MK attached to the housing of the terminal device 10.

[0097] The detection unit 133 detects whether a foreign object is attached to the housing of the terminal device 10 based on the characteristics of the sound when a foreign object is attached to the housing of the terminal device 10 and the characteristics of the sound when no foreign object is attached.

[0098] (Learning Section 134) The learning unit 134 constructs a model by learning sound characteristics when a foreign object is attached to the housing of the terminal device 10. At this time, the detection unit 133 detects the attachment of a foreign object to the housing of the terminal device 10 using the model.

[0099] For example, the learning unit 134 constructs a model by learning the characteristics of the acquired sound acquired by the microphone MK when a foreign object is attached to the housing of the terminal device 10, or by learning both the characteristics of the output sound output from the speaker SK and the characteristics of the acquired sound acquired by the microphone MK.

[0100] Furthermore, the learning unit 134 learns the difference between the characteristics of the output sound output from the speaker SK when a foreign object is attached to the housing of the terminal device 10 and the characteristics of the acquired sound acquired by the microphone MK, and constructs a model.

[0101] The learning unit 134 also learns the characteristics of the acquired sound when a foreign object is attached to a predetermined part of the housing of the terminal device 10, and constructs a model indicating which part is wet. At this time, the detection unit 133 uses the model to detect (or estimate) which part of the housing of the terminal device 10 is wet.

[0102] The learning unit 134 also learns and constructs a model for each type of terminal device 10. The learning unit 134 also learns and constructs a model for each peripheral device, part, or accessory of the terminal device 10.

[0103] Furthermore, the learning unit 134 learns sound characteristics when a foreign substance is attached to the housing of the terminal device 10 and sound characteristics when the housing of the terminal device 10 is dirty, and constructs a model indicating whether a foreign substance is attached or dirty. At this time, the detection unit 133 detects (or estimates) whether a foreign substance is attached or dirty using the model. That is, the detection unit 133 detects (or estimates) whether the housing of the terminal device 10 is in a state where a foreign substance is attached or dirty using the model.

[0104] (Adjustment section 135) The adjustment unit 135 performs various controls based on the detection result of foreign matter adhering to the housing of the terminal device 10 of each user U. For example, the adjustment unit 135 may change a threshold value for determining contact with a touch panel or the like on the housing of the terminal device 10 based on the detection result of foreign matter adhering to the housing of the terminal device 10. Furthermore, the adjustment unit 135 may determine whether an operation on the terminal device 10 is valid or invalid based on the detection result of foreign matter adhering to the housing of the terminal device 10.

[0105] In addition, the adjustment unit 135 may perform control such as "cancel the operation because a foreign object has been attached" based on the results that "there is a mysterious touch" and "foreign object attachment has been detected (foreign object detection)" on the terminal device 10 of each user U.

[0106] (Provider 136) The providing unit 136 provides information regarding the detection results of foreign matter adhesion to the housing of each user U's terminal device 10 via the communication unit 110 to the terminal device 10 being detected (or the other terminal device 10 communicating with that terminal device).

[0107] [5. Processing Procedure] Next, a processing procedure performed by the terminal device 10 and the server device 100 according to the embodiment will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the processing procedure according to the embodiment. Note that the processing procedure shown below is repeatedly executed by the control unit 30 of the terminal device 10 and the control unit 130 of the server device 100.

[0108] 8, the command unit 132 of the server device 100 issues a command to the terminal device 10 of each target user U via the communication unit 110 to output sound from a speaker SK attached to the housing of the terminal device 10 (step S101). At this time, the speaker SK attached to the housing of the terminal device 10 outputs sound.

[0109] Next, the detection unit 133 of the server device 100 receives the sound acquired by the microphone MK attached to the housing of the terminal device 10 via the communication unit 110 (step S102). At this time, the microphone MK attached to the housing of the terminal device 10 acquires the sound output from the speaker SK and transmits information about the acquired sound to the server device 100.

[0110] Next, the detection unit 133 of the server device 100 detects whether the housing of the terminal device 10 is wet with water using active acoustic sensing based on the sound picked up by the microphone MK attached to the housing of the terminal device 10 (step S103).

[0111] Next, the learning unit 134 of the server device 100 learns the characteristics of sound when the housing of the terminal device 10 is wet, and constructs a model (step S104). For example, the learning unit 134 constructs a model by learning the characteristics of the acquired sound acquired by the microphone MK when the housing of the terminal device 10 is wet, or by learning both the characteristics of the output sound output from the speaker SK and the characteristics of the acquired sound acquired by the microphone MK. Alternatively, the learning unit 134 constructs a model by learning the difference between the characteristics of the output sound output from the speaker SK and the characteristics of the acquired sound acquired by the microphone MK when the housing of the terminal device 10 is wet.

[0112] Next, the learning unit 134 of the server device 100 learns the characteristics of the acquired sound when a predetermined part of the housing of the terminal device 10 is wet, and constructs a model indicating which part is wet (step S105). At this time, the detection unit 133 detects (or estimates) which part of the housing of the terminal device 10 is wet using the model.

[0113] Next, the learning unit 134 of the server device 100 learns and constructs a model for each type of terminal device 10 and for each peripheral device / part / accessory of the terminal device 10 (step S106).

[0114] Next, the learning unit 134 of the server device 100 learns the sound characteristics of the case of the terminal device 10 when it is wet and the sound characteristics of the case of the terminal device 10 when it is dirty, and constructs a model indicating whether it is wet or dirty (step S107). At this time, the detection unit 133 detects (or estimates) whether it is wet or dirty using the model.

[0115] Next, the adjustment unit 135 of the server device 100 performs various controls based on the detection result of water wetting of the housing of the terminal device 10 of each user U (step S108).

[0116] Next, the providing unit 136 of the server device 100 provides information regarding the detection result of water damage to the housing of each user U's terminal device 10 via the communication unit 110 to the terminal device 10 to be detected (or the other terminal device 10 communicating with that terminal device) (step S109).

[0117] [6. Modifications] The terminal device 10 and the server device 100 described above may be implemented in various different forms other than the above embodiment. Therefore, modifications of the embodiment will be described below.

[0118] In the above embodiment, some or all of the processing executed by the server device 100 may actually be executed by the terminal device 10. For example, the processing may be completed in a stand-alone manner (by the terminal device 10 alone). In this case, the terminal device 10 is assumed to have the functions of the server device 100 in the above embodiment. Furthermore, in the above embodiment, the terminal device 10 is linked to the server device 100, and therefore, from the perspective of the user U, it appears that the processing of the server device 100 is also being executed by the terminal device 10. In other words, from another perspective, the terminal device 10 can also be said to be equipped with the server device 100.

[0119] Furthermore, in the above embodiment, a single terminal device 10 may have a plurality of speakers SK and a plurality of microphones MK attached to its housing. The number of speakers SK and the number of microphones MK do not have to be the same. For example, sound output from a single speaker SK attached to the housing of the terminal device 10 may be acquired by microphones NK attached to multiple locations on the housing of the terminal device 10. Conversely, sound output from speakers SK attached to multiple locations on the housing of the terminal device 10 may be acquired by a single microphone NK attached to the housing of the terminal device 10. Note that if the microphone MK cannot properly acquire sound when multiple speakers SK are simultaneously output, the timing at which the sounds are output from each speaker SK may be staggered. In this case, the server device 100 may estimate the location of water damage on the housing of the terminal device 10 based on which microphone MK acquired the sound output from which speaker SK.

[0120] Furthermore, in the above embodiment, a plurality of pairs (sets) of a speaker SK and a microphone MK may be installed on the housing of one terminal device 10. In this case, the number of speakers SK and microphones MK is the same. For example, a pair of a speaker SK and a microphone MK may be installed on each of the front side (screen side) and the back side of one terminal device 10. Note that if the microphone MK cannot properly pick up the sound when multiple speakers SK are output simultaneously, the timing at which the sound is output from each speaker SK may be staggered.

[0121] [7. Effects] As described above, the information processing device (terminal device 10 and server device 100) according to the present application includes a command unit 132 that issues a command to output sound from a speaker SK attached to the housing of the terminal device 10, and a detection unit 133 that detects the attachment of foreign matter to the housing of the terminal device 10 using active acoustic sensing based on sound acquired by a microphone MK attached to the housing of the terminal device 10.

[0122] The detection unit 133 detects whether a foreign object is attached to the housing of the terminal device 10 based on the characteristics of the sound when a foreign object is attached to the housing of the terminal device 10 and the characteristics of the sound when no foreign object is attached.

[0123] The information processing device according to the present application further includes a learning unit 134 that builds a model by learning sound characteristics when a foreign object is attached to the housing of the terminal device 10. The detection unit 133 detects the attachment of a foreign object to the housing of the terminal device 10 using the model.

[0124] The learning unit 134 constructs a model by learning the characteristics of the acquired sound acquired by the microphone MK when a foreign object is attached to the housing of the terminal device 10, or by learning both the characteristics of the output sound output from the speaker SK and the characteristics of the acquired sound acquired by the microphone MK.

[0125] The learning unit 134 learns the difference between the characteristics of the output sound output from the speaker SK and the characteristics of the acquired sound acquired by the microphone MK in a state where a foreign object is attached to the housing of the terminal device 10, and constructs a model.

[0126] The learning unit 134 learns the characteristics of the acquired sound when a foreign substance is attached to a predetermined part of the housing of the terminal device 10, and constructs a model indicating which part is wet.

[0127] The learning unit 134 learns for each type of terminal device 10 and constructs a model.

[0128] The learning unit 134 learns and constructs a model for each peripheral device, part, or accessory of the terminal device 10.

[0129] The learning unit 134 learns sound characteristics when the housing of the terminal device 10 is wet and sound characteristics when the housing of the terminal device 10 is dirty, and constructs a model indicating whether the housing is wet or dirty.

[0130] By performing any one or a combination of the above-described processes, the information processing device according to the present application can detect water wetting of the surface of the terminal device by the sound output from the speaker and picked up by the microphone.

[0131] [8. Hardware Configuration] The terminal device 10 and the server device 100 according to the above-described embodiments are realized by a computer 1000 having a configuration as shown in Fig. 9, for example. The following description will be given taking the server device 100 as an example. Fig. 9 is a diagram showing an example of a hardware configuration. The computer 1000 is connected to an output device 1010 and an input device 1020, and has a configuration in which a calculation device 1030, a primary storage device 1040, a secondary storage device 1050, an output I / F (Interface) 1060, an input I / F 1070, and a network I / F 1080 are connected via a bus 1090.

[0132] The arithmetic device 1030 operates based on programs stored in the primary storage device 1040 and the secondary storage device 1050, programs read from the input device 1020, and the like, and executes various processes. The arithmetic device 1030 is realized by, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or the like.

[0133] The primary storage device 1040 is a memory device such as a RAM (Random Access Memory) that temporarily stores data used by the arithmetic device 1030 for various calculations. The secondary storage device 1050 is a storage device in which data used by the arithmetic device 1030 for various calculations and various databases are registered, and is realized by a ROM (Read Only Memory), an HDD (Hard Disk Drive), an SSD (Solid State Drive), a flash memory, or the like. The secondary storage device 1050 may be an internal storage device or an external storage device. The secondary storage device 1050 may also be a removable storage medium such as a USB (Universal Serial Bus) memory or an SD (Secure Digital) memory card. The secondary storage device 1050 may also be cloud storage (online storage), a NAS (Network Attached Storage), a file server, or the like.

[0134] The output I / F 1060 is an interface for transmitting information to be output to an output device 1010 that outputs various types of information, such as a display, a projector, a printer, etc., and is realized by a connector conforming to a standard such as USB (Universal Serial Bus), DVI (Digital Visual Interface), or HDMI (High Definition Multimedia Interface), etc. The input I / F 1070 is an interface for receiving information from various input devices 1020, such as a mouse, a keyboard, a keypad, a button, a scanner, etc., and is realized by a USB, etc.

[0135] Furthermore, the output I / F 1060 and the input I / F 1070 may be wirelessly connected to the output device 1010 and the input device 1020, respectively. That is, the output device 1010 and the input device 1020 may be wireless devices.

[0136] The output device 1010 and the input device 1020 may be integrated into one device, such as a touch panel. In this case, the output I / F 1060 and the input I / F 1070 may also be integrated into one device as an input / output I / F.

[0137] The input device 1020 may be a device that reads information from, for example, an optical recording medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), or a PD (Phase Change Rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0138] The network I / F 1080 receives data from other devices via the network N and sends it to the arithmetic device 1030, and also transmits data generated by the arithmetic device 1030 to other devices via the network N.

[0139] The arithmetic unit 1030 controls the output device 1010 and the input device 1020 via the output I / F 1060 and the input I / F 1070. For example, the arithmetic unit 1030 loads a program from the input device 1020 or the secondary storage device 1050 onto the primary storage device 1040 and executes the loaded program.

[0140] For example, when the computer 1000 functions as the server device 100, the arithmetic unit 1030 of the computer 1000 executes a program loaded onto the primary storage device 1040 to realize the functions of the control unit 130. The arithmetic unit 1030 of the computer 1000 may also load a program acquired from another device via the network I / F 1080 onto the primary storage device 1040 and execute the loaded program. The arithmetic unit 1030 of the computer 1000 may also cooperate with the other device via the network I / F 1080 to call and use the functions and data of a program from another program of the other device.

[0141] [9. Other] Although the embodiments of the present application have been described above, the present invention is not limited to the contents of these embodiments. Furthermore, the above-described components include those that can be easily imagined by a person skilled in the art, those that are substantially the same, and those that are within the scope of so-called equivalents. Furthermore, the above-described components can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the above-described embodiments.

[0142] Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using a known method. In addition, the information including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown in the drawings.

[0143] Furthermore, the components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

[0144] For example, the above-mentioned server device 100 may be realized by multiple server computers, and depending on the function, the configuration can be flexibly changed, such as by calling an external platform using an API (Application Programming Interface) or network computing.

[0145] Furthermore, the above-described embodiments and modifications can be combined as appropriate within the scope of not causing any contradiction in the processing content.

[0146] Furthermore, the above-mentioned "section, module, unit" can be read as "means" or "circuit," etc. For example, an acquisition unit can be read as an acquisition means or an acquisition circuit. [Explanation of symbols]

[0147] 1. Information Processing Systems 10 Terminal Equipment SK Speaker MK Microphone 100 Server device 110 Communications Department 120 Storage section 121 User Information Database 122 Historical Information Database 123 Detection Information Database 130 Control Unit 131 Acquisition Department 132 Command section 133 Detection unit 134 Learning Department 135 Adjustment section 136 Provision Department

Claims

1. a command unit that issues a command to output sound from a speaker attached to a housing of the terminal device; a detection unit that detects the attachment of a foreign object to a housing of the terminal device using active acoustic sensing based on a sound acquired by a microphone attached to the housing of the terminal device; An information processing device comprising:

2. The detection unit detects whether a foreign object is attached to the housing of the terminal device based on sound characteristics when a foreign object is attached to the housing of the terminal device and sound characteristics when no foreign object is attached.

2. The information processing apparatus according to claim 1, wherein:

3. a learning unit that learns characteristics of sounds when a foreign object is attached to a housing of the terminal device and constructs a model; Furthermore, The detection unit detects adhesion of a foreign object to a housing of the terminal device using the model.

3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

4. The learning unit builds a model by learning features of the acquired sound acquired by the microphone in a state where a foreign object is attached to the housing of the terminal device, or by learning both features of the output sound output from the speaker and features of the acquired sound acquired by the microphone.

4. The information processing apparatus according to claim 3,

5. The learning unit learns a difference between a feature of the output sound output from the speaker in a state where a foreign object is attached to the housing of the terminal device and a feature of the acquired sound acquired by the microphone, and constructs a model.

4. The information processing apparatus according to claim 3,

6. The learning unit learns characteristics of the acquired sound when a foreign object is attached to a predetermined part of the housing of the terminal device, and constructs a model indicating which part is wet.

4. The information processing apparatus according to claim 3,

7. The learning unit learns and builds a model for each type of the terminal device.

4. The information processing apparatus according to claim 3,

8. The learning unit learns and builds a model for each peripheral device, part, or accessory of the terminal device.

4. The information processing apparatus according to claim 3,

9. The learning unit learns sound characteristics when the housing of the terminal device is wet and sound characteristics when the housing of the terminal device is dirty, and constructs a model indicating whether the housing is wet or dirty.

4. The information processing apparatus according to claim 3,

10. An information processing method executed by an information processing device, a command step of issuing a command to output sound from a speaker attached to a housing of the terminal device; a detection step of detecting whether a foreign object is attached to a housing of the terminal device using active acoustic sensing based on a sound acquired by a microphone attached to the housing of the terminal device; An information processing method comprising:

11. a command procedure for issuing a command to output sound from a speaker attached to a housing of the terminal device; a detection step of detecting whether a foreign object is attached to a housing of the terminal device using active acoustic sensing based on a sound acquired by a microphone attached to the housing of the terminal device; An information processing program that causes a computer to execute the above.

Images