Information Processing Systems

Abstract

To appropriately determine whether or not sound information is suspicious that the sound information is scam or annoyance when the sound information is transmitted to a destination of the specific person from the third person, and to prevent a transmission of suspicious information in accordance with scam or annoyance to the specific person when transferring the sound information to the specific person from the third person.SOLUTION: An information processing system 5 comprises: an information management unit 5B that transmits sound information with a destination transmitted from a general communication device to an information notification device of a registered user as a destination after temporarily storing the sound information; and an information suspiciousness level determination unit 5A that determines a suspiciousness level related to the sound information. The information management unit 5B generates suspiciousness information for sound information determined as suspicious by the information suspiciousness level determination unit 5A, and provides the information to at least one of the registered user and a person related to the registered user.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an information processing system that manages data communications between information alarms installed in the homes of registered users and general-purpose communication devices, and also suppresses the intrusion of suspicious information. [Background technology]

[0002] The system according to Patent Document 1, when the telephone used by the responder answers an incoming call from the telephone used by the caller via the public telephone network, enters into this call state, recognizes and analyzes the voice during the call, and if the conversation during the call contains a pre-registered dangerous keyword, authenticates the caller.If authentication is not possible, the responder is made to listen to a voice that warns them that the current call is dangerous and may involve fraud, etc.

[0003] The telephone according to Patent Document 2 answers an incoming call from a telephone line and transitions to a call, and at the same time inputs the voice during the call into a learning model, and if the voice includes voice related to fraud or nuisance, notifies the user that it is related to fraud or nuisance. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2011-135328 [Patent Document 2] Patent Publication No. 2021-35045 Summary of the Invention [Problem to be solved by the invention]

[0005] The fraud prevention technologies disclosed in Patent Documents 1 and 2 process the voice of an incoming telephone call to determine whether the voice contains voice related to fraud or nuisance, and if the voice contains voice related to fraud or nuisance, notify the user via telephone that the current call may be a fraud, etc. Such technologies require the user to intercept the telephone call and determine whether the content of the call contains voice related to fraud or nuisance, which imposes a heavy computational burden on the determination process and requires significant costs for computing hardware and software. Furthermore, for information and communication systems that do not provide voice information from a third party to a specific person through a direct telephone call, but instead temporarily store voice information from a third party in a management server or the like and then send the voice information to a terminal owned by the specific person, it is necessary to use a technology different from the fraud prevention technologies disclosed in Patent Documents 1 and 2, taking into account the computational load, etc.

[0006] An object of the present invention is to appropriately determine whether or not voice information sent from a third party to a specific person is suspicious voice information related to fraud or nuisance. [Means for solving the problem]

[0007] An information processing system according to the present invention, which manages data communication between an information alarm and a general-purpose communication device installed in the home of a registered user, comprises an information management section which temporarily stores voice information with a destination address sent from the general-purpose communication device and then transmits the voice information to the information alarm of the destination address, and an information suspiciousness determination unit which determines the degree of suspiciousness of the voice information, wherein the information management section generates suspiciousness information regarding the voice information determined to be suspicious by the information suspiciousness determination unit, and notifies the registered user and / or a person related to the registered user. The information suspiciousness determination unit includes a content determination section that determines the suspiciousness of the content of the voice information, and the content determination section includes an image processing module that processes image information that is attached to the voice information and captures at least the mouth of the person inputting the voice, and a voice recognition module that receives the voice information as input and outputs voice recognition information, and the content determination section determines the suspiciousness of the content of the voice information based on the degree of coincidence obtained by comparing the image features of the mouth output from the image processing module with the voice recognition information in a time series manner. .

[0008] According to this configuration, when voice information addressed to an alarm installed in a registered user's home is sent to the information management unit, the voice information is stored, and then the suspiciousness level of the stored voice information is determined. If the voice information is determined to be suspicious, suspicious information about the voice information is generated, and this suspicious information is notified to at least one of the registered user and a person related to the registered user. Since the voice information is stored or read out after it has been stored, and the suspiciousness level of the voice information is determined, there is ample time for the determination process, allowing for appropriate determination. The transfer of suspicious voice information, which is voice information determined to be suspicious, to the alarm may be prohibited, or only the suspicious information may be sent, leaving the handling of the suspicious voice information up to the user. It is also possible to evaluate the content of the transmitted voice information itself using various voice analysis methods, and determine the degree of suspiciousness of the voice information from the evaluation results. Furthermore, by combining voice information with a photograph (image information) of the sender's face, it becomes possible to highly reliably judge the suspiciousness of voice information. For example, one highly accurate suspiciousness judgment rule that combines voice information with a photograph (image information) compares the voice information with the movement of the mouth over time, and voice information in which the pronunciation indicated in the voice information and the movement of the mouth at that time do not match is deemed to be fraudulently manipulated voice information (suspicious voice information).

[0009] An information processing system according to the present invention, which manages data communication between an information alarm placed in the home of a registered user and a general-purpose communication device, comprises an information management unit that temporarily stores voice information with an address attached sent from the general-purpose communication device and then transmits the voice information to the information alarm of the addressee, and an information suspiciousness judgment unit that judges the suspiciousness of the voice information, wherein the information management unit generates suspicious information regarding the voice information judged to be suspicious by the information suspiciousness judgment unit and notifies at least one of the registered user and a person related to the registered user, and the information suspiciousness judgment unit comprises a content judgment unit that judges the suspiciousness of the content of the voice information, and the content judgment unit has an image processing module that processes image information accompanying the voice information, which captures at least the mouth of the person entering the voice at the time of voice input, and a voice recognition module that inputs the voice information and outputs voice recognition information, and the content judgment unit judges the suspiciousness of the content of the voice information based on the difference between the pronunciation time calculated from the voice recognition information and the pronunciation time calculated from the image features of the mouth.

[0010] According to this configuration, when voice information addressed to an alarm installed in a registered user's home is sent to the information management unit, the voice information is stored, and then the suspiciousness level of the stored voice information is determined. If the voice information is determined to be suspicious, suspicious information about the voice information is generated, and this suspicious information is notified to at least one of the registered user and a person related to the registered user. Since the voice information is stored or read out after it has been stored, and the suspiciousness level of the voice information is determined, there is ample time for the determination process, allowing for appropriate determination. The transfer of suspicious voice information, which is voice information determined to be suspicious, to the alarm may be prohibited, or only the suspicious information may be sent, leaving the handling of the suspicious voice information up to the user. It is also possible to evaluate the content of the transmitted voice information itself using various voice analysis methods, and determine the degree of suspiciousness of the voice information from the evaluation results. Furthermore, in a preferred example of technology for comparative evaluation of voice information and image features of the sender's mouth, the content of the voice information is compared with mouth movements (calculated from lip reading technology or image features), and if the time indicated by the voice information is delayed (timing mismatch) by more than a predetermined time (e.g., about 70 milliseconds) from the time indicated by the mouth movements, the voice information can be deemed to have been altered using some kind of voice converter in order to impersonate a person related to the registered user (suspicious voice). Furthermore, if the time indicated by the mouth movements does not match the time indicated by the voice information, the voice can be deemed to be voice using a recorded voice or an artificial voice (suspicious voice). Furthermore, even if the voice information is music information such as singing, the voice can be deemed to be suspicious due to the mismatch or unnaturalness between the voice information and the mouth movements.

[0011] An information processing system according to the present invention, which manages data communication between an information alarm installed in the home of a registered user and a general-purpose communication device, comprises an information management section which temporarily stores voice information with a destination address sent from the general-purpose communication device and then transmits the voice information to the information alarm of the destination address, and an information suspiciousness determination unit which determines the suspiciousness level of the voice information, wherein the information management section generates suspiciousness information regarding the voice information determined to be suspicious by the information suspiciousness level determination unit, and notifies the registered user and / or a person related to the registered user, The information suspiciousness determination unit includes a content determination unit that determines the suspiciousness of the content of the audio information. The audio information includes music information, and the content determination unit has a music information determination module that generates music feature data of the music information, and the content determination unit determines the suspiciousness of the music information based on the music feature data. .

[0012] According to this configuration, when voice information addressed to an alarm installed in a registered user's home is sent to the information management unit, the voice information is stored, and then the suspiciousness level of the stored voice information is determined. If the voice information is determined to be suspicious, suspicious information about the voice information is generated, and this suspicious information is notified to at least one of the registered user and a person related to the registered user. Since the voice information is stored or read out after it has been stored, and the suspiciousness level of the voice information is determined, there is ample time for the determination process, allowing for appropriate determination. The transfer of suspicious voice information, which is voice information determined to be suspicious, to the alarm may be prohibited, or only the suspicious information may be sent, leaving the handling of the suspicious voice information up to the user. It is also possible to evaluate the content of the transmitted voice information itself using various voice analysis methods, and determine the degree of suspiciousness of the voice information from the evaluation results. Furthermore, information alarms that can emit audio information transmitted over a network may be the destination of unauthorized music (including recitations). To prevent information processing systems from becoming relay stations between the source and destination of unauthorized music, it is necessary to treat such unauthorized music as suspicious audio information and block its transmission. Measures to achieve this include, if the audio information is music information, determining whether the source is legitimate, determining whether the music information contains evidence of copyright infringement, and blocking music information other than a specific music format.

[0013] In the process of determining the suspiciousness of voice information sent from a third party, the simplest and most important rule for determination is to confirm the identity of the sender. If the sender is limited to a registered user or a person related to the registered user, the identity can be confirmed based on attribute information such as the name, address, telephone number, password, etc. of the registered user or a person related to the registered user. Therefore, in a preferred embodiment of the present invention, the information suspiciousness determination unit includes a sender determination unit that determines the suspiciousness of the sender of the voice information, and the sender determination unit determines that the sender is a suspicious sender if the sender is not the registered user or a person related to the registered user. Make a judgment.

[0014] If a facial photograph (image information) is required to be sent at the same time when transmitting voice information, the facial photograph can be used to determine the suspiciousness of the sender of the voice information. If facial photographs of registered users and their associates (relatives and friends) are registered and a facial photograph is required to be sent at the same time when transmitting voice information, the identity of the sender of the voice information can be confirmed through the facial photograph (application of face recognition technology). For this reason, in a preferred embodiment of the present invention, the sender determination unit determines the suspiciousness of the sender of the voice information based on a comparison between image information of the face of the person inputting the voice at the time of voice input, which is attached to the voice information, and registered image information. Determine the level of suspiciousness.

[0015] In the process of determining the suspiciousness of voice information sent from a third party, it is also possible to convert the voice information into text and determine the suspiciousness of the voice information from the character string. Detecting words from the character string makes it easier to evaluate the content (meaning) of the voice information. For example, in the case of "it's me" scams or bank transfer scams, if typical words used in these cases, such as "transfer" or "account number," are detected, it can be determined that the voice information is highly likely to be suspicious. For this reason, in a preferred embodiment of the present invention, the content determination unit includes a voice recognition module that performs voice recognition processing on the voice information, the voice recognition module has a function of converting the voice information into text, and the content determination unit determines the content of the voice information based on the textual voice information. Determine the level of suspiciousness.

[0016] Other features, operations, and advantages of the present invention will become apparent from the following description of the invention using the accompanying drawings. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a schematic diagram illustrating an outline of an information distribution system that eliminates the distribution of unauthorized audio information. [Figure 2] 10 is a schematic diagram showing the flow of information in an information processing management computer that manages data communication between an information alarm and a general-purpose communication device. FIG. [Figure 3] FIG. 2 is a functional block diagram showing functions of the information processing system. [Figure 4] FIG. 10 is a schematic diagram illustrating a sender suspiciousness determination performed by a sender determination unit. [Figure 5] 10A and 10B are schematic diagrams illustrating a suspiciousness determination of audio information using image information. [Figure 6] 10A and 10B are schematic diagrams illustrating a suspiciousness determination of audio information using image information. [Figure 7] 10A and 10B are schematic diagrams illustrating a suspiciousness determination of audio information using only audio information. [Figure 8] 10A and 10B are schematic diagrams illustrating a suspiciousness determination of audio information using only audio information. DETAILED DESCRIPTION OF THE INVENTION

[0018] 1 shows a schematic diagram of an information distribution system related to the present invention. This information distribution system includes an information notification device 1, a general-purpose communication device 2, an information management computer 4, and an external information server 100.

[0019] The alarm 1 is a gas alarm with communication capabilities. It not only detects gas leaks but also provides voice alerts for useful information, weather information, disaster prevention information, and more. The alarm 1 is installed in registered homes and connected to the Internet or public data lines via a Wi-Fi network. The general-purpose communication device 2 is a general term for mobile phones and computers with communication capabilities. It can transmit input voice data as voice information over the Internet or public data lines. It can also simultaneously capture the face of the voice inputter and transmit the image data over the Internet or public data lines, as well as send and receive emails. The information management computer 4 is a computer system capable of cloud services and server-client services. It has the functions of relaying information exchange between the alarm 1 and the general-purpose communication device 2 and communicating data individually with each of the alarm 1 and the general-purpose communication device 2. The alarm 1 is assigned identification information (such as a destination or address) that identifies registered users, and the information management computer 4 manages this identification information. Furthermore, the information management computer 4 also has the function of importing information from an external information server 100 such as an administrative information server or a weather information server, performing the necessary conversion processing, and then sending the information to the information alarm 1 in the form of audio information, or in some cases, text information or image information.

[0020] Figure 2 is a functional block diagram showing the functions particularly related to the present invention possessed by the information alarm 1, general-purpose communication device 2, and information management computer 4, and the information flowing between them, and Figure 3 is a functional block diagram showing the core functions of the present invention possessed by the information management computer 4.

[0021] The alarm 1 in this embodiment has a gas leak detection function and is equipped with a speaker 10 that emits audio information and a lamp (not shown). When a gas leak is detected by the gas detection function, the speaker 10 notifies the user of the gas leak using a synthesized voice. At the same time, this gas leak information (self-detected information) is sent to the information management computer 4. In this embodiment, the information management computer 4 is operated by a gas service center, and when the information management computer 4 receives gas leak information, it takes action in response to the gas leak information.

[0022] In this embodiment, the general-purpose communication device 2 is a mobile phone equipped with a microphone as an audio input unit 20 and a camera as an image capture unit 21. The general-purpose communication device 2 can send audio information, image information, and email addressed to a specific information alarm 1 to the information management computer 4. The information management computer 4 performs a suspiciousness check on the received audio information addressed to a specific information alarm 1, as will be described in detail later, and forwards audio information that is not deemed suspicious to the addressed information alarm 1 at any time. Furthermore, the general-purpose communication device 2 receives email information from the information management computer 4, such as notification emails containing information about the transmission results of the sent audio information. Furthermore, if the general-purpose communication device 2 is registered as the property of a related party (such as a family member or acquaintance) of the registered user of the information alarm 1, the information management computer 4 can also receive other information that it has learned through the information alarm 1 as email information.

[0023] As shown in Fig. 3, the information management computer 4 comprises an input / output unit 40, a user management unit 41, and an information processing system 5. The input / output unit 40 inputs and outputs information via a data communication network such as the Internet. The user management unit 41 manages registered users of the information alarm 1 and related parties of the registered users. This user management uses information such as name, address, telephone number, email address, facial photograph, and password.

[0024] The information processing system 5 has the function of relaying information exchange between the information alarm 1 and the general-purpose communication device 2, the function of communicating individual data with each of the information alarm 1 and the general-purpose communication device 2, the function of transmitting external notification information to the information alarm 1, and the function of preventing suspicious information from entering into this information exchange. To this end, the information processing system 5 comprises an information suspiciousness level determination unit 5A, an information management section 5B, and an information storage section 5C.

[0025] The information suspiciousness determination unit 5A determines the suspiciousness level of voice information sent from the general-purpose communication device 2 and addressed to the registered user's information alarm 1. The information management unit 5B temporarily stores the addressed voice information sent from the general-purpose communication device 2 in the information storage unit 5C, and then transmits the voice information to the addressed information alarm 1. For example, the voice information may be, "Happy birthday," or "Time for medicine." Before the voice information is transmitted to the addressed information alarm 1 by the information management unit 5B, the information suspiciousness determination unit 5A determines the suspiciousness level of the voice information. If the voice information is determined to be suspicious by the information suspiciousness determination unit 5A, transmission to the destination is generally stopped, but if it is transmitted, a warning message is attached to the information. In either case, the information management unit 5B generates suspicious information regarding the voice information determined to be suspicious and notifies the registered user, a person related to the registered user, or both of the registered user and the registered user of the suspicious information. For example, the warning message may be, "A voice that may be a special fraud has been recorded!" or "The user may not be registered."

[0026] The information management unit 5B includes a voice information management unit 53, a suspicious information management unit 54, and a mail management unit 55. The voice information management unit 53 stores voice information sent from the general-purpose communication device 2 in the information storage unit 5C so that it can be managed by sender, destination, and date and time, and manages its history. The voice information management unit 53 also has a function to forward received voice information to its destination. Furthermore, the voice information management unit 53 also has a function to forward external alarm information obtained from the external information server 100 to the alarm 1.

[0027] The suspicious information management unit 54 manages the suspiciousness determination results output from the information suspiciousness level determination unit 5A. Furthermore, the suspicious information management unit 54 stores the voice information determined to be suspicious in the information storage unit 5C so that it can be managed by sender, destination, and date and time, and manages its history. The email management unit 55 creates and sends information emails to registered users and their associates reporting the transmission results of the sent voice information and the occurrence of suspicious voice information. Furthermore, the email management unit 55 sends self-detection information detected by the information alarm 1 as email information to registered users and their associates.

[0028] The information suspiciousness determination unit 5A includes a sender determination unit 51 and a content determination unit 52. The sender determination unit 51 determines whether the sender of the received voice information is a legitimate sender, that is, whether the sender is a suspicious sender who is not a registered user or a related person of the registered user. The sender determination unit 51 performs a suspicious sender determination by verifying the sender's identity. If the sender of the voice information is a registered user, the sender determination unit 51 verifies the sender's identity based on the name, address, telephone number, password, facial photograph, fingerprints, etc. managed by the user management unit 41. For example, if the attachment of a facial photograph or fingerprint photograph is set as a condition for sending addressed voice information, the sender determination unit 51 verifies the sender's identity by comparing the attached image information such as the facial photograph and fingerprints with the image information of the facial photograph and fingerprint photograph managed by the user management unit 41 (image authentication processing). Specifically, the sender's identity may be confirmed by determining whether the sender is a registered user or a related person based on image information at the time of voice input (an image of the voice inputter or sender at the time of voice input).

[0029] As shown in Fig. 4, when audio information (suspicious audio) with image information (suspicious image) attached is sent from a suspicious terminal 2Z, the sender determination unit 51 checks the authenticity of the suspicious image in cooperation with the user management unit 41. If the sender is determined to be invalid, the audio information with the suspicious image attached is deemed to be suspicious audio, is not registered, and a warning is issued to the corresponding user, etc. as suspicious information. Note that in Figs. 4 to 7, the general-purpose communication device 2 is referred to as the suspicious terminal 2Z, but when transmitting audio information, it is not determined whether the general-purpose communication device 2 is the suspicious terminal 2Z. However, since the suspicious audio is being handled here, for the sake of explanation, the general-purpose communication device 2 is referred to as the suspicious terminal 2Z.

[0030] Furthermore, the information suspiciousness determination unit 5A includes a content determination unit 52 that determines the suspiciousness of the content of the voice information. The content determination unit 52 does not determine the legitimacy of the sender, but determines the suspiciousness of the content of the voice information itself. The content determination unit 52 in this embodiment includes an image processing module 521 and a voice recognition module 522. Therefore, image information attached to addressed voice information is also used to determine the suspiciousness of the content of the voice information.

[0031] The content determination unit 52 receives as input image information (an image of the voice inputter or sender at the time of voice input) attached to the addressed voice information output from the image processing module 521, and determines the degree of suspiciousness of the content of the voice information based on the image features output from the image processing module 521, and on the voice information input from the voice recognition module 522.

[0032] An example of how the content determining unit 52 determines the suspiciousness of the content of audio information using image features and audio recognition information will be described below.

[0033] (Example 1) The suspiciousness of the content of the voice information is determined based on the degree of coincidence obtained by comparing, over time, the mouth movements as an image feature output from the image processing module 521 with the recognized voice as voice recognition information output from the voice recognition module 522. As shown in FIG. 5, the degree of suspiciousness increases as the detected difference increases in a comparison over time between the voice data (words) output from the voice recognition module 522 and the mouth movements (electronic lip reading technology) output from the image processing module 521. For example, the degree of suspiciousness increases when there is no voice information at the time when there is mouth movement, or when the voice indicated by the mouth movement and the voice information express different words. If the suspiciousness exceeds a threshold, the voice information is determined to be suspicious voice information, and the information is not registered, and a warning (notification of suspicious information) is issued to the corresponding user, etc.

[0034] (Example 2) In this example, the content determination unit 52 determines the suspiciousness of the content of the audio information based on the difference between the pronunciation time calculated from the audio recognition information and the pronunciation time calculated from the image features. Specifically, based on a comparison between the elapsed time of the lip movements as an image feature output from the image processing module 521 and the elapsed time of the audio as audio recognition information output from the audio recognition module 522, it is determined whether the audio information is a "spoofed audio" (suspicious audio) generated using a voice changer or the like. As shown in FIG. 6 , when comparing the time interval (audio interval) between the start and end of the audio output from the audio recognition module 522 and the time interval (image interval) between the start and end of the lip movements output from the image processing module 521 for the same word, the suspiciousness level increases as the detected difference increases. For example, if the audio interval is approximately 70 milliseconds longer than the image interval, it is deemed that audio processing such as a voice changer has been involved, and the audio information is determined to be suspicious audio information. The audio information is not registered, and a warning (notification of suspicious information) is issued to the corresponding user or the like.

[0035] Next, an example will be described below in which the content determination unit 52 determines the suspiciousness of the content of received voice information based on the textual voice information (character string) output as voice recognition information from the voice recognition module 522, which has the function of converting voice information into text (voice recognition processing).

[0036] (Example 3) In this example, as shown in Fig. 7, the voice information (character string) converted to text by the voice recognition module 522 is checked to see if it contains suspicious words such as "It's me" ("It's me") or "Transfer." For this reason, an electronic dictionary of suspicious words is prepared in advance. If suspicious words are present, the voice information is determined to be suspicious voice information, and is not registered. In addition, a warning (notification of suspicious information) is issued to the corresponding user, etc.

[0037] (Example 4) In this example, as shown in Fig. 8, the voice information (character string) converted into text by the voice recognition module 522 is converted into sentence data and is run through a sentence interpretation program attached to the voice recognition module 522 to determine whether the sentence is suspicious. If the sentence is deemed to be suspicious, the voice information is determined to be suspicious voice information, and is not registered, and a warning (notification of suspicious information) is issued to the corresponding user, etc.

[0038] In this embodiment, the content determination unit 52 further includes a music information determination module 523. In the present invention, audio information also includes music information. This music information is a music source that can be played through the speaker 10 of the information alarm 1. The music information determination module 523 is used to determine whether this music information is fraudulent. Because the information alarm 1 can play music or a narration of music information received as audio information through the speaker 10, there is a possibility that fraudulent music information may be transmitted using this system. To avoid this problem, the music information determination module 523 derives music feature data from the music information and determines whether the music information contains evidence of copyright infringement or whether it contains music data in a specific music format that is not permitted to be transmitted. The content determination unit 52 determines the suspiciousness of the audio information based on the determination result of the music information determination module 523. If the audio information (music information) is determined to be suspicious, the transfer of the audio information is prohibited. In addition, even for music information, the sender determination unit 51 determines whether the sender is a legitimate sender, and if the sender is not a legitimate sender, the transfer of the music information is prohibited. Furthermore, suspicious information indicating the transfer prohibition is notified.

[0039] [Another embodiment] (1) In the above-described embodiment, the information alarm 1 is a gas leak alarm with a communication function that has a gas leak detection function and the speaker 10. However, it may also be a fire alarm, or a multi-function alarm that has additional detection functions other than gas leaks, such as fire detection. Furthermore, it may be a device that is equipped with a motion sensor, a camera, etc., and can be used to monitor elderly people, young children, etc. Furthermore, the information alarm 1 may also be equipped with a display in addition to the speaker 10, and configured to be able to display emails and other messages.

[0040] (2) The functional units shown in the functional blocks of Figure 3 can be divided into multiple functional units, multiple functional units can be integrated into a single functional unit, or at least some of the functional units can be assigned to the alarm 1 or distributed to other computers or processing units. In other words, the functions of the information processing system 5 of the present invention can be distributed to the alarm 1 and the information management computer 4 as a distributed processing system.

[0041] The configurations disclosed in the above embodiments (including other embodiments, the same applies below) can be applied in combination with configurations disclosed in other embodiments, as long as no contradiction arises. Furthermore, the embodiments disclosed in this specification are examples, and the embodiments of the present invention are not limited to these, and can be modified as appropriate within the scope that does not deviate from the purpose of the present invention. [Industrial Applicability]

[0042] The present invention can be applied to an information processing system that, when voice information is sent from a third party to a specific person, appropriately determines whether the voice information is suspicious voice information related to fraud or nuisance. [Explanation of symbols]

[0043] 1: Information alarm 2: General-purpose communication device 2Z: Suspicious terminal 4: Information management computer 5: Information Processing Systems 5A: Information Suspiciousness Judgment Unit 5B: Information Management Department 5C: Information storage section 10: Speaker 20: Audio input section 21: Photography Department 40: Input / output section 41: User Management Section 51: Sender identification unit 52:Content judgment section 53: Audio information management section 54: Suspicious Information Management Department 55: Email Management Department 100: External information server 521: Image processing module 522: Speech recognition module 523: Music information determination module

Claims

1. An information processing system for managing data communication between an information alarm device installed in a registered user's home and a general-purpose communication device, an information management unit that temporarily stores voice information with a destination address sent from the general-purpose communication device and then transmits the voice information to the information annunciator of the destination address; an information suspiciousness determination unit for determining the suspiciousness of the audio information, the information management unit generates suspicious information regarding the voice information determined to be suspicious by the information suspiciousness degree determination unit, and notifies the information management unit of the suspicious information to at least one of the registered user and a person related to the registered user; the information suspiciousness determination unit includes a content determination unit that determines the suspiciousness of the content of the audio information, The content determination unit has an image processing module that processes image information that is attached to the voice information and captures at least the mouth of the person inputting the voice, and a voice recognition module that takes the voice information as input and outputs voice recognition information, and the content determination unit determines the suspiciousness of the content of the voice information based on the degree of match obtained by comparing the image features of the mouth output from the image processing module with the voice recognition information in a chronological order.

2. An information processing system for managing data communication between an information alarm installed in a registered user's home and a general-purpose communication device, comprising: an information management unit that temporarily stores voice information with a destination address sent from the general-purpose communication device and then transmits the voice information to the information annunciator of the destination address; an information suspiciousness determination unit for determining the suspiciousness of the audio information, the information management unit generates suspicious information regarding the voice information determined to be suspicious by the information suspiciousness degree determination unit, and notifies the information management unit of the suspicious information to at least one of the registered user and a person related to the registered user; the information suspiciousness determination unit includes a content determination unit that determines the suspiciousness of the content of the audio information, The content determination unit has an image processing module that processes image information that is attached to the voice information and captures at least the mouth of the person inputting the voice, and a voice recognition module that takes the voice information as input and outputs voice recognition information, and the content determination unit determines the suspiciousness of the content of the voice information based on the difference between the pronunciation time calculated from the voice recognition information and the pronunciation time calculated from the image features of the mouth.

3. An information processing system for managing data communication between an information alarm installed in a registered user's home and a general-purpose communication device, an information management unit that temporarily stores voice information with a destination address sent from the general-purpose communication device and then transmits the voice information to the information annunciator of the destination address; an information suspiciousness determination unit for determining the suspiciousness of the audio information, the information management unit generates suspicious information regarding the voice information determined to be suspicious by the information suspiciousness degree determination unit, and notifies the information management unit of the suspicious information to at least one of the registered user and a person related to the registered user; the information suspiciousness determination unit includes a content determination unit that determines the suspiciousness of the content of the audio information, An information processing system in which the audio information includes music information, the content judgment unit has a music information judgment module that generates music feature data for the music information, and the content judgment unit judges the suspiciousness of the music information based on the music feature data.

4. The information suspiciousness determination unit is provided with a sender determination unit that determines the suspiciousness of the sender of the audio information, and the sender determination unit determines the sender to be a suspicious sender if the sender is not the registered user or a related person of the registered user.

5. The information processing system described in claim 4, wherein the sender determination unit determines the degree of suspiciousness of the sender of the voice information based on a comparison between image information of the face of the person entering the voice information at the time of voice input that accompanies the voice information and registered image information.

6. 6. An information processing system as described in any one of claims 1 to 5, wherein the content determination unit includes a voice recognition module that performs voice recognition processing on the voice information, the voice recognition module has a function of converting the voice information into text, and the content determination unit determines the suspiciousness of the content of the voice information based on the converted text.

Images