Intercom system and fraud notification method

Abstract

The system detects the possibility of fraud and other crimes occurring and promptly notifies third parties. [Solution] The intercom system comprises a doorbell-type device installed at the front door and having a microphone to pick up the voice of visitors, a database in which keywords related to crimes such as fraud are registered, a device that performs voice recognition processing to recognize the voice of visitors, and a control device that, if the result of the voice recognition processing contains keywords, notifies a notification device of warning information regarding the visitor. After the call between the visitor and the resident inside the house ends, the control device continues keyword detection processing, causing the device to perform voice recognition processing on the voice of the visitor picked up by the microphone and determining whether or not the result of the voice recognition processing contains keywords.

Description

Technical Field

[0001] The present disclosure relates to an intercom system and a fraud notification method.

Background Art

[0002] Patent Document 1 discloses a fraud call detection and reporting system that detects, when responding to an incoming call, that the call is a fraud call from the content of the call and alerts the responder. This fraud call detection and reporting system recognizes and analyzes the voice during the call, detects whether a pre-registered dangerous keyword is included in the conversation during the call, and authenticates the caller by the voiceprint of the caller and the PIN number entered by the caller when it is included. Further, this fraud call detection and reporting system makes the responder listen to a voice alerting that the current call is a dangerous one with a risk of fraud or the like when authentication fails.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The usage scenario assumed in Patent Document 1 corresponds to a conversation during a call started by the responder receiving a call from the caller, and the voice during the conversation listened to by the responder during the response (that is, the voice of the caller) is the target of recognition and analysis. Therefore, for example, a conversation during a call via an intercom system between a person visiting a private house or an apartment building and a resident is considered not to be assumed.

[0005] Therefore, Patent Document 1 had the problem that, after the conversation between the caller and the respondent ended and the call was disconnected, the caller's voice itself could not be subject to recognition analysis by the fraud call detection and reporting system. Even after a person committing fraud (a so-called fraudster) has finished talking with a person who could be the target of fraud (a so-called potential victim), the voice emitted by that fraudster may contain voices intended to commit fraud against the potential victim. For this reason, there was room for improvement over the conventional technology from the perspective of preventing crimes such as fraud.

[0006] This disclosure was devised in light of the aforementioned circumstances and aims to detect the possibility of fraud and other crimes occurring and to notify third parties at an early stage. [Means for solving the problem]

[0007] This disclosure provides an intercom system comprising: a doorbell-type device installed at the entrance and having a microphone for picking up the voice of a visitor; a database in which keywords related to crimes such as fraud are registered; a device that performs voice recognition processing to recognize the voice of the visitor; and a control device that, if the result of the voice recognition processing includes the keywords, notifies a notification device of warning information regarding the visitor, wherein the control device, after the call between the visitor and the resident inside the house via the doorbell-type device has ended, causes the device to perform the voice recognition processing targeting the voice of the visitor picked up by the microphone and continues keyword detection processing to determine whether or not the result of the voice recognition processing includes the keywords.

[0008] Furthermore, this disclosure provides a fraud notification method comprising: using a doorbell-type device installed at the entrance and having a microphone for picking up the visitor's voice to pick up the visitor's voice; performing a voice recognition process to recognize the picked-up voice of the visitor; if the result of the voice recognition process includes keywords related to crimes such as fraud, notifying a notification device of warning information regarding the visitor; and, after the call between the visitor and the resident inside the house via the doorbell-type device has ended, continuing the voice recognition process targeting the visitor's voice picked up by the microphone and keyword detection processing to determine whether or not the result of the voice recognition process includes the keywords.

[0009] These comprehensive or specific embodiments may be implemented as systems, devices, methods, integrated circuits, computer programs, or recording media, or as any combination of systems, devices, methods, integrated circuits, computer programs, and recording media. [Effects of the Invention]

[0010] According to this disclosure, it is possible to detect the possibility of fraud or other crimes occurring and notify third parties at an early stage. [Brief explanation of the drawing]

[0011] [Figure 1] This diagram schematically shows an example of the operation overview of the intercom system according to this embodiment. [Figure 2] A diagram showing an example of the system configuration of the intercom system according to this embodiment. [Figure 3] Block diagram showing example hardware configurations for the door station and intercom master unit. [Figure 4] Block diagram showing an example of server computer hardware configuration. [Figure 5] Diagram showing an example of the structure of a special fraud keyword database. [Figure 6] Block diagram showing an example of smartphone hardware configuration. [Figure 7]Sequence diagram showing an example of the operation procedure of the intercom system according to this embodiment in chronological order. [Modes for carrying out the invention]

[0012] The following description will detail embodiments of the intercom system and fraud notification method disclosed herein, with appropriate reference to the drawings. However, unnecessarily detailed explanations may be omitted. For example, detailed explanations of already well-known matters and redundant explanations of substantially identical components may be omitted. This is to avoid the following description becoming unnecessarily verbose and to facilitate understanding by those skilled in the art. The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand this disclosure and are not intended to limit the subject matter of the claims. In addition, in the following description, the same elements may be assigned the same reference numerals to simplify or omit explanations.

[0013] 1. Overview of the Intercom System's Operation First, with reference to Figure 1, the operation overview of the intercom system 100 according to this embodiment will be explained. Figure 1 is a schematic diagram showing an example of the operation overview of the intercom system 100 according to this embodiment. In the following explanation, as a use case of the intercom system 100 according to this embodiment, a scenario in which resident P1 of a private home speaks at the front door to respond to a visitor (for example, visitor P2) who is a con artist visiting the private home. It goes without saying that the use cases of the intercom system 100 according to this embodiment are not limited to the example shown in Figure 1.

[0014] The door station 10 (an example of a doorbell-type device) that constitutes the intercom system 100 according to this embodiment starts recording the voice of a visitor (e.g., visitor P2) when the resident P1 answers the door station 10. The intercom system 100 starts voice recognition processing on the voice of the visitor (e.g., visitor P2) recorded by the door station 10 (see [1]). The intercom system 100 performs keyword detection processing to determine whether a predetermined keyword (see Figure 5) is included in the result of the voice recognition processing on the voice of the visitor (e.g., visitor P2) (see [2]). For example, suppose the voice of the visitor (e.g., visitor P2) is, "I'm △ from the ○○ branch. I'd like to register my credit card..." and "credit card" is pre-registered as a predetermined keyword. As will be described in detail with reference to Figure 5, the predetermined keyword is a keyword that directly or indirectly indicates an attempt to commit a crime such as special fraud and is pre-registered in a database.

[0015] If the intercom system 100 determines that the voice recognition processing result for the voice of a visitor (e.g., visitor P2) contains a predetermined keyword (see Figure 5), it will notify the intercom master unit 20 used by resident P1's cohabitant P3, or the smartphone 50 owned by a distant family member P4, which have been pre-registered as notification destination devices, of a warning information W1 suggesting that resident P1 may be subjected to fraudulent activity. Note that the person who owns the notification destination device is not limited to resident P1's cohabitant P3 or distant family member P4, but may also be an acquaintance of resident P1. The warning information W1 may be, for example, "We detected suspicious, fraudulent-sounding words at your front door!", but it goes without saying that it is not limited to this phrase.

[0016] 2. Intercom System Configuration Next, referring to FIGS. 2 to 6, a configuration example of the intercom system 100 according to the present embodiment will be described. FIG. 2 is a diagram showing a system configuration example of the intercom system according to the present embodiment. FIG. 3 is a block diagram showing a hardware configuration example of the entrance sub-unit and the intercom main unit. FIG. 4 is a block diagram showing a hardware configuration example of the server computer. FIG. 5 is a diagram showing a configuration example of the special fraud keyword database. FIG. 6 is a block diagram showing a hardware configuration example of the smartphone.

[0017] As shown in FIG. 2, the intercom system 100 according to the present embodiment includes an entrance sub-unit 10, an intercom main unit 20, a voice recognition server 40a, and a special fraud detection server 40b. The entrance sub-unit 10 and the intercom main unit 20, the intercom main unit 20 and the voice recognition server 40a, the voice recognition server 40a and the special fraud detection server 40b, and the intercom main unit 20 and the smartphone 50 are connected to each other via a known wired or wireless network so that data signal communication is possible. In particular, the intercom main unit 20 and the smartphone 50 may be connected via a network NW1 such as a mobile phone network using 4G (the fourth-generation mobile communication system) or 5G (the fifth-generation mobile communication system).

[0018] The entrance sub-unit 10 is an example of a doorbell-type device, is installed at the entrance of a private house (see FIG. 1), and has a microphone 16 (see FIG. 3) capable of picking up the voice of a visitor P2 (see FIG. 1). The entrance sub-unit 10 transmits a data signal of the voice picked up by the microphone 16 (including the voice of the visitor P2) to the intercom main unit 20. Details of the hardware configuration example of the entrance sub-unit 10 will be described later with reference to FIG. 3.

[0019] The intercom master unit 20 is an example of a control device, which receives and acquires a data signal of voice from the entrance sub-unit 10 and executes various processes using this data signal. The intercom master unit 20 generates an execution instruction for voice recognition processing targeting the data signal of the voice including the data signal of voice from the entrance sub-unit 10 and sends it to the voice recognition server 40a, or receives the processing result of the special fraud detection server 40b via the voice recognition server 40a. Based on the processing result of the special fraud detection server 40b, the intercom master unit 20 generates and notifies (transmits) alert information W1 (see FIG. 1) to the smartphone 50 registered in advance as a notification destination device. Details of an example of the hardware configuration of the intercom master unit 20 will be described later with reference to FIG. 3.

[0020] The voice recognition server 40a is an example of a device that, based on execution instructions from the intercom master unit 20, performs voice recognition processing on the voice data signal of visitor P2 sent from the intercom master unit 20. The voice data signal sent from the intercom master unit 20 may include not only the voice spoken by visitor P2, but also the voice spoken by resident P1 who conversed with visitor P2. This is because it is conceivable that resident P1 might ask for the words of the phrase to be repeated, not only if visitor P2 speaks a phrase that suggests fraud, etc. Furthermore, the voice data signal targeted for voice recognition processing is not limited to the voices of resident P1 and visitor P2, but may also include the voice of a person other than resident P1 in the house (for example, a family member living together) who speaks to the intercom master unit 20. The voice recognition processing performed by the voice recognition server 40a may be a known method, and may be implemented, for example, by referring to a database such as a dictionary for voice recognition processing that the voice recognition server 40a has in advance and performing text conversion processing corresponding to the voice data signal. The speech recognition server 40a generates an execution instruction for keyword detection processing, including the results of the speech recognition processing, and sends it to the special fraud detection server 40b. In Figure 2, the speech recognition server 40a is provided separately from the intercom master unit 20, but the configuration of the speech recognition server 40a may be built into the intercom master unit 20 or the special fraud detection server 40b. Details of the hardware configuration example of the speech recognition server 40a will be described later with reference to Figure 4.

[0021] The special fraud detection server 40b performs keyword detection processing based on execution instructions from the voice recognition server 40a, using the results of the voice recognition processing sent from the voice recognition server 40a to determine whether or not there are any matching special fraud keywords. The special fraud detection server 40b performs keyword detection processing by referring to the special fraud keyword database 49b, for example, shown in Figure 5, and checking whether or not there are any matching keywords in the results of the voice recognition processing sent from the voice recognition server 40a that are registered in the special fraud keyword database 49b. The special fraud detection server 40b sends the results of the keyword detection processing to the voice recognition server 40a. In Figure 2, the special fraud detection server 40b is provided separately from the intercom master unit 20, but the configuration of the special fraud detection server 40b may be built into the intercom master unit 20 or the voice recognition server 40a. Details of the hardware configuration example of the special fraud detection server 40b will be described later with reference to Figure 4.

[0022] Smartphone 50 is an example of a notification recipient device and is held by a cohabitant P3 of resident P1 or a distant family member P4. Smartphone 50 receives and displays the warning information W1 (see Figure 1) generated and sent by the intercom master unit 20. As a result, cohabitant P3 or distant family member P4 can notice and view the warning information W1 displayed on smartphone 50, thereby becoming aware early of the possibility of fraudulent activity against, for example, their parents, resident P1, and taking appropriate action. Details of the hardware configuration example of smartphone 50 will be described later with reference to Figure 6.

[0023] (1) Configuration of the door station 10 and the intercom master unit 20 As shown in Figure 3, the door station 10 includes a ROM 11, RAM 12, a camera 13, a button 14, a control unit 15, a microphone 16, and a speaker 17. The ROM 11, RAM 12, camera 13, button 14, microphone 16, and speaker 17 are each connected to the control unit 15 via an internal bus or the like to enable signal input and output.

[0024] ROM11 is Read Only Memory and holds the programs and data necessary for the operation of the door station 10. RAM12 is Random Access Memory (RAM) and is a work memory that temporarily holds data signals acquired or generated by the control unit 15 during operation.

[0025] Camera 13 is an imaging device positioned at the entrance of a private residence where the door station 10 is installed, and primarily captures images of people approaching the entrance. Camera 13 sends video data signals based on the image data obtained through imaging to the control unit 15.

[0026] Button 14 is a button that emits a chime sound to indicate that a person has arrived in front of the door station 10. When button 14 is pressed, a signal indicating that it has been pressed is transmitted to the control unit 15, and the control unit 15 can detect that someone has pressed button 14. Note that button 14 is not limited to a button that emits a chime sound.

[0027] The control unit 15 is composed of at least one processor, such as a Central Processing Unit (CPU), Digital Signal Processor (DSP), Field Programmable Gate Array (FPGA), or Graphical Processing Unit (GPU). The control unit 15 functions as a controller that oversees the overall operation of the door station 10. The control unit 15 performs control processing to coordinate the operation of each part of the door station 10, input / output processing of data signals between each part of the door station 10, arithmetic processing of data signals, and storage processing of data signals. The control unit 15 operates according to the program stored in the ROM 11. When operating, the control unit 15 uses the ROM 11 to perform various processes in cooperation with the control unit 15 and temporarily stores the data signals generated or acquired by the control unit 15 in the RAM 12. The control unit 15 also transmits the generated or acquired data signals to the intercom master unit 20.

[0028] The microphone 16 is a sound-collecting device that picks up sound from in front of the location where the door station 10 is installed (for example, the sound emitted by a visitor (e.g., visitor P2) who presses button 14). The data signal of the sound picked up by the microphone 16 is transmitted to the control unit 15. The speaker 17 acoustically outputs the sound transmitted (transmitted) from the intercom master unit 20 (for example, the sound emitted by resident P1 inside the house).

[0029] As shown in Figure 3, the intercom master unit 20 includes a ROM 21, a RAM 22, a door station communication I / F unit 23, a wired LAN communication I / F unit 24, a display / operation unit 25, a control unit 26, an audio signal bus 27, an audio signal input / output control unit 28, a microphone 29, a speaker 30, a wireless LAN communication I / F unit 31, and an antenna 32. The ROM 21, RAM 22, door station communication I / F unit 23, wired LAN communication I / F unit 24, audio signal input / output control unit 28, and wireless LAN communication I / F unit 31 are each connected to the control unit 26 via an internal bus including the audio signal bus 27, etc., to enable signal input and output.

[0030] ROM21 is Read Only Memory and holds the programs and data necessary for the operation of the intercom master unit 20. RAM22 is Random Access Memory (RAM) and is a work memory that temporarily holds data signals acquired or generated by the control unit 26 during operation.

[0031] The door station communication interface 23 is configured using an integrated circuit that performs data signal communication between the intercom master unit 20 and the door station 10. The door station communication interface 23 receives data signals from the door station 10 and transmits data signals to the door station 10. As described above, the data signal communication between the door station 10 and the intercom master unit 20 may be wired or wireless.

[0032] The wired LAN communication interface 24 is configured using an integrated circuit that communicates data signals with external devices (for example, the voice recognition server 40a, the special fraud detection server 40b, or other devices) connected via a wired Local Area Network (LAN). The wired LAN communication interface 24 receives data signals from external devices (see above) and sends them to the control unit 26 via the sound signal bus 27, or transmits data signals sent via the sound signal bus 27 to external devices (see above).

[0033] The display / operation unit 25 is a display configured using, for example, a Liquid Crystal Display (LCD) or an organic EL element, and receives operations from the user (i.e., the person operating the intercom master unit 20) and sends a signal corresponding to that operation to the control unit 26. The display / operation unit 25 may be configured as a touch panel display and may, for example, display warning information W1 generated by the control unit 26, or display video data signals from the camera 13 sent from the door station 10 (for example, video of visitor P2).

[0034] The control unit 26 is composed of at least one processor, such as a Central Processing Unit (CPU), Digital Signal Processor (DSP), Field Programmable Gate Array (FPGA), or Graphical Processing Unit (GPU). The control unit 26 functions as a controller that oversees the overall operation of the intercom master unit 20. The control unit 26 performs control processing to coordinate the operation of each part of the intercom master unit 20, input / output processing of data signals between each part of the intercom master unit 20, arithmetic processing of data signals, and storage processing of data signals. The control unit 26 operates according to a program stored in the ROM 21. During operation, the control unit 26 uses the ROM 21 to perform various processes in cooperation with the control unit 26 and temporarily stores the data signals generated or acquired by the control unit 26 in the RAM 22. The control unit 26 also transmits (transmits) the generated or acquired data signals to the door station 10 or displays them on the display / operation unit 25.

[0035] The audio signal bus 27 is an internal bus through which audio signals input to the control unit 26 and audio signals sent from the control unit 26 to other parts pass. It handles the input and output of audio signals between the wired LAN communication interface 24, the control unit 26, the audio signal input / output control unit 28, and the wireless LAN communication interface 31.

[0036] The sound signal input / output control unit 28 receives the voice of a person (for example, a resident P1 or cohabitant P3 in a private home) picked up by the microphone 29 and transmits it directly to the control unit 26 or to the wired LAN communication I / F unit 24 or the wireless LAN communication I / F unit 31 via the sound signal bus 27. The sound signal input / output control unit 28 acoustically outputs the voice data signal from the control unit 26 through the speaker 30.

[0037] Microphone 29 is a sound-collecting device that captures the voices of people inside a private home (for example, resident P1 or cohabitant P3). The data signal of the voice captured by microphone 29 is transmitted to the sound signal input / output control unit 28. Speaker 30 acoustically outputs the voice captured by the door station 10 (for example, the voice of visitor P2) via the door station communication interface 23, control unit 26, sound signal bus 27, and sound signal input / output control unit 28.

[0038] The wireless LAN communication interface unit 31 is configured using an integrated circuit that communicates data signals with external devices (e.g., the voice recognition server 40a, the special fraud detection server 40b, or other devices) connected via a wireless Local Area Network (LAN). The wireless LAN communication interface unit 31 receives data signals from external devices (see above) and sends them via the sound signal bus 27 to the control unit 26, the wired LAN communication interface unit 24, or the sound signal input / output control unit 28, or transmits the data signals sent via the sound signal bus 27 to external devices (see above) via the antenna 33.

[0039] (2) Configuration of the voice recognition server 40a and the special fraud detection server 40b Figure 4 shows an example of the hardware configuration of the server computer 40 that is common to both the speech recognition server 40a and the special fraud detection server 40b. Therefore, in the explanation of Figure 4, the example of the server computer 40 hardware configuration common to both the speech recognition server 40a and the special fraud detection server 40b will be explained first, and the configurations specific to the speech recognition server 40a and the special fraud detection server 40b will be explained later.

[0040] As shown in Figure 4, the server computer 40 includes a ROM 41, RAM 42, a wired LAN communication I / F unit 43, a control unit 44, an OS processing unit 45, an application processing unit 46, a display / operation unit 47, an HDD 48, and a DB 49. The ROM 41, RAM 42, wired LAN communication I / F unit 43, display / operation unit 47, and HDD 48 are each connected to the control unit 44 via an internal bus or the like to enable signal input and output.

[0041] ROM 41 is Read Only Memory and holds the programs and data necessary for the operation of the server computer 40. RAM 22 is Random Access Memory (RAM) and is a work memory that temporarily holds data signals acquired or generated by the control unit 44 during operation.

[0042] The wired LAN communication interface unit 43 is configured using an integrated circuit that communicates data signals with external devices (for example, the intercom master unit 20, the voice recognition server 40a, or the special fraud detection server 40b) connected via a wired Local Area Network (LAN). The wired LAN communication interface unit 43 receives data signals from external devices (see above) and sends them to the control unit 44, and also transmits data signals from the control unit 44 to external devices (see above).

[0043] The control unit 44 is composed of at least one processor, such as a Central Processing Unit (CPU), Digital Signal Processor (DSP), Field Programmable Gate Array (FPGA), or Graphical Processing Unit (GPU). The control unit 44 functions as a controller that oversees the overall operation of the server computer 40. The control unit 44 performs control processing to coordinate the operation of each part of the server computer 40, input / output processing of data signals between the server computer 40 and each part, arithmetic processing of data signals, and storage processing of data signals. The control unit 44 operates according to the program stored in the ROM 41. During operation, the control unit 44 uses the ROM 41 to perform various processes in cooperation with the control unit 44 and temporarily stores the data signals generated or acquired by the control unit 44 in the RAM 42. The control unit 44 also transmits the generated or acquired data signals to external devices (see above) or displays them on the display / operation unit 47. Furthermore, the control unit 44 refers to the program stored in the ROM 41 and functionally implements the OS processing unit 45 and the application processing unit 46.

[0044] The OS processing unit 45 executes software processing as an operating system, which is the basic processing of the server computer 40.

[0045] The application processing unit 46 executes the processing of various applications that are installed and executable on the server computer 40. For example, if the server computer 40 is a speech recognition server 40a, the application processing unit 46 refers to a database of speech recognition dictionaries (not shown) stored in DB49 of HDD 48 and performs speech recognition processing of the voice data signal from the door station 10 sent from the intercom master unit 20. Also, if the server computer 40 is a special fraud detection server 40b, the application processing unit 46 refers to a special fraud keyword database 49b (see Figure 5) stored in DB49 of HDD 48 and performs keyword detection processing to determine whether or not there are any results in the speech recognition processing that match special fraud keywords (see below).

[0046] The display / operation unit 47 is a display configured using, for example, a Liquid Crystal Display (LCD) or an organic EL element, and receives user input and sends a signal corresponding to that input to the control unit 44. The display / operation unit 47 may be configured as a touch panel display. Note that the display / operation unit 47 may be omitted from the configuration of the server computer 40.

[0047] HDD48 is an example of storage, and it stores data signals generated or acquired by the control unit 44, or data signals sent from external devices (e.g., the door station 10 or the intercom master unit 20). HDD48 is equipped with DB49.

[0048] DB49 is a database that is referenced depending on the content of the processing performed by the control unit 44 or the application processing unit 46. For example, if the server computer 40 is a speech recognition server 40a, DB49 is a database of dictionaries for speech recognition processing that is referenced during speech recognition processing. For example, if the server computer 40 is a special fraud detection server 40b, DB49 is a special fraud keyword database 49b (see Figure 5) that is referenced during keyword detection processing.

[0049] Now, with reference to Figure 5, we will explain the special fraud keyword database 49b.

[0050] As shown in Figure 5, the special fraud keyword database 49b is a database in which keywords related to special fraud are pre-registered. In the example in Figure 5, keywords such as credit card, cash, renovation, inspection, etc. are shown. If such keywords are detected in the voice of visitor P2 during the keyword detection process, it is suggested that visitor P2 is likely to engage in fraudulent activity. Therefore, in order to prevent fraudulent activity from occurring, this special fraud keyword database 49b is pre-registered in the server computer 40.

[0051] It should be noted that keywords related to special fraud may change from moment to moment in light of social conditions, etc., therefore, the keywords registered in the Special Fraud Keyword Database 49b may be updated (i.e., added, edited, or deleted) as needed. Editing here refers to changing the name of a keyword. This Special Fraud Keyword Database 49b may be updated regularly or irregularly.

[0052] (3) Configuration of Smartphone 50 As shown in Figure 6, the smartphone 50 includes a ROM 51, RAM 52, microphone 53, speaker 54, display / operation unit 55, antenna 56, location information positioning unit 57, battery 58, sound signal input / output control unit 59, antenna 60, wireless LAN communication I / F unit 61, antenna 62, BT communication I / F unit 63, short-range wireless control unit 64, sound signal bus 65, antenna 66, public network communication I / F unit 67, public network protocol control unit 68, control unit 69, smartphone OS processing unit 70, and smartphone application processing unit 71. The ROM 51, RAM 52, display / operation unit 55, location information positioning unit 57, sound signal input / output control unit 59, short-range wireless control unit 64, and public network protocol control unit 68 are each connected to the control unit 69 via an internal bus including the sound signal bus 65, etc., to enable signal input and output.

[0053] ROM 51 is Read Only Memory and holds the programs and data necessary for the operation of the smartphone 50. RAM 52 is Random Access Memory (RAM) and is a work memory that temporarily holds data signals acquired or generated by the control unit 69 during operation.

[0054] Microphone 53 is a sound-collecting device that captures the voice of a person P3 who lives with resident P1 or a distant family member P4. The voice data signal captured by microphone 53 is transmitted to the sound signal input / output control unit 59. Speaker 54 acoustically outputs the voice data signal of the warning information W1, which is sent from, for example, the intercom master unit 20 and acquired by the control unit 69, via the control unit 69, the sound signal bus 65, and the sound signal input / output control unit 59.

[0055] The display / operation unit 55 is a display configured using, for example, a Liquid Crystal Display (LCD) or an organic EL element, and receives operations from the user (i.e., a cohabitant P3 of resident P1 or a distant family member P4) and sends a signal corresponding to that operation to the control unit 69. The display / operation unit 55 may be configured as a touch panel display and, for example, displays image data of the warning information W1 sent from the intercom master unit 20 and acquired by the control unit 69.

[0056] The location information positioning unit 57 determines the current location of the smartphone 50 based on the signal received via the antenna 56. The positioning result from the location information positioning unit 57 is periodically input to the control unit 69.

[0057] The battery 58 is, for example, a rechargeable battery, and supplies the power necessary for the operation of each part of the smartphone 50, according to each part.

[0058] The sound signal input / output control unit 59 receives the voice of a person (for example, a cohabitant P3 of resident P1 or a distant family member P4) picked up by the microphone 53 and transmits it directly to the control unit 69 or to the short-range wireless control unit 64 or the public line protocol control unit 68 via the sound signal bus 65. The sound signal input / output control unit 59 acoustically outputs the voice data signal from the control unit 69 through the speaker 54.

[0059] The wireless LAN communication interface 61 is configured using an integrated circuit that communicates data signals with external devices (e.g., the intercom master unit 20) connected via a network NW1 (see Figure 2), such as a wireless Local Area Network (LAN). The wireless LAN communication interface 61 receives data signals from external devices (see above) via the antenna 60 and sends them to the short-range wireless control unit 64, and also sends data signals from the short-range wireless control unit 64 to external devices (see above) via the antenna 60.

[0060] The BT communication interface unit 63 is configured using an integrated circuit that communicates data signals with an external device (e.g., the intercom master unit 20) connected via a network NW1 (see Figure 2), such as Bluetooth®. The BT communication interface unit 63 receives data signals from the external device (see above) via the antenna 62 and sends them to the short-range wireless control unit 64, and also sends data signals from the short-range wireless control unit 64 to the external device (see above) via the antenna 62.

[0061] The short-range wireless control unit 64 performs input and output of audio signals (voice data signals) between the wireless LAN communication I / F unit 61 and the BT communication I / F unit 63 and the audio signal bus 65.

[0062] The audio signal bus 65 is an internal bus through which audio signals input to the control unit 69 and audio signals sent from the control unit 69 to other parts pass, and it performs audio signal input and output between the audio signal input / output control unit 59, the short-range wireless control unit 64, the control unit 69, and the public line protocol control unit 68.

[0063] The public network communication interface (I / F) unit 67 is configured using an integrated circuit that communicates data signals with external devices (e.g., the intercom master unit 20) connected via a network NW1 such as a mobile phone network. The public network communication interface (I / F) unit 67 receives data signals from external devices (see above) via the antenna 66 and sends them to the public network protocol control unit 68, and transmits data signals sent from the public network protocol control unit 68 to external devices (see above) via the antenna 66.

[0064] The public network protocol control unit 68 performs input and output of sound signals (voice data signals) with the public network communication I / F unit 67, the sound signal bus 65, and the control unit 69, respectively.

[0065] The control unit 69 is composed of at least one processor, such as a Central Processing Unit (CPU), Digital Signal Processor (DSP), Field Programmable Gate Array (FPGA), or Graphical Processing Unit (GPU). The control unit 69 functions as a controller that oversees the overall operation of the smartphone 50. The control unit 69 performs control processing to coordinate the operation of each part of the smartphone 50, input / output processing of data signals between each part of the smartphone 50, arithmetic processing of data signals, and storage processing of data signals. The control unit 69 operates according to a program stored in the ROM 51. During operation, the control unit 69 uses the ROM 51 to perform various processes in cooperation with other components and temporarily stores the data signals generated or acquired by the control unit 69 in the RAM 52. The control unit 69 also transmits the generated or acquired data signals to one of the sound signal input / output control unit 59, the short-range wireless control unit 64, and the public network protocol control unit 68, or displays them on the display / operation unit 55. Furthermore, the control unit 69 refers to the program stored in the ROM 51 and functionally implements the smartphone OS processing unit 70 and the smartphone application processing unit 71.

[0066] The smartphone OS processing unit 70 performs software processing as an operating system, which is the basic processing of the smartphone 50.

[0067] The smartphone application processing unit 71 executes the processing of various applications installed and executable on the smartphone 50. For example, the smartphone application processing unit 71 displays the warning information W1 sent from the intercom master unit 20 on the display / operation unit 55.

[0068] 3. Operation of the intercom system Next, the operation of the intercom system 100 according to this embodiment will be described with reference to Figure 7. Figure 7 is a sequence diagram showing an example of the operation procedure of the intercom system 100 according to this embodiment in chronological order.

[0069] In Figure 7, suppose visitor P2 arrives at the private residence and presses the chime sound button 14 located on the door station 10 (St1). The door station 10 detects the press of button 14 and, based on this detection, starts recording video data including image data captured by the camera 13 and audio data picked up by the microphone 16 (St2), and transmits the video data to the intercom master unit 20. While the intercom master unit 20 receives the video data sent from the door station 10, it extracts at least the audio data from the video data and transmits it to the voice recognition server 40a (St3).

[0070] Furthermore, after starting recording in step St2, the intercom master unit 20 generates an in-room visitor notification indicating that a visitor (e.g., visitor P2) has arrived at the entrance of the private residence and displays it on the display / operation unit 25 (St4). This allows the resident P1 inside the private residence to become aware of visitor P2's arrival, and a call between resident P1 and visitor (e.g., visitor P2) via the door station 10 and intercom master unit 20 is initiated and continues, and video data including audio and video of the call is stored in the intercom master unit 20. If resident P1 ends the call via the door station 10 and intercom master unit 20 in order to have a face-to-face conversation with visitor P2, resident P1 performs an end-of-call operation on the intercom master unit 20 (e.g., display / operation unit 25) (St5). This ends the call between resident P1 and visitor (e.g., visitor P2) via the door station 10 and intercom master unit 20 (end of call) (St6). However, even after the call has ended, the intercom master unit 20 does not immediately send a request to stop the recording that was started in step St2 to the door station 10, but continues to receive video data sent from the door station 10.

[0071] From step St6 onward, resident P1 arrives at the front door, and the conversation between resident P1 and visitor P2 resumes in a face-to-face format at the front door. Even after this conversation resumes, the recording of video data by the door station 10, which was started in step St2, continues.

[0072] Meanwhile, when the voice recognition server 40a receives voice data sent from the intercom master unit 20 in step St3, it performs voice recognition processing using that voice data. The voice recognition server 40a generates a keyword matching request including the results of the voice recognition processing and sends it to the special fraud detection server 40b (St7). Based on the keyword matching request sent in step St7, the special fraud detection server 40b performs keyword detection processing to determine whether or not there is a match with a special fraud keyword (see Figure 5) in the results of the voice recognition processing (St8). The special fraud detection server 40b returns the result of the keyword detection processing (matching result) to the voice recognition server 40a only when it determines that there is a match with a special fraud keyword (see Figure 5) in the results of the voice recognition processing (St9). The voice recognition server 40a receives the matching result returned from the special fraud detection server 40b in step St9 and forwards it to the intercom master unit 20 (St10).

[0073] Based on the matching results transmitted in step St10, the intercom master unit 20 generates warning information W1 (see Figure 1) and displays it on the display / operation unit 25 or outputs audio from the speaker 30 (St11). After or almost simultaneously with step St11, the intercom master unit 20 generates warning information W1 that a conversation containing utterances matching the special fraud keyword has been made with resident P1 and notifies the smartphone 50 (St12). The smartphone 50 displays the warning information W1 notified in step St12 as an alert on the display / operation unit 55 (St13). The content of the screen shown by the warning information W1 may be the same on the smartphone 50 and the intercom master unit 20, or the same content may be displayed on the smartphone 50 and the intercom master unit 20, but the display format (e.g., the design of the screen) may be different.

[0074] Let's assume that the face-to-face conversation between resident P1 and visitor P2 at the front door has ended. Even at the end of this conversation, the video data recording by the door station 10, which was started in step St2, continues.

[0075] Then, at step St2, the door station 10 determines that a predetermined time Tout has elapsed since recording started, and stops the recording process (St14). In other words, the door station 10 continues recording for the predetermined time Tout from step St2 to step St14. In the example in Figure 7, the recording process is stopped based on the elapsed time Tout, but the door station 10 may also stop recording when it stops receiving audio from visitor P2.

[0076] (Summary of this disclosure) The above description of embodiments discloses the technical concepts corresponding to the following items.

[0077] (Item 1) The intercom system related to this disclosure is A doorbell-type device (door station 10) is installed at the entrance and has a microphone (16) that picks up the voice of the visitor (P2), A database containing keywords related to crimes such as fraud (Special Fraud Keyword Database 49b), A device (speech recognition server 40a) that performs speech recognition processing to recognize the voice of the aforementioned visitor, The system includes a control device (intercom master unit 20) that, if the result of the voice recognition processing contains the keyword, notifies a notification device (smartphone 50) of warning information regarding the visitor, After the call between the visitor via the doorbell-type device and the resident inside the house has ended, the control device continues to perform the voice recognition processing on the visitor's voice picked up by the microphone, and then continues the keyword detection processing to determine whether or not the keyword is included in the result of the voice recognition processing. As a result, the intercom system can properly detect the presence or absence of phrases that suggest fraud or other crimes (e.g., special fraud) from the voice of a visitor, even after the conversation between the visitor and the resident has started and ended. Therefore, it can detect the possibility of fraud or other crimes (e.g., special fraud) occurring and notify a third party at an early stage.

[0078] (Item 2) In the intercom system described in item 1, The control device is an intercom master unit located inside the house and capable of communicating with the doorbell type device via wired or wireless means. The aforementioned intercom master unit incorporates the database and the device. This allows the intercom system to perform not only voice recognition processing of visitors' voices but also the extraction of keywords related to crimes such as fraud using the results of that voice recognition processing, thereby simplifying the system configuration.

[0079] (Item 3) In the intercom system described in item 1, The doorbell-type device incorporates the database and the device itself. This allows the intercom system to perform not only voice recognition processing of visitors' voices in a doorbell-type device, but also to extract keywords related to crimes such as fraud using the results of that voice recognition processing, thereby simplifying the system configuration.

[0080] (Item 4) In the intercom system described in item 1, The system further comprises servers (voice recognition server 40a, special fraud detection server 40b) that are connected to the doorbell-type device and the control device via wired or wireless communication, The server performs the speech recognition process and the keyword detection process. As a result, according to the intercom system, the server can not only perform speech recognition processing of the visitor's voice but also extract keywords related to crimes such as fraud using the results of that speech recognition processing, thus simplifying the configuration of the doorbell-type device and the control device.

[0081] (Item 5) In the intercom system described in item 2, The aforementioned intercom master unit acquires and displays the warning information as the notification destination device. This means that, according to the intercom system, cohabitants living in the same house as the resident can detect the possibility of fraud or other crimes occurring early on, and take appropriate action against the resident.

[0082] (Item 6) In the intercom system described in item 1, The doorbell-type device outputs a notification based on the warning information as the notification recipient device. According to the intercom system, this allows the doorbell-type device to output warning information, making it possible to prompt visitors to leave early if they are scammers.

[0083] (Item 7) In the intercom system described in any one of items 1 to 6, The notification recipient device is a smartphone owned by a family member or acquaintance of the resident in the house and connected wirelessly to the control device. This allows the intercom system to promptly notify family members or acquaintances of residents inside the house of warning information via their smartphones.

[0084] (Item 8) In the intercom system described in any one of items 1 to 7, The control device continues the keyword detection process until a predetermined time has elapsed since the end of the telephone conversation between the visitor and the resident inside the house. This means that, according to the intercom system, even after the call between the visitor and the resident inside the house has ended, the visitor's monologue or the words of a third party (such as an accomplice in a crime like fraud) can be recorded. This allows for a more accurate detection of crimes such as fraud and enables appropriate action (e.g., contacting the police).

[0085] (Item 9) In the intercom system described in any one of items 1 to 8, The control device continues the keyword detection process until it stops detecting the visitor's voice. As a result, according to the intercom system, recording continues even after the conversation between the visitor and the resident inside the house has ended, until the visitor stops speaking. This allows for recording of things like monologues or conversations with third parties (such as accomplices in fraud or other crimes), enabling more accurate detection of fraud and other crimes, and allowing for appropriate action (such as contacting the police).

[0086] (Item 10) The method of notifying fraud related to this disclosure is: The doorbell-type device, installed at the entrance and equipped with a microphone to capture the visitor's voice, will capture the visitor's voice. This involves performing speech recognition processing to recognize the recorded voice of the visitor, If the results of the aforementioned speech recognition processing include keywords related to crimes such as fraud, a warning information regarding the visitor will be sent to the notification device. After the call between the visitor and the resident inside the house via the doorbell-type device has ended, the system continues with the execution of the speech recognition process targeting the visitor's voice picked up by the microphone, and the keyword detection process that determines whether or not the keyword is included in the result of the speech recognition process. As a result, this fraud notification method allows for the proper detection of phrases suggesting fraud or other crimes (e.g., special fraud) from the voice of a visitor, even after the conversation between the visitor and the resident has begun and ended. Therefore, it is possible to detect the possibility of fraud or other crimes (e.g., special fraud) occurring and notify a third party at an early stage.

[0087] While embodiments have been described above with reference to the attached drawings, this disclosure is not limited to such examples. It is clear to those skilled in the art that various modifications, alterations, substitutions, additions, deletions, and equivalents can be conceived within the scope of the claims, and these are also understood to fall within the technical scope of this disclosure. Furthermore, the components of the embodiments described above can be combined in any way without departing from the spirit of the invention. [Industrial applicability]

[0088] The technology disclosed herein is useful as an intercom system and a method for notifying third parties of the possibility of fraud or other crimes occurring and at an early stage. [Explanation of Symbols]

[0089] 10 Door station 11 ROM 12 RAM 13 Cameras 14 buttons 15 Control Unit 16 Mike 17 speakers 20 Intercom Master Unit 21 ROM 22 RAM 23. Entrance Station Communication I / F Section 24 Wired LAN communication I / F section 25 Display / operation section 26 Control Unit 27 Tone signal bus 28 Audio signal input / output control unit 29 Mike 30 speakers 31 Wireless LAN communication I / F section 32 Antennas 40 Server Computers 40a Speech Recognition Server 40b Special fraud detection server 41 ROM 42 RAM 43 Wired LAN communication I / F section 44 Control Unit 45 OS Processing Unit 46 Application Processing Unit 47 Display / operation section 48 HDD 49 DB 49b Special Fraud Keyword Database 50 Smartphones NW1 Network

Claims

1. A doorbell-type device installed at the entrance, equipped with a microphone to capture the voice of visitors, A database containing keywords related to crimes such as fraud, A device that performs speech recognition processing to recognize the voice of the aforementioned visitor, The system includes a control device that, if the result of the speech recognition process contains the keyword, notifies a notification destination device of warning information regarding the visitor, After the call between the visitor via the doorbell-type device and the resident inside the house has ended, the control device continues to perform the voice recognition processing on the visitor's voice picked up by the microphone and to perform keyword detection processing to determine whether or not the keyword is included in the result of the voice recognition processing. Intercom system.

2. The control device is an intercom master unit located inside the house and capable of communicating with the doorbell type device via wired or wireless means. The aforementioned intercom master unit incorporates the database and the device, The intercom system according to claim 1.

3. The doorbell-type device incorporates the database and the device, The intercom system according to claim 1.

4. The system further comprises a server that is connected to the doorbell-type device and the control device via wired or wireless communication, The server performs the speech recognition process and the keyword detection process. The intercom system according to claim 1.

5. The aforementioned intercom master unit acquires and displays the warning information as the notification destination device. The intercom system according to claim 2.

6. The doorbell-type device outputs a notification based on the warning information as the notification recipient device. The intercom system according to claim 1.

7. The notification recipient device is a smartphone owned by a family member or acquaintance of the resident in the house and connected wirelessly to the control device. The intercom system according to claim 1.

8. The control device continues the keyword detection process until a predetermined time has elapsed since the end of the telephone conversation between the visitor and the resident inside the house. The intercom system according to claim 1.

9. The control device continues the keyword detection process until it no longer detects the visitor's voice. The intercom system according to claim 1.

10. The doorbell-type device, installed at the entrance and equipped with a microphone to capture the visitor's voice, will capture the visitor's voice. This involves performing speech recognition processing to recognize the recorded voice of the visitor, If the results of the aforementioned speech recognition processing include keywords related to crimes such as fraud, a warning information regarding the visitor will be sent to the notification device. After the call between the visitor and the resident inside the house via the doorbell-type device has ended, the system continues by performing the speech recognition process on the visitor's voice picked up by the microphone and determining whether or not the keyword is included in the result of the speech recognition process. Fraud notification methods.

Images