Image processing device and image processing system

Abstract

This image processing device comprises: an acquisition unit that acquires a captured image of a visitor; an extraction unit that extracts a feature of the visitor from the captured image; and an inference unit that infers, on the basis of the feature of the visitor, visitor information which indicates an attribute of the visitor and a purpose of visit of the visitor.

Description

Image Processing Apparatus and Image Processing System

[0001] The present invention relates to an image processing apparatus and an image processing system.

[0002] In a home where a residential intercom is installed, even when the intercom rings, a resident may pretend to be out and not answer. However, when the resident pretends to be out, unnecessary operation is caused because a visitor such as a parcel delivery service provider needs to visit again. In addition, when the resident pretends to be out, a burglar may determine that the home is vacant, increasing the risk that the resident will be involved in a crime.

[0003] Patent Document 1 discloses an intercom that can identify a visiting service provider.

[0004] Japanese Unexamined Patent Publication No. 2011-223328

[0005] However, the intercom according to Patent Document 1 cannot estimate the purpose of a visitor's visit.

[0006] Accordingly, an object of the present invention is to provide an image processing apparatus capable of estimating the purpose of a visitor's visit when the visitor calls a resident using an intercom.

[0007] An image processing apparatus according to a preferred aspect of the present invention includes: an acquisition unit that acquires a captured image of a visitor; an extraction unit that extracts features of the visitor from the captured image; and an estimation unit that estimates visitor information indicating an attribute of the visitor and a purpose of the visitor's visit based on the features of the visitor.

[0008] According to the present invention, when a visitor calls a resident using an intercom, it is possible to estimate the purpose of the visitor's visit.

[0009] A block diagram showing an example of the overall configuration of the image processing system 1. A block diagram showing an example of the configuration of the response device 40[k]. An explanatory diagram showing an example of the configuration of the outdoor unit 60[k]. An explanatory diagram showing an example of the configuration of the indoor unit 70[k]. A block diagram showing an example of the configuration of the machine learning device 20. A block diagram showing an example of the configuration of the image processing device 10. A functional block diagram of the decision unit 115. An explanatory diagram showing an example of the configuration of the behavior information database ADB. An explanatory diagram showing an example of the configuration of the history information database HDB. A flowchart showing an example of the operation of the image processing device 10. A block diagram showing an example of the overall configuration of the image processing system 1A. A functional block diagram of the decision unit 115.

[0010] 1: The first embodiment of the image processing system 1 will be described below with reference to Figures 1 to 10.

[0011] 1-1: Configuration of the First Embodiment 1-1-1: Overall Configuration Figure 1 is a block diagram showing an example of the overall configuration of the image processing system 1 according to this embodiment. As shown in Figure 1, the image processing system 1 comprises an image processing device 10, a machine learning device 20, and response devices 40[1] to 40[n]. The image processing device 10, the machine learning device 20, and the response devices 40[1] to 40[n] are connected to each other via a communication network NET so that they can communicate with one another. n is an integer of 1 or more.

[0012] In the following explanation, response devices 40[1] to 40[n] may be collectively referred to as "response device 40". In Figure 1, user U uses response device 40. Also, user U[1] uses response device 40[1]. User U[2] uses response device 40[2]. User U[k] uses response device 40[k]. User U[n] uses response device 40[n]. k is an integer between 1 and n, inclusive. In the following explanation, user U[k] may be used as a representative example of user U, and response device 40[k] may be used as a representative example of response device 40.

[0013] The response device 40[k] is, for example, a response device installed in the residence of user U[k]. The response device 40[k] is, for example, an intercom. When a visitor arrives at the residence of user U[k], the visitor rings the response device 40[k]. In this embodiment, it is assumed that the visitor tells the response device 40[k] their attributes and the purpose of their visit. User U[k] responds to the visitor using the response device 40[k]. The response device 40[k] also captures an image of the visitor.

[0014] The image processing device 10 extracts the characteristics of the visitor from the captured image PI of the visitor acquired from the response device 40[k]. Subsequently, the image processing device 10 estimates visitor information indicating the visitor's attributes and purpose of visit based on the visitor's characteristics. A learning model LM determined based on the results of machine learning is used for this estimation.

[0015] The machine learning device 20 determines a learning model LM that the image processing device 10 will use to estimate visitor information. The determined learning model LM is transmitted from the machine learning device 20 to the image processing device 10.

[0016] 1-1-2: Diagram 2 of the response device configuration is a block diagram showing an example configuration of the response device 40[k]. As shown in Figure 2, the response device 40[k] comprises a processing device 41, a storage device 42, an imaging device 43, a first microphone 44, a first speaker 45, a second microphone 46, a second speaker 47, a display device 48, an input device 49, an alarm device 50, and a communication device 51. Each element of the response device 40[k] is interconnected by one or more buses for communicating information.

[0017] The processing unit 41 is a processor that controls the entire response device 40[k]. The processing unit 41 is configured using, for example, one or more chips. The processing unit 41 is also configured using, for example, a central processing unit (CPU) that includes interfaces with peripheral devices, an arithmetic unit, and registers. Some or all of the functions of the processing unit 41 may be implemented by hardware such as a DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), and FPGA (Field Programmable Gate Array). The processing unit 41 executes various processes in parallel or sequentially.

[0018] The storage device 42 is a recording medium that can be read from and written to by the processing device 41. The storage device 42 includes, for example, non-volatile memory and volatile memory. Non-volatile memory includes, for example, ROM (Read Only Memory), EPROM (Erasable Programmable Read Only Memory), and EEPROM (Electrically Erasable Programmable Read Only Memory). Volatile memory includes, for example, RAM (Random Access Memory). The storage device 42 stores the control program PR4 executed by the processing device 41. The storage device 42 functions as a work area for the processing device 41.

[0019] The imaging device 43 is a camera installed at the entrance of user U[k]'s residence. The primary purpose of the imaging device 43 is to capture images of visitors who come to user U[k]'s residence.

[0020] The first microphone 44 is equipped with a sound-collecting section that collects sound and converts the collected sound into an audio signal for output. The sound-collecting section may have any structure as long as it is capable of collecting sound. For example, a windbreak structure is one such structure. The first microphone 44 is installed at the entrance of user U[k]'s residence. The primary purpose of the first microphone 44 is to collect the voices of visitors who come to user U[k]'s residence.

[0021] The first speaker 45 is equipped with a sound-emitting section that emits sound. The first speaker 45 is installed at the entrance of user U[k]'s residence. The primary purpose of the first speaker 45 is to emit the voices of the residents of user U[k]'s residence, which have been picked up by the second microphone 46, toward visitors.

[0022] The second microphone 46, like the first microphone 44, is equipped with a sound-collecting unit that collects sound and converts the collected sound into an audio signal for output. The second microphone 46 is installed inside the residence of user U[k]. The primary purpose of the second microphone 46 is to collect the voices of the residents of user U[k]'s residence.

[0023] The second speaker 47, like the first speaker 45, is equipped with a sound-emitting section that emits sound. The second speaker 47 is installed inside the residence of user U[k]. The primary purpose of the second speaker 47 is to emit the voice of a visitor, picked up by the first microphone 44, towards the residents of user U[k]'s residence.

[0024] The display device 48 is a device that displays images and character information. The display device 48 displays various images under the control of the processing device 41. For example, various display panels such as liquid crystal display panels and organic EL display panels are preferably used as the display device 48.

[0025] The input device 49 is a device that receives operations from residents of the user U[k]'s residence. For example, the input device 49 may include a touchpad or a touch panel. If the input device 49 includes a touch panel, the input device 49 may also function as a display device 48.

[0026] The alarm device 50 outputs an alarm. The alarm device 50 includes, as an example, a red light and a speaker. The second speaker 47 may also serve as the speaker for the alarm device 50. The alarm device 50 outputs an alarm by, as an example, flashing the red light. Alternatively, as another example, the alarm device 50 outputs an alarm by emitting an alarm sound from the speaker.

[0027] The communication device 51 is hardware acting as a transmitting and receiving device for communicating with other devices. The communication device 51 is also called, for example, a network device, network controller, network card, or communication module. The communication device 51 may be equipped with a connector for wired connection and an interface circuit corresponding to the connector. The communication device 51 may also be equipped with a wireless communication interface. Examples of connectors and interface circuits for wired connection include products compliant with wired LAN, IEEE1394, and USB. Examples of wireless communication interfaces include products compliant with wireless LAN and Bluetooth®.

[0028] The processing unit 41 reads the control program PR4 from the storage device 42, for example. By executing the read control program PR4, the processing unit 41 functions as the communication control unit 411, the voice analysis unit 412, the acquisition unit 413, and the display control unit 414.

[0029] The communication control unit 411 causes the communication device 51 to send and receive various information, data, and signals with other devices.

[0030] The voice analysis unit 412 analyzes the visitor's voice captured by the first microphone 44.

[0031] The acquisition unit 413 acquires the captured image PI of the visitor captured by the imaging device 43. The captured image PI of the visitor acquired by the acquisition unit 413 is transmitted to the image processing device 10 by the communication control unit 411. In addition, the combination of the captured image PI of the visitor and the analysis result of the visitor's voice analyzed by the voice analysis unit 412 is transmitted to the machine learning device 20. For example, if the visitor is a delivery worker, the combination of the captured image PI of the worker and the voice analysis result including the name of the delivery company stated by the worker and the purpose of the worker's visit is transmitted to the machine learning device 20.

[0032] The display control unit 414 causes the display device 48 to display images and text information. As an example, the display control unit 414 causes the display device 48 to display an image of a visitor captured by the imaging device 43.

[0033] Furthermore, the response device 40[k] is composed of an outdoor unit 60[k] and an indoor unit 70[k].

[0034] Figure 3 is an explanatory diagram showing an example configuration of the outdoor unit 60[k]. The outdoor unit 60[k] includes a housing 61. The housing 61 is equipped with an imaging device 43, a first microphone 44, a first speaker 45, and a call button 62.

[0035] Figure 4 is an explanatory diagram showing an example configuration of the indoor unit 70[k]. The indoor unit 70[k] comprises a housing 71. The housing 71 is equipped with a second microphone 46, a second speaker 47, a display device 48, an input device 49, and an alarm device 50. The display device 48 displays the captured image PI of the visitor and icon ICs [1] to [4]. The input device 49 includes, as an example, a response button 49[1], a selection button 49[2], and a confirmation button 49[3].

[0036] A visitor arriving at user U[k]'s residence operates the call button 62 on the outdoor unit 60[k]. As a result, the second speaker 47 of the indoor unit 70[k] emits a call tone. The visitor is also captured by the imaging device 43, and the captured image PI is displayed on the display device 48 of the indoor unit 70[k]. After the resident of user U[k]'s residence operates the response button 49[1] and responds to the visitor using the indoor unit 70[k], the resident's voice is picked up by the second microphone 46 of the indoor unit 70[k]. The resident's voice picked up by the second microphone 46 is emitted from the first speaker 45 of the outdoor unit 60[k]. In this embodiment, it is assumed that the resident's response includes an inquiry about the visitor's attributes and the purpose of the visit.

[0037] When a resident makes an inquiry, the visitor responds by stating their attributes and purpose of visit, and the visitor's voice is picked up by the first microphone 44 of the outdoor unit 60[k]. The visitor's voice picked up by the first microphone 44 is emitted from the second speaker 47 of the indoor unit 70[k]. For example, the resident uses the selection button 49[2] to select one of the icons IC[1] to IC[4] according to the visitor's attributes or purpose of visit indicated by the visitor's response, and confirms using the confirmation button 49[3]. In the example shown in Figure 4, icon IC[1] indicates that the visitor is a delivery worker. Icon IC[2] indicates that the visitor is a salesman. Icon IC[3] indicates that the visitor is a worker who performs maintenance on electricity, gas, and landline telephones. Icon IC[4] indicates that the visitor is a suspicious person. In the example shown in Figure 4, icons IC[1] to IC[4] are shown as icons indicating the attributes of the visitor. However, icons IC[1] to IC[4] may also be icons indicating the purpose of the visitor's visit.

[0038] Furthermore, if the display device 48 is equipped with a touch panel, the user U[k] may select an icon IC using the selection button 49[2] and confirm the selected icon IC using the confirmation button 49[3], or may confirm the icon IC by touching the touch panel on the icon IC.

[0039] As described above, the combination of the visitor's captured image PI and the analysis result of the visitor's voice captured by the first microphone 44 is transmitted from the response device 40[k] to the machine learning device 20. In this case, it is more preferable that the combination of the visitor's captured image PI, the analysis result of the visitor's voice, and the selection result of the icon IC is also transmitted from the response device 40[k] to the machine learning device 20. As will be described later, if the machine learning device 20 is unable to extract the visitor's attributes from the analysis result of the visitor's voice, the machine learning device 20 can use the selection result of the icon IC as a label indicating the visitor's attributes in the training data TD or the purpose of the visitor's visit.

[0040] 1-1-3: Configuration of the Machine Learning Device Figure 5 is a block diagram showing an example configuration of the machine learning device 20. As shown in Figure 5, the machine learning device 20 comprises a processing unit 21, a storage device 22, an input device 23, and a communication device 24. Each element of the machine learning device 20 is interconnected by one or more buses for communicating information.

[0041] The processing unit 21 is a processor that controls the entire machine learning device 20. The processing unit 21 is configured using, for example, one or more chips. The processing unit 21 is also configured using, for example, a central processing unit (CPU) that includes interfaces with peripheral devices, arithmetic units, and registers. Some or all of the functions of the processing unit 21 may be implemented by hardware such as a DSP, ASIC, PLD, and FPGA. The processing unit 21 executes various processes in parallel or sequentially.

[0042] The storage device 22 is a recording medium that can be read from and written to by the processing device 21. The storage device 22 includes, for example, non-volatile memory and volatile memory. The non-volatile memory is, for example, ROM, EPROM, and EEPROM. The volatile memory is, for example, RAM. The storage device 22 stores the control program PR2 executed by the processing device 21. The storage device 22 functions as a work area for the processing device 21.

[0043] Furthermore, the memory device 22 stores the first training data TD1 and the second training data TD2.

[0044] The first teacher data TD1 is teacher data used by a first model determining unit 217 described later in machine learning for determining a first learning model LM1. The first learning model LM1 is a learning model LM that has learned, through machine learning, the relationship between characteristics of past visitors and attributes of past visitors. The first teacher data TD1 is teacher data TD used for said machine learning, and is data in which characteristics of past visitors and attributes of past visitors are paired. The attributes of said past visitors are based on results obtained by analyzing the visitor's voice in the response device 40. Alternatively, the attributes of said past visitors are based on results of selection of an icon IC in the response device 40.

[0045] The second teacher data TD2 is teacher data TD used by a second model determining unit 218 described later in machine learning for determining a second learning model LM2. The second learning model LM2 is a learning model LM that has learned, through machine learning, the relationship between characteristics of past visitors and visit purposes of past visitors. The second teacher data TD2 is teacher data TD used for said machine learning, and is data in which characteristics of past visitors and visit purposes of past visitors are paired. The visit purposes of said past visitors are based on results obtained by analyzing the visitor's voice in the response device 40. Alternatively, the visit purposes of said past visitors are based on results of selection of an icon IC in the response device 40.

[0046] The input device 23 is a device that accepts operations from an administrator of the machine learning device 20. For example, the input device 23 is configured including a keyboard, a touchpad, a touch panel, or a pointing device such as a mouse.

[0047] The communication device 24 is hardware serving as a transmission / reception device for communicating with other devices. The communication device 24 is also called, for example, a network device, a network controller, a network card, or a communication module. The communication device 24 may include a connector for wired connection and may include an interface circuit corresponding to the connector. Further, the communication device 24 may include a wireless communication interface. Examples of the connector and interface circuit for wired connection include products compliant with wired LAN, IEEE1394, and USB. Further, examples of the wireless communication interface include products compliant with wireless LAN, Bluetooth (registered trademark), and the like.

[0048] The processing device 21 functions as a communication control unit 211, a first extraction unit 212, a second extraction unit 213, a third extraction unit 214, a first generation unit 215, a second generation unit 216, a first model determination unit 217, and a second model determination unit 218, for example, by reading and executing the control program PR2 from the storage device 22.

[0049] The communication control unit 211 causes the communication device 24 to transmit and receive various types of information, data, and signals to and from other devices.

[0050] In the present embodiment, the communication control unit 211 causes the communication device 24 to receive, from the response devices 40[1] to 40[n], a set of a captured image PI of a visitor, an analysis result of the visitor's voice, and a selection result of an icon IC. Further, the communication control unit 211 causes the communication device 24 to transmit the first learning model LM1 and the second learning model LM2 to the image processing device 10.

[0051] The first extraction unit 212 extracts features of the visitor from the captured image PI, from among the set of the captured image PI of the visitor, the analysis result of the visitor's voice, and the selection result of the icon IC that are received from the response devices 40[1] to 40[n]. As an example, the features of the visitor are the visitor's clothes and belongings. As an example, the first extraction unit 212 inputs the captured image PI to an unillustrated learning model LM received from a device different from the machine learning device 20, thereby extracting first text information indicating features of the visitor output from the learning model LM.

[0052] The second extraction unit 213 extracts the visitor's attributes from the analysis results of the visitor's voice, among the pairs of the visitor's captured image PI, the analysis results of the visitor's voice, and the selection results of the icon IC received from the response devices 40[1] to 40[n]. As an example, the second extraction unit 213 uses a large language model (LLM) stored in a device different from the machine learning device 20 to extract second text information indicating the visitor's attributes from the analysis results of the visitor's voice. Furthermore, if the second extraction unit 213 is unable to extract the visitor's attributes from the analysis results of the visitor's voice, it determines the visitor's attributes based on the selection results of the icon IC.

[0053] The third extraction unit 214 extracts the purpose of the visitor's visit from the analysis results of the visitor's voice, among the pairs of the visitor's captured image PI, the analysis results of the visitor's voice, and the selection results of the icon IC received from the response devices 40[1] to 40[n]. As an example, the third extraction unit 214 uses a large-scale language model (LLM) stored in a device different from the machine learning device 20 to extract third text information indicating the purpose of the visitor's visit from the analysis results of the visitor's voice. Furthermore, if the third extraction unit 214 is unable to extract the purpose of the visitor's visit from the analysis results of the visitor's voice, it determines the purpose of the visitor's visit based on the selection results of the icon IC.

[0054] The first generation unit 215 generates first training data TD1 by combining the visitor characteristics extracted by the first extraction unit 212 and the visitor attributes extracted by the second extraction unit 213. The first generation unit 215 also stores the generated first training data TD1 in the storage device 22.

[0055] The second generation unit 216 generates second training data TD2 by combining the visitor characteristics extracted by the first extraction unit 212 and the visitor's purpose of visit extracted by the third extraction unit 214. The second generation unit 216 also stores the generated second training data TD2 in the storage device 22.

[0056] The first model determination unit 217 determines the first learning model LM1 by performing machine learning using the first training data TD1. In other words, the first model determination unit 217 determines the first learning model LM1 by performing machine learning on the relationship between the characteristics of past visitors and the attributes of past visitors, using the first training data TD1. As an example, the first model determination unit 217 determines the first learning model LM1 by performing machine learning on the relationship between the characteristics of past visitors and the attributes of past visitors obtained by analyzing the voice of past visitors at the time of their visit, using the first training data TD1. As another example, the first model determination unit 217 determines the first learning model LM1 by performing machine learning on the relationship between the characteristics of past visitors and the attributes of visitors determined based on the icon IC selected by the user U at the time of the past visit, using the first training data TD1.

[0057] The second model determination unit 218 determines the second learning model LM2 by performing machine learning using the second training data TD2. In other words, the second model determination unit 218 determines the second learning model LM2 by performing machine learning on the relationship between the characteristics of past visitors and the purpose of their visits, using the second training data TD2. For example, the second model determination unit 218 determines the second learning model LM2 by performing machine learning on the relationship between the characteristics of past visitors and the purpose of their visits, which is obtained by analyzing the voice of past visitors at the time of their visits, using the second training data TD2. As another example, the second model determination unit 218 determines the second learning model LM2 by performing machine learning on the relationship between the characteristics of past visitors and the purpose of their visits, which is determined based on the icon IC selected by the user U at the time of their visits, using the second training data TD2.

[0058] 1-1-4: Image Processing Device Configuration Diagram 6 is a block diagram showing an example configuration of the image processing device 10. As shown in Figure 6, the image processing device 10 comprises a processing device 11, a storage device 12, an input device 13, and a communication device 14. Each element of the image processing device 10 is interconnected by one or more buses for communicating information.

[0059] The processing unit 11 is a processor that controls the entire image processing unit 10. The processing unit 11 is configured using, for example, one or more chips. The processing unit 11 is also configured using, for example, a central processing unit (CPU) that includes interfaces with peripheral devices, an arithmetic unit, and registers. Some or all of the functions of the processing unit 11 may be implemented by hardware such as a DSP, ASIC, PLD, and FPGA. The processing unit 11 executes various processes in parallel or sequentially.

[0060] The storage device 12 is a recording medium that can be read from and written to by the processing device 11. The storage device 12 includes, for example, non-volatile memory and volatile memory. The non-volatile memory is, for example, ROM, EPROM, and EEPROM. The volatile memory is, for example, RAM. The storage device 12 stores the control program PR1 that the processing device 11 executes. The storage device 12 functions as a work area for the processing device 11.

[0061] Furthermore, the storage device 12 stores the first learning model LM1, the second learning model LM2, the behavior information database ADB, and the history information database HDB.

[0062] The first learning model LM1 is the first learning model LM1 that the image processing device 10 receives from the machine learning device 20.

[0063] The second learning model LM2 is the second learning model LM2 that the image processing device 10 receives from the machine learning device 20.

[0064] The behavioral information database ADB is a database that stores behavioral information AI related to the actions of users U[1] to U[n].

[0065] Figure 8 is an explanatory diagram showing an example of the configuration of the behavioral information database ADB. The behavioral information AI stored in the behavioral information database ADB includes the following items: "User name," "Date and time of activity," "Activity details," "Related businesses," "Purpose of visit," and "Scheduled date and time of visit."

[0066] The "User Name" field indicates which user U's behavioral information AI corresponds to.

[0067] The "Action Date and Time" field indicates the date and time when User U, indicated by the "Username," performed the action described in the "Action Details" section below.

[0068] "Action Details" is an item that shows the details of the actions taken by User U, indicated by "User Name".

[0069] "Related Businesses" is an item that indicates businesses related to the actions of User U, indicated by "Username," as indicated by "Action Details." These "Related Businesses" indicate businesses that visit User U's residence in connection with the actions indicated by "Action Details."

[0070] The "Purpose of Visit" field indicates the purpose for which the company, as indicated by "Related Company," is visiting the residence of User U, as indicated by "User Name."

[0071] In the example shown in Figure 8, the user "U[k]" performed an action with the "action content" of "purchasing product B on site A" at the "action date and time" of "12:23 on January 23, 2025". In connection with this action, the AI ​​action information indicating that a "related company" called "delivery company C" will visit "U[k]'s" residence with the "purpose of visit" being "delivery" is stored in the action information database ADB.

[0072] Each piece of behavioral information AI stored in the behavioral information database ADB is, for example, received by the communication control unit 111 (described later) from a terminal device (not shown) used by each of the users U[1] to U[n], and transmitted to the communication device 14. This terminal device may be, for example, a PC (Personal Computer), a smartphone, or a tablet.

[0073] For example, suppose user U[k] uses their terminal device to purchase product B from site A at 12:23 PM on January 23, 2025. The communication control unit 111 receives information from the terminal device used by user U[k] indicating the purchase itself, the date and time of purchase, and information indicating that delivery company C will visit user U[k]'s residence for the purpose of delivering product B. As a result, the behavioral information AI, the first row of the behavioral information database ADB as illustrated in Figure 8, is stored in the behavioral information database ADB.

[0074] Furthermore, the information that the communication control unit 111 receives from the terminal device used by user U[k] is not limited to the purchase information described above, i.e., information indicating the purchase itself and the date and time of purchase. For example, the communication control unit 111 may receive from the terminal device used by user U[k] information indicating the history of outgoing and incoming calls, and information indicating the schedule of user U[k] recorded in the calendar application of the terminal device. This history information and schedule information, as well as behavioral information AI, is stored in the behavioral information database ADB.

[0075] The history information database HDB is a database that stores history information HI regarding the history of visitors to the residences of users U[1] to U[n].

[0076] Figure 9 is an explanatory diagram showing an example configuration of the history information database HDB. The history information HI stored in the history information database HDB includes the following items: "user name," "date and time of visit," "visitor attributes," and "purpose of visit."

[0077] The "Username" field indicates which user's history information HI corresponds to each history information HI.

[0078] The "Visit Date and Time" field indicates the date and time when a visitor visited the residence of User U, indicated by the "Username".

[0079] "Visitor Attributes" indicates the attributes of visitors who visited the residence of User U, indicated by "Username".

[0080] "Purpose of Visit" indicates the purpose of visit by a visitor who came to the residence of User U, indicated by "Username".

[0081] In the example shown in Figure 9, history information HI is stored in the history information database HDB, indicating that a visitor whose "username" is "U[n]", whose "date and time of visit" is "19:15 on October 23, 2024", whose "visitor attribute" is that of a worker for "J Electric Power Company", and whose "purpose of visit" is "inspection".

[0082] Each of the historical information entries HI stored in the historical information database HDB is generated, for example, based on information that the communication control unit 111 receives from the response devices 40[1] to 40[n] used by each of the users U[1] to U[n] to the communication device 14.

[0083] As an example, the communication control unit 111 causes the communication device 14 to receive a set of the visitor's arrival date and time, the analysis result of the visitor's voice, and the selection result of the icon IC from the response devices 40[1] to 40[n]. In addition, the second and third extraction units (not shown) provided in the processing device 11 extract the visitor's attributes and the purpose of the visit from the analysis result of the visitor's voice, similar to the second and third extraction units 213 and 214 provided in the machine learning device 20. Furthermore, if the second and third extraction units are unable to extract the visitor's attributes and the purpose of the visit from the analysis result of the visitor's voice, they determine the visitor's attributes and the purpose of the visit based on the selection result of the icon IC. Subsequently, a generation unit (not shown) in the processing unit 11 generates history information HI by combining the visitor's date and time received from any of the users U[1] to U[n] and any of the response devices 40[1] to 40[n] corresponding to any of the users U[1] to U[n], and the visitor's attributes and the purpose of the visit, which are extracted or determined by the second and third extraction units (not shown).

[0084] Alternatively, instead of the second and third extraction units (not shown) extracting or determining the visitor's attributes and purpose of visit from the analysis results of the visitor's voice, the communication control unit 111 may receive the visitor's attributes and the purpose of visit stored in the first training data TD1 and second training data TD2 provided in the machine learning device 20 from the machine learning device 20.

[0085] The input device 13 is a device that receives operations from the administrator of the image processing device 10. For example, the input device 13 is configured to include a keyboard, touchpad, touch panel, or pointing device such as a mouse.

[0086] The communication device 14 is hardware that acts as a transmitting and receiving device for communicating with other devices. The communication device 14 is also called, for example, a network device, a network controller, a network card, or a communication module. The communication device 14 may be equipped with a connector for wired connection and an interface circuit corresponding to the connector. The communication device 14 may also be equipped with a wireless communication interface. Examples of connectors and interface circuits for wired connection include products compliant with wired LAN, IEEE1394, and USB. Examples of wireless communication interfaces include products compliant with wireless LAN and Bluetooth®.

[0087] The processing unit 11 functions as a communication control unit 111, acquisition unit 112, extraction unit 113, estimation unit 114, determination unit 115, first control unit 116, second control unit 117, and display control unit 118, for example, by reading and executing the control program PR1 from the storage device 12.

[0088] The communication control unit 111 causes the communication device 14 to send and receive various information, data, and signals with other devices.

[0089] In this embodiment, the communication control unit 111 causes the communication device 14 to receive the first learning model LM1 and the second learning model LM2 from the machine learning device 20.

[0090] The acquisition unit 112 acquires the visitor's captured image PI. Specifically, the acquisition unit 112 acquires the visitor's captured image PI transmitted from the response device 40[k].

[0091] The extraction unit 113 extracts the visitor's features from the captured image PI of the visitor acquired by the acquisition unit 112. These features may include, for example, the visitor's clothing and belongings. The extraction unit 113 also inputs the captured image PI to a learning model LM (not shown), received from a device different from the machine learning device 20, and extracts fourth text information indicating the visitor's features output from the learning model LM. Preferably, the learning model LM is the same learning model LM used by the first extraction unit 212 in the machine learning device 20.

[0092] The estimation unit 114 estimates visitor information, including the visitor's attributes and purpose of visit, based on the visitor's characteristics extracted by the extraction unit 113. More specifically, the estimation unit 114 inputs the visitor's characteristics extracted by the extraction unit 113 into the first learning model LM1, thereby obtaining the visitor's attributes output from the first learning model LM1. The estimation unit 114 uses the visitor's attributes output from the first learning model LM1 as the estimated result of the visitor's attributes. In other words, the estimation unit 114 estimates the visitor's attributes using the first learning model LM1, which has already learned the relationship between past visitor characteristics and past visitor attributes through machine learning. Furthermore, the estimation unit 114 inputs the visitor's characteristics extracted by the extraction unit 113 into the second learning model LM2, thereby obtaining the visitor's purpose of visit output from the second learning model LM2. The estimation unit 114 uses the visitor's purpose of visit, output from the second learning model LM2, as the estimated result of the visitor's purpose of visit. In other words, the estimation unit 114 estimates the visitor's purpose of visit using the second learning model LM2, which has already learned the relationship between the characteristics of past visitors and the purposes of past visitors through machine learning.

[0093] For example, if a visitor is wearing a delivery company uniform and carrying luggage and a cart, the estimation unit 114 estimates that the visitor is a delivery company employee and that the purpose of the visit is to deliver or collect luggage. Another example is if a visitor is wearing a postal worker uniform and carrying luggage and a cart, the estimation unit 114 estimates that the visitor is a postal worker and that the purpose of the visit is to deliver or collect luggage. Yet another example is if a visitor is wearing a police officer uniform and carrying a police ID, the estimation unit 114 estimates that the visitor is a police officer and that the purpose of the visit is to confirm a consultation or report. Yet another example is if a visitor is wearing a suit and carrying catalogs and sample products, the estimation unit 114 estimates that the visitor is a door-to-door salesperson and that the purpose of the visit is to sell goods or services. As another example, if a visitor is dressed simply and is carrying a pamphlet and religious texts, the estimation unit 114 estimates that the visitor is a religious proselytizer and that the purpose of the visit is religious proselytization.

[0094] The decision unit 115 determines the degree of risk for user U[k] in dealing with the above-mentioned visitor. Figure 7 is a functional block diagram of the decision unit 115. As shown in Figure 7, the decision unit 115 comprises a first decision unit 115[1] and a second decision unit 115[2].

[0095] The first decision unit 115[1] determines the degree of risk in responding to a visitor based on behavioral information AI related to the user U[k]'s behavior stored in the behavioral information database ADB and visitor information estimated by the estimation unit 114. Here, the visitor image PI used by the estimation unit 114 to estimate visitor information is an image acquired from the response device 40[k] used by user U[k], so user U[k] and the visitor related to the visitor information estimated by the estimation unit 114 correspond to each other.

[0096] As an example, the first decision unit 115[1] narrows down the behavioral information AIs stored in the behavioral information database ADB to those in which the "user name" is "U[k]". If, in any of the narrowed-down behavioral information AIs, the "related business" and the "purpose of visit" each match the visitor attributes and purpose of visit indicated by the estimated visitor information, the first decision unit 115[1] sets the risk level of response to low. Furthermore, if, in any of the narrowed-down behavioral information AIs, the "related business" and the "purpose of visit" each match the visitor attributes and purpose of visit indicated by the visitor information, the first decision unit 115[1] sets the risk level of response to medium. Furthermore, if, among the narrowed-down behavioral information AI, there is no behavioral information AI that matches at least one of the visitor's attributes and purpose of visit indicated by the visitor information, the first decision unit 115 [1] sets the risk level of response to high. These low, medium, and high risk levels are indicated by numerical values.

[0097] The second decision unit 115[2] determines the level of risk in dealing with a visitor based on the history information HI stored in the history information database HDB and the visitor information estimated by the estimation unit 114. The history information HI is information about the history of past visitors who have visited the residence of user U[k].

[0098] For example, the second decision unit 115[2] narrows down the history information HI stored in the history information database HDB to history information HI in which the "user name" is "U[k]". If the "visitor attributes" in any of the narrowed-down history information HI match the visitor attributes indicated by the estimated visitor information, the second decision unit 115[2] determines that the risk of handling the interaction is low. If there is no history information HI among the narrowed-down history information HI that matches the visitor attributes indicated by the estimated visitor information, the second decision unit 115[2] determines that the risk of handling the interaction is high. These low and high risk levels are indicated by numerical values.

[0099] The determination unit 115 may determine the risk level of user U[k]'s interaction with visitors by adding the risk level determined by the first determination unit 115[1] and the risk level determined by the second determination unit 115[2]. Alternatively, the determination unit 115 may use either the risk level determined by the first determination unit 115[1] or the risk level determined by the second determination unit 115[2] as the risk level of user U[k]'s interaction with visitors.

[0100] The first control unit 116 activates the alarm device 50 provided in the response device 40[k] based on the comparison result between the risk level determined by the determination unit 115 and a pre-set threshold. The risk level determined by the determination unit 115 may be the sum of the risk level determined by the first determination unit 115[1] and the risk level determined by the second determination unit 115[2], as described above. Alternatively, the risk level determined by the determination unit 115 may be either the risk level determined by the first determination unit 115[1] or the risk level determined by the second determination unit 115[2], as described above.

[0101] As an example, the first control unit 116 flashes a red light on the alarm device 50 when the risk level determined by the determination unit 115 exceeds a threshold. In another example, the first control unit 116 emits an alarm sound from the speaker on the alarm device 50 when the risk level determined by the determination unit 115 exceeds a threshold.

[0102] The second control unit 117 silences the ringtone of the response device 40[k] based on the comparison result between the risk level determined by the determination unit 115 and a pre-set threshold. As described above, the response device 40[k] is a device that notifies the arrival of a visitor. Also, as described above, the response device 40[k] is an example of an "intercom".

[0103] For example, the second control unit 117 causes the response device 40[k] to mute the ringtone when the risk level determined by the determination unit 115 exceeds a threshold.

[0104] The display control unit 118 causes the display device 48 in the response device 40[k] to display at least one of the visitor's attributes and the visitor's purpose, based on the visitor information estimated by the estimation unit 114.

[0105] 1-2: The operation diagram 10 of the first embodiment is a flowchart showing an example of the operation of the image processing device 10 according to the first embodiment.

[0106] In step S1, the processing unit 11 of the image processing device 10 functions as an acquisition unit 112. The processing unit 11 acquires the captured image PI of the visitor.

[0107] In step S2, the processing unit 11 functions as an extraction unit 113. The extraction unit 113 extracts the characteristics of the visitor from the captured image PI acquired in step S1.

[0108] In step S3, the processing unit 11 functions as an estimation unit 114. Based on the visitor characteristics extracted in step S2, the processing unit 11 estimates visitor information indicating the visitor's attributes and the purpose of the visit.

[0109] 1-3: Effects of the First Embodiment The image processing apparatus 10 according to this embodiment comprises an acquisition unit 112, an extraction unit 113, and an estimation unit 114. The extraction unit 113 acquires an image PI of a visitor. The extraction unit 113 extracts the visitor's characteristics from the image PI. The estimation unit 114 estimates visitor information indicating the visitor's attributes and the purpose of the visitor's visit based on the visitor's characteristics.

[0110] By having the above configuration, the image processing device 10 can estimate the purpose of a visitor's visit when the visitor uses the intercom to call a resident.

[0111] More specifically, the image processing device 10 extracts visitor characteristics, such as the visitor's clothing and belongings, from the visitor's captured image PI taken by the intercom, and estimates visitor information indicating the visitor's attributes and purpose of visit based on these visitor characteristics. As a result, residents of the residence where the intercom is installed can recognize the estimated visitor's attributes and purpose of visit before responding to the visitor.

[0112] Furthermore, the image processing apparatus 10 according to this embodiment further comprises a first determination unit 115[1]. The first determination unit 115[1] determines the degree of risk in dealing with the visitor based on behavioral information AI relating to the actions of user U[k] corresponding to the visitor and visitor information estimated by the estimation unit 114.

[0113] The image processing device 10, with the above configuration, allows residents of a residence where an intercom is installed to recognize the degree of risk involved in responding to a visitor, in accordance with the resident's actions, even before responding to the visitor.

[0114] Furthermore, the image processing apparatus 10 according to this embodiment further comprises a second determination unit 115[2]. The second determination unit 115[2] determines the degree of risk in dealing with a visitor based on history information HI relating to the history of past visitors and visitor information estimated by the estimation unit 114.

[0115] By having the above configuration, the image processing device 10 allows residents of a residence where an intercom is installed to recognize the degree of risk in responding to a visitor, based on the history of past visitors to the residence, even before responding to the visitor.

[0116] Furthermore, the image processing device 10 according to this embodiment further comprises a first control unit 116. The first control unit 116 activates the alarm device 50 based on the comparison result between the risk level and the threshold.

[0117] By having the above configuration, the image processing device 10 allows residents of a residence where an intercom is installed to recognize the level of risk in dealing with a visitor before responding to the visitor, in cases where the level of risk exceeds a threshold.

[0118] Furthermore, the image processing device 10 according to this embodiment further comprises a second control unit 117. Based on the comparison result between the risk level and the threshold, the second control unit 117 causes the response device 40[k], which acts as an intercom to notify of the arrival of a visitor, to silence the ringtone.

[0119] By having the above configuration, the image processing device 10 allows residents of a residence where an intercom is installed to recognize the level of risk in dealing with a visitor before responding to the visitor, in cases where the level of risk exceeds a threshold.

[0120] Furthermore, the image processing apparatus 10 according to this embodiment further comprises a display control unit 118. Based on the visitor information described above, the display control unit 118 causes at least one of the visitor's attributes and the visitor's purpose to be displayed on the display device 48.

[0121] By having the above configuration, the image processing device 10 can visually see the estimated results of the visitor's attributes and the purpose of their visit before responding to the visitor.

[0122] Furthermore, in the image processing apparatus 10 according to this embodiment, the estimation unit 114 estimates the attributes of a visitor using a first learning model LM1 that has been trained by machine learning to learn the relationship between the characteristics of past visitors and the attributes of past visitors. The estimation unit 114 also estimates the purpose of a visitor's visit using a second learning model LM2 that has been trained by machine learning to learn the relationship between the characteristics of past visitors and the purpose of a past visitor's visit.

[0123] The image processing device 10, with the above configuration, can estimate the attributes of a visitor based on the relationship between the characteristics of past visitors and the attributes of past visitors. Furthermore, the image processing device 10, with the above configuration, can estimate the purpose of a visitor's visit based on the relationship between the characteristics of past visitors and the purpose of their visit.

[0124] Furthermore, the image processing system 1 includes an image processing device 10 and a first model determination unit 217. The first model determination unit 217 determines a first learning model LM1 by machine learning the relationship between the characteristics of past visitors and the attributes of past visitors, using the first training data TD1 as the basis.

[0125] By having the above configuration, the image processing system 1 can determine the first learning model LM1 that the image processing device 10 uses to estimate the attributes of visitors.

[0126] Furthermore, the image processing system 1 includes an image processing device 10 and a second model determination unit 218. The second model determination unit 218 determines a second learning model LM2 by machine learning the relationship between the characteristics of past visitors and the purpose of their visits, using the second training data TD2 as the second training data.

[0127] By having the above configuration, the image processing system 1 can determine a second learning model LM2 that the image processing device 10 uses to estimate the purpose of a visitor's visit.

[0128] 2. Modifications The present disclosure is not limited to the embodiments illustrated above. Specific examples of modifications are given below.

[0129] 2-1: Modification 1 In the above embodiment, the image processing device 10 and the machine learning device 20 were separate devices. However, the image processing system 1 may also include a single device in which the image processing device 10 and the machine learning device 20 are housed in the same enclosure, instead of the image processing device 10 and the machine learning device 20 being separate devices.

[0130] Furthermore, in the above embodiment, the image processing device 10 and the response device 40[k] were separate devices. However, the image processing system 1 may also include a single device in which the image processing device 10 and the response device 40[k] are incorporated into the same housing, instead of the image processing device 10 and the response device 40[k] being separate devices.

[0131] 2-2: Modification 2 In the above embodiment, the alarm device 50 was a component of the response device 40[k]. However, the alarm device 50 may be a separate device from the response device 40[k].

[0132] 2-3: Modification 3 In the above embodiment, the first extraction unit 212 provided in the machine learning device 20 extracted the visitor's features from the visitor's captured image PI. Similarly, the extraction unit 113 provided in the image processing device 10 extracts the visitor's features from the visitor's captured image PI. However, the first extraction unit 212 and the extraction unit 113 may extract feature quantities of the captured image PI from the visitor's captured image PI instead of the visitor's features.

[0133] 2-4: Modification 4 In the above embodiment, the second control unit 117 silenced the ringtone of the response device 40[k] based on the comparison result between the risk level determined by the determination unit 115 and a pre-set threshold. However, the second control unit 117 may silence the ringtone of the response device 40[k] without using the comparison result.

[0134] For example, the storage device 12 stores an NG list, which is a list of attributes of visitors that users U[1] to U[n] do not want to answer. The second control unit 117 may silence the ringing tone of the answering device 40[k] if the attributes of the visitor estimated by the estimation unit 114 match the attributes of a visitor that user U[k] has set as NG in the NG list.

[0135] Alternatively, the storage device 12 stores an NG list, which is a list of visits by users U[1] to U[n] whose purpose of visit is not to be answered. The second control unit 117 may silence the ringtone of the answering device 40[k] if the visitor's purpose of visit estimated by the estimation unit 114 matches a visitor's purpose that user U[k] has set as NG in the NG list.

[0136] For example, the information included in these NG lists is received by the communication control unit 111 from the response devices 40[1] to 40[n] to the communication device 14.

[0137] 2-5: Modification 5 In the above embodiment, the machine learning device 20 includes a first extraction unit 212 to a third extraction unit 214. However, instead of the machine learning device 20, the response device 40[k] may include the first extraction unit 212 to the third extraction unit 214. In this case, the response device 40[k] transmits the visitor's characteristics, visitor's attributes, and the visitor's purpose to the machine learning device 20.

[0138] Similarly, in the above embodiment, the image processing device 10 includes an acquisition unit 112 and an extraction unit 113. However, instead of the image processing device 10, the response device 40[k] may also include the acquisition unit 112 and the extraction unit 113. In this case, the characteristics of the visitor are transmitted from the response device 40[k] to the image processing device 10[k].

[0139] 2-6: Modification 6 In the above embodiment, the display control unit 118 caused the display device 48 provided in the response device 40[k] to display at least one of the visitor's attributes and the visitor's purpose, based on the visitor information estimated by the estimation unit 114.

[0140] However, the display control unit 118 may display at least one of the visitor's attributes and the visitor's purpose on another display device. For example, the display control unit 118 may display at least one of the visitor's attributes and the visitor's purpose on a display device provided on a terminal device used by user U[k]. The terminal device may be a PC, smartphone, or tablet, for example.

[0141] 2-7: Modification 7 In the above embodiment, the display control unit 118 caused the display device 48 provided in the response device 40[k] to display at least one of the visitor's attributes and the visitor's purpose, based on the visitor information estimated by the estimation unit 114.

[0142] However, instead of displaying at least one of the visitor's attributes and the purpose of the visit on the display device 48, an audio recording indicating at least one of the visitor's attributes and the purpose of the visit may be emitted from the second speaker 47. Alternatively, in addition to displaying at least one of the visitor's attributes and the purpose of the visit on the display device 48, an audio recording indicating at least one of the visitor's attributes and the purpose of the visit may be emitted from the second speaker 47.

[0143] 2-8: Modification 8 In the above embodiment, the image processing system 1 comprises an image processing device 10, a machine learning device 20, and response devices 40[1] to 40[n]. However, the image processing system 1 may include further devices.

[0144] Figure 11 is a block diagram showing an example of the overall configuration of the image processing system 1A according to this modified example. The image processing system 1A includes an image processing device 10, a machine learning device 20, response devices 40[1] to 40[n], and an information providing device 30. The information providing device 30 is a device that provides information on suspicious persons in each area.

[0145] In the image processing system 1A, when response devices 40[1] to 40[k] transmit an image PI of a visitor to the image processing device 10, they also transmit the location information of the response devices 40[1] to 40[k] in addition to the image PI.

[0146] Figure 12 is a functional block diagram of the determination unit 115 in this modified example. The determination unit 115 includes a third determination unit 115[3] in addition to the first determination unit 115[1] and the second determination unit 115[2] in the above embodiment.

[0147] The extraction unit 113 in the image processing system 1A extracts the visitor's characteristics from the captured image PI of the responder transmitted from the response device 40[k]. The third determination unit 115[3] may determine the degree of risk for user U[k] in responding to the visitor by comparing the combination of the visitor's characteristics and the location information of the response device 40[k] with suspicious person information for each region provided by the information providing device 30. The suspicious person information for each region provided by the information providing device 30 includes the combination of the suspicious person's characteristics and the location information of that region.

[0148] The determination unit 115 may determine the risk level of user U[k]'s interaction with visitors by summing the risk level determined by the first determination unit 115[1], the risk level determined by the second determination unit 115[2], and the risk level determined by the third determination unit 115[3]. Alternatively, the determination unit 115 may use one of the risk levels determined by the first determination unit 115[1], the risk level determined by the second determination unit 115[2], and the risk level determined by the third determination unit 115[3] as the risk level of user U[k]'s interaction with visitors.

[0149] 3. Other (1) In the embodiments described above, the storage devices 12, 22, and 42 were exemplified as ROM and RAM, but other suitable storage media include flexible disks, magneto-optical disks (e.g., compact disks, digital multipurpose disks, Blu-ray® disks), smart cards, flash memory devices (e.g., cards, sticks, key drives), CD-ROMs (Compact Disc-ROMs), registers, removable disks, hard disks, floppy® disks, magnetic strips, databases, servers, and other appropriate storage media.

[0150] (2) In the embodiments described above, the information, signals, etc. may be represented using any of the various different techniques. For example, the data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be mentioned throughout the above description may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0151] (3) In the embodiments described above, the input and output information may be stored in a specific location (e.g., memory) or managed using a management table. The input and output information may be overwritten, updated, or appended to. The output information may be deleted. The input information may be transmitted to other devices.

[0152] (4) In the embodiments described above, the determination may be made by a value represented using one bit (0 or 1), by a boolean value (true or false), or by a numerical comparison (for example, a comparison with a predetermined value).

[0153] (5) The processing procedures, sequences, flowcharts, etc., exemplified in the embodiments described above may be rearranged in order, as long as there is no contradiction. For example, the methods described in this disclosure present various step elements using an exemplary order and are not limited to the specific order presented.

[0154] (6) Each function illustrated in Figures 1 to 12 is realized by any combination of at least one of hardware and software. Furthermore, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized using one device that is physically or logically coupled, or it may be realized using two or more physically or logically separated devices that are directly or indirectly connected (for example, using wired or wireless connections). A functional block may also be realized by combining the aforementioned one device or the aforementioned multiple devices with software.

[0155] (7) The programs illustrated in the embodiments described above should be broadly interpreted to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, etc., whether they are called software, firmware, middleware, microcode, hardware description languages ​​or by other names.

[0156] Furthermore, software, instructions, information, etc., may be transmitted and received via a transmission medium. For example, if software is transmitted from a website, server, or other remote source using at least one of wired technology (such as coaxial cable, fiber optic cable, twisted pair, or digital subscriber line (DSL)) and wireless technology (such as infrared or microwave), then at least one of these wired and wireless technologies is included in the definition of a transmission medium.

[0157] (8) In each of the above-mentioned forms, the terms “system” and “network” shall be used interchangeably.

[0158] (9) The information, parameters, etc. described in this disclosure may be expressed using absolute values, relative values ​​from a given value, or other corresponding information.

[0159] (10) In the embodiments described above, the responders 40[1] to 40[n], outdoor units 60[1] to 60[n], and indoor units 70[1] to 70[n] may be a mobile station (MS). A mobile station may also be referred to by those skilled in the art as a subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, access terminal, mobile terminal, wireless terminal, remote terminal, handset, user agent, mobile client, client, or several other appropriate terms. In this disclosure, terms such as “mobile station,” “user terminal,” “user equipment (UE),” and “terminal” may be used interchangeably.

[0160] (11) In the embodiments described above, the terms “connected,” “coupled,” or any variation thereof, mean any direct or indirect connection or coupling between two or more elements, and may include the presence of one or more intermediate elements between two elements that are “connected” or “coupled” with each other. The coupling or connection between elements may be a physical coupling or connection, a logical coupling or connection, or a combination thereof. For example, “connection” may be reinterpreted as “access.” As used in this disclosure, two elements may be considered to be “connected” or “coupled” with each other using at least one of one or more wires, cables and printed electrical connections, and, in some non-limiting and non-exclusive examples, electromagnetic energy having wavelengths in the radio frequency domain, microwave domain and optical (both visible and invisible) domain.

[0161] (12) In the embodiments described above, the phrase "based on" does not mean "based solely on" unless otherwise specified. In other words, the phrase "based on" means both "based solely on" and "based at least on".

[0162] (13) The terms “determining” and “determining” as used in this disclosure may encompass a wide variety of actions. “Determining” may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, or inquiring (e.g., searching in a table, database, or other data structure), or ascertaining. “Determining” may also include receiving (e.g., receiving information), transmitting (e.g., sending information), inputting, outputting, or accessing (e.g., accessing data in memory). Furthermore, "judgment" and "decision" can include considering something as having been "judged" or "decided" after resolving, selecting, choosing, establishing, comparing, etc. In other words, "judgment" and "decision" can include considering something as having been "judged" or "decided" after some action. Also, "judgment (decision)" can be reinterpreted as "assuming," "expecting," or "considering."

[0163] (14) In the embodiments described above, where “include,” “including,” and variations thereof are used, these terms are intended to be inclusive, as is the term “comprising.” Furthermore, the term “or” as used in this disclosure is not intended to be exclusive OR.

[0164] (15) In the present disclosure, if articles are added by translation, such as a, an, and the in English, the present disclosure may include the fact that the noun following these articles is plural.

[0165] (16) In this disclosure, the term “A and B are different” may mean “A and B are different from each other.” The term may also mean “A and B are each different from C.” Terms such as “separate” and “combine” may be interpreted in the same way as “different.”

[0166] (17) Each aspect / embodiment described herein may be used individually, in combination, or switched between as needed during implementation. Furthermore, notification of the specified information (e.g., notification that "it is X") is not limited to explicit notification, but may also be implicit (e.g., by not providing such notification).

[0167] Although the present disclosure has been described in detail above, it will be clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the intent and scope of the present disclosure as defined by the claims. Accordingly, the descriptions in the present disclosure are illustrative and not restrictive in any way.

[0168] 1...Image processing system, 1A...Image processing system, 10...Image processing device, 11...Processing device, 12...Storage device, 13...Input device, 14...Communication device, 20...Machine learning device, 21...Processing device, 22...Storage device, 23...Input device, 24...Communication device, 30...Information providing device, 40...Response device, 41...Processing device, 42...Storage device, 43...Imaging device, 44...First microphone, 45...First speaker, 46...First 2 microphones, 47...second speaker, 48...display device, 49...input device, 49[1]...response button, 49[2]...selection button, 49[3]...confirm button, 50...alarm device, 51...communication device, 60...outdoor unit, 61...housing, 62...call button, 70...indoor unit, 71...housing, 111...communication control unit, 112...acquisition unit, 113...extraction unit, 114...estimation unit, 115...determination unit, 115[1]...first determination unit , 115[2]...Second determination unit, 115[3]...Third determination unit, 116...First control unit, 117...Second control unit, 118...Display control unit, 211...Communication control unit, 212...First extraction unit, 213...Second extraction unit, 214...Third extraction unit, 215...First generation unit, 216...Second generation unit, 217...First model determination unit, 218...Second model determination unit, 411...Communication control unit, 412...Voice analysis unit, 413...Acquisition unit, 414... Display control unit, ADB...behavioral information database, AI...behavioral information, HDB...history information database, HI...history information, LM...learning model, LM1...first learning model, LM2...second learning model, NET...communication network, PI...captured image, PR1...control program, PR2...control program, PR4...control program, TD...training data, TD1...first training data, TD2...second training data, U...user

Claims

1. An image processing apparatus comprising: an acquisition unit that acquires an image of a visitor; an extraction unit that extracts the visitor's characteristics from the image; and an estimation unit that estimates visitor information indicating the visitor's attributes and the purpose of the visit based on the visitor's characteristics.

2. The image processing apparatus according to claim 1, further comprising a first determination unit that determines the degree of risk in dealing with the visitor based on behavioral information relating to the actions of the user dealing with the visitor and the visitor information estimated by the estimation unit.

3. The image processing apparatus according to claim 1, further comprising a second determination unit that determines the degree of risk in dealing with the visitor based on historical information relating to the history of past visitors and the visitor information estimated by the estimation unit.

4. The image processing apparatus according to claim 2, further comprising a first control unit that activates an alarm device based on the result of comparing the risk level with a threshold.

5. The image processing apparatus according to claim 2, further comprising a second control unit that silences the ringing tone of the intercom that notifies of the visitor's arrival based on the result of comparing the risk level with a threshold.

6. The image processing apparatus according to claim 1, further comprising a display control unit that causes a display device to display at least one of the visitor's attributes and the visitor's purpose based on the visitor information.

7. The image processing apparatus according to claim 1, wherein the estimation unit estimates the attributes of the visitor using a first learning model which has been trained by machine learning to determine the relationship between the characteristics of past visitors and the attributes of the past visitors, and estimates the purpose of the visitor using a second learning model which has been trained by machine learning to determine the relationship between the characteristics of past visitors and the purpose of the visitor.

8. An image processing system comprising: an image processing device according to claim 7; and a first model determination unit that determines the first learning model by machine learning using the relationship between the characteristics of past visitors and the attributes of past visitors as first training data.

9. An image processing system comprising: an image processing device according to claim 7; and a second model determination unit that determines the second learning model by machine learning using the relationship between the characteristics of past visitors and the purpose of the past visitors' visits as second training data.

Images