Monitoring device, monitoring system, monitoring method, and computer program

Abstract

To provide a server device that monitors a person other than an operator who peeps into contents displayed by an image processing apparatus through the operator's operation, a medium to which a drawn image is output, or a document to be read.SOLUTION: A server device 4 comprises: a network communication circuit 456 that acquires person information indicating persons who are within a predetermined range from an image forming apparatus; and a determination and control unit 447 that determines whether persons other than an operator among the persons specified by the acquired person information have a predetermined relationship with the operator, and when determining that the specified persons include at least one person not having the predetermined relationship, controls to output warning information, and when determining that all the persons have a specific relationship, controls to prevent the output of the warning information.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present disclosure relates to a technique for monitoring whether or not another person is looking at content displayed by an image processing device due to an operation by an operator, a medium on which a drawing is output, or a document to be read. [Background technology]

[0002] In recent years, there has been an increase in shared offices, where multiple users share one office, and free address systems, where employees can freely choose the seat they use to work in an office. In particular, the spread of infection caused by the new coronavirus (COVID-19) has accelerated the movement to minimize company office space, and as a result, it is expected that the use of the above office formats will increase.

[0003] In such an office configuration, it is often the case that a single multifunction peripheral is shared by a plurality of users from a plurality of different companies, or a plurality of users from a plurality of departments within a single company.

[0004] When multiple users from multiple different companies, or multiple users from multiple different departments within a single company share the use of one multifunction device, there are security issues, such as the document or printed matter of the user operating the multifunction device (hereinafter referred to as the operator) being viewed by people from other companies or departments, or the preview image of the document, file name, client company name, etc., displayed on the operation panel of the multifunction device being operated being viewed by people from other companies or departments.

[0005] According to Patent Document 1, in order to detect someone peeking at a mobile phone screen, a face image is detected from an image acquired by shooting, a feature amount is calculated from the detected face image, and it is determined from the calculated feature amount whether it is the face of another person. If it is the face of another person, the feature amount of the other person's face is calculated, and it is determined from the calculated feature amount whether the display screen of the mobile phone is being peeked at. If the display screen is being peeked at, a warning mark is displayed on the display screen. In this way, by the mobile phone displaying the warning mark, the user can know that someone is peeking at the screen and can take action. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2010-231304 A Summary of the Invention [Problem to be solved by the invention]

[0007] In an environment where a multifunction device is installed in a shared office as described above, there are many cases where other users stand near the multifunction device and wait their turn to operate the multifunction device, or other people pass in front of the multifunction device, which may cause a risk of other people peeking at the device. In such an environment, if the technology described in Prior Art Document 1 is applied, a warning will be issued to all people around the multifunction device, even if they are related to the operator and do not pose a security problem. This causes a problem that the displayed warning is annoying to the operator.

[0008] In order to solve the above problems, one aspect of the present disclosure aims to provide a monitoring device, a monitoring system, a monitoring method, and a computer program that can reduce the annoyance caused by warnings for operators of image processing devices. [Means for solving the problem]

[0009] In order to achieve the above object, one aspect of the present disclosure is a method for detecting an image by an image processing device in response to an operation by an operator. On the display Displayed content, drawing output and placed on the output tray. Media, or Placed on the original tray The document to be scanned Or documents placed on the document output tray after scanning A monitoring device for monitoring other people looking into a screen, the monitoring device comprising: an acquisition means for acquiring personal information indicating people present within a predetermined range from the image processing device; and a determination means for making a determination as to whether or not people other than the operator, among the people identified by the acquired personal information, have a predetermined relationship with the operator. A calculation means for calculating a viewpoint position of a person other than the operator; If it is determined that even one of the identified persons does not have the specified relationship, and it is determined that the calculated viewpoint position for the person is on the display surface of the display unit, the paper output tray, or the document tray or the document output tray. In this case, the system controls to output warning information, and if it is determined that all persons have a specific relationship, And, in any case where it is determined that the calculated position of the viewpoint is different from the display surface of the display unit, the paper output tray, the document tray, or the document output tray, and a control means for controlling so as to suppress the output of the warning information.

[0010] Here, the judgment means may obtain the name of a first organization to which the operator belongs and the name of a second organization to which people identified by the personal information, excluding the operator, belong, and may judge whether or not the first organization name and the second organization name match each other, thereby determining whether or not the people have a specified relationship.

[0011] Here, the judgment means may determine that the person does not have a specified relationship if the first organization name and the second organization name do not match, and may determine that the person has a specified relationship if the first organization name and the second organization name match.

[0012] Here, an image of a predetermined range of an area including the image processing device is captured, and the acquisition means acquires a facial image of a person as the personal information from the image obtained by capturing the image, and further includes a storage means for storing an identifier for identifying a user using the image processing device, the facial image of the user, and the name of an organization to which the user belongs in correspondence with each other, and a reading means for acquiring an operator identifier for identifying the operator, reading out from the storage means an organization name corresponding to the operator identifier as a first organization name, searching from the storage means for a facial image that matches a facial image of a person other than the operator among the facial images acquired by the acquisition means, and reading out the organization name corresponding to the facial image as a second organization name, and the judgment means may make the judgment using the first organization name and the second organization name that have been read out.

[0013] Here, a person in the vicinity of the image processing device is wearing an employee ID card showing the name of the organization to which the person belongs with the display side facing outward, and a specified area including the image processing device is photographed, and the acquisition means acquires an employee ID card image showing the display side of the employee ID card worn by the person as the personal information from the image obtained by photographing, and further includes a storage means which stores an identifier identifying a user using the image processing device and the name of the organization to which the user belongs in correspondence with each other, a reading means which acquires an operator identifier identifying the operator and reads out the organization name corresponding to the operator identifier from the storage means as a first organization name, and an extraction means which extracts an organization name as a second organization name from the employee ID cards of people excluding the operator among the employee ID cards acquired by the acquisition means, and the judgment means makes the judgment using the read first organization name and the extracted second organization name.

[0014] Here, a photographing device that photographs a predetermined range of an area including the image processing device may be installed in the vicinity of the image processing device, and the acquisition means may receive the image from the photographing device.

[0015] Here, a person within a specified range of the image processing device has a portable authentication device that communicates via short-range wireless communication and stores an identifier that identifies the person, and the authentication device transmits the identifier via short-range wireless communication, and the acquisition means acquires the identifier as the personal information, and further includes a storage means that stores an identifier that identifies a user who uses the image processing device in correspondence with the name of an organization to which the user belongs, and a reading means that acquires an operator identifier that identifies the operator, reads out from the storage means the organization name corresponding to the operator identifier as a first organization name, and reads out from the storage means the organization name corresponding to the identifiers of persons excluding the operator from the identifiers acquired by the acquisition means as a second organization name, and the judgment means makes the judgment using the read first organization name and second organization name.

[0016] Here, the image processing device may include a communication circuit for performing wireless communication with the authentication device via short-range wireless, and the acquisition means may acquire the identifier from the authentication device via the image processing device.

[0017] Another aspect of the present disclosure is a monitoring device that monitors other people looking at content displayed by an image processing device due to an operator's operation, a medium on which a drawing is output, or a document to be read, and includes an acquisition means for acquiring personal information indicating people present within a predetermined range from the image processing device, a judgment means for making a judgment including whether or not people other than the operator, among the people identified by the acquired personal information, have a predetermined relationship with the operator, and a control means for controlling to output warning information if it is determined that there is even one person among the identified people who does not have the predetermined relationship, and for controlling to suppress the output of the warning information if it is determined that all people have a specific relationship. A storage means for storing the number of times that the operator and the persons specified by the personal information, excluding the operator, held a conference in the same closed space in the past. and, The determination means reads out the number of times from the storage means, compares the read number of times with a predetermined threshold value, and determines whether or not the person has a predetermined relationship. The present invention is characterized by the above.

[0018] Furthermore, another aspect of the present disclosure is a monitoring device that monitors other people looking at the content displayed by an image processing device due to the operation of an operator, a medium on which a drawing is output, or a document to be read, and includes an acquisition means for acquiring personal information indicating people present within a predetermined range from the image processing device, a judgment means for making a judgment including whether or not people other than the operator, among the people identified by the acquired personal information, have a predetermined relationship with the operator, and a control means for controlling to output warning information if it is determined that there is even one person among the identified people who does not have the predetermined relationship, and for controlling to suppress the output of the warning information if it is determined that all people have a specific relationship. a storage means for storing the number of times that the operator and the persons specified by the personal information, excluding the operator, have viewed the same document in the past; and, The determination means reads out the number of times from the storage means, compares the read number of times with a predetermined threshold value, and determines whether or not the person has a predetermined relationship. The present invention is characterized by the above.

[0019] Here, the judgment means may judge that the person has a specified relationship if the number of times is equal to or greater than a specified threshold, and may judge that the person does not have a specified relationship if the number of times is less than the specified threshold.

[0020] Furthermore, another aspect of the present disclosure is a monitoring device that monitors other people looking into the content displayed by an image processing device due to an operator's operation, a medium on which a drawing is output, or a document to be read, and includes an acquisition means for acquiring personal information indicating people present within a predetermined range from the image processing device, a determination means for making a determination including whether or not people other than the operator, among the people identified by the acquired personal information, have a predetermined relationship with the operator, and a control means for controlling to output warning information if it is determined that there is even one person among the identified people who does not have the predetermined relationship, and for controlling to suppress the output of the warning information if it is determined that all people have a specific relationship. When the determining means cannot determine whether or not a person other than the operator among the persons specified by the person information has a predetermined relationship, the determining means notifies the image processing device to request the operator to make such a determination. The present invention is characterized by the above.

[0021] Here, an imaging device that captures an image of a predetermined range of an area including the image processing device is installed in the vicinity of the image processing device, and the acquisition means acquires, from the image obtained by imaging, a person image representing a person other than the operator as the personal information, and if the judgment means cannot recognize a facial image of the person from the acquired person image, it may determine that it is not possible to judge whether the person has a predetermined relationship or not.

[0022] Here, an imaging device that captures an image of a predetermined range of an area including the image processing device is installed in the vicinity of the image processing device, and the acquisition means acquires facial images of people excluding the operator as the personal information from the image obtained by imaging, and further includes a memory means that stores an identifier that identifies a user using the image processing device, the facial image of the user, and the name of the organization to which the user belongs in correspondence with each other, and a search means that searches the memory means for a facial image that matches the facial image acquired by the acquisition means, and the judgment means may decide that it is not possible to determine whether or not a person has a predetermined relationship if the search means cannot find a facial image that matches the acquired facial image from the storage means.

[0023] Here, the image processing device may further comprise a notification unit that, under the control of the control unit, notifies the image processing device of control information indicating whether to output warning information or to suppress output of warning information.

[0024] Here, when the judgment means judges that the person does not have a specified relationship, the notification means may notify the image processing device to warn the person to move away from the person by a certain distance or more.

[0025] Here, when the judgment means judges that the person does not have a specified relationship, the notification means may notify the image processing device to issue a warning to the operator that there is a person who does not have the specified relationship.

[0026] Here, the notification means may notify the image processing device to disable an operation key when the determination means determines that the person does not have a predetermined relationship.

[0027] Furthermore, another aspect of the present disclosure is a monitoring device that monitors other people looking into the content displayed by an image processing device due to an operation by an operator, a medium on which a drawing is output, or a document to be read, and includes an acquisition means for acquiring personal information indicating people present within a predetermined range from the image processing device, a determination means for making a determination including whether or not people other than the operator, among the people specified by the acquired personal information, have a predetermined relationship with the operator, a control means for controlling to output warning information if it is determined that there is even one person among the specified people who does not have the predetermined relationship, and for controlling to suppress output of the warning information if it is determined that all of the specified people have the specified relationship, and a notification means for notifying the image processing device of control information indicating whether to output warning information or suppress output of the warning information under the control of the control means, The image processing device is an image reading device that reads an original and generates image data, or a printing device that produces a printed matter, and the notification means, when it is determined by the determination means that the person does not have a predetermined relationship and when it is detected in the image processing device that the operator has forgotten to take in the original or the printed matter, issues a more emphasized warning to the image processing device than when it is determined that the person has a predetermined relationship. The present invention is characterized by the above.

[0028] Here, when the judgment means judges that the operator does not have a specified relationship, the notification means may notify the image processing device to issue a warning when it detects that the operator has left the image processing device.

[0029] Furthermore, another aspect of the present disclosure is a monitoring device that monitors other people looking into the content displayed by an image processing device due to an operation by an operator, a medium on which a drawing is output, or a document to be read, and includes an acquisition means for acquiring personal information indicating people present within a predetermined range from the image processing device, a determination means for making a determination including whether or not people other than the operator, among the people specified by the acquired personal information, have a predetermined relationship with the operator, a control means for controlling to output warning information if it is determined that there is even one person among the specified people who does not have the predetermined relationship, and for controlling to suppress output of the warning information if it is determined that all of the specified people have the specified relationship, and a notification means for notifying the image processing device of control information indicating whether to output warning information or suppress output of the warning information under the control of the control means, When the determining means determines that the person does not have a predetermined relationship, the notifying means notifies the image processing device to issue a warning when security settings are made in the image processing device. The present invention is characterized by the above.

[0030] Here, the notification means may notify to issue the warning according to the level of the security settings.

[0031] Here, the image processing device is an image reading device that reads a document surface and generates image data, and the notification means may notify the image processing device to issue a warning when the judgment means determines that the person does not have a specified relationship and security information is detected by the image processing device reading the document surface.

[0032] Here, the notification means may notify to issue the warning according to the level of the security information.

[0033] Further, one aspect of the present disclosure is an image processing device and an image processing device that performs image processing due to an operation by an operator. On the display Displayed content, drawing output and placed on the output tray. Media, or Placed on the original tray The document to be scanned Or documents placed on the document output tray after scanning A surveillance system comprising a surveillance device that monitors whether or not other people are peeking into the image processing device, the surveillance device comprising: an acquisition means for acquiring personal information indicating people present within a predetermined range from the image processing device; and a determination means for determining whether or not a person other than the operator, among the people identified by the acquired personal information, has a predetermined relationship with the operator. A calculation means for calculating a viewpoint position of a person other than the operator; If it is determined that even one of the identified persons does not have the specified relationship, and it is determined that the calculated viewpoint position for the person is on the display surface of the display unit, the paper output tray, or the document tray or the document output tray. In this case, the system controls to output warning information, and if it is determined that all persons have a specific relationship, And, in any case where it is determined that the calculated position of the viewpoint is different from the display surface of the display unit, the paper output tray, the document tray, or the document output tray,The image processing device is characterized in that it comprises a control means for controlling to suppress the output of warning information, and a notification means for, under the control of the control means, notifying the image processing device of control information indicating whether to output warning information or suppress the output of warning information, and the image processing device comprises a receiving means for receiving the control information, a control means for controlling to output the warning information when the received control information indicates to output the warning information, and suppressing the output of the warning information when the received control information indicates to suppress the output of the warning information, and an output means for outputting the warning information only when the received control information indicates to output the warning information.

[0034] In addition, one aspect of the present disclosure is a method for detecting an image by an image processing device in response to an operation by an operator. On the display Displayed content, drawing output and placed on the output tray. Media, or Placed on the original tray The document to be scanned Or documents placed on the document output tray after scanning A monitoring method used in a monitoring device that monitors other people's peeking into a camera, the method comprising: acquiring personal information indicating people present within a predetermined range from the image processing device; and judging whether or not any person, other than the operator, among the people identified by the acquired personal information has a predetermined relationship with the operator. A calculation step of calculating a viewpoint position of a person other than the operator; If it is determined that even one of the identified persons does not have the specified relationship, and it is determined that the calculated viewpoint position for the person is on the display surface of the display unit, the paper output tray, or the document tray or the document output tray. In this case, the system controls to output warning information, and if it is determined that all persons have a specific relationship, And, in any case where it is determined that the calculated position of the viewpoint is different from the display surface of the display unit, the paper output tray, the document tray, or the document output tray, and a control step of controlling so as to suppress output of the warning information. Furthermore, another aspect of the present disclosure is a monitoring method used in a monitoring device that monitors other people's peeking at the content displayed by an image processing device due to the operation of an operator, a medium on which a drawing is output, or a manuscript to be read, the monitoring method including an acquisition step of acquiring personal information indicating people present within a predetermined range from the image processing device, a judgment step of judging whether or not any of the people identified by the acquired personal information, excluding the operator, has a predetermined relationship with the operator, and a control step of controlling to output warning information if it is judged that there is even one person among the identified people who does not have the predetermined relationship, and controlling to suppress the output of the warning information if it is judged that all of the people have the predetermined relationship, the monitoring device including a storage means that stores the number of times that the operator and the people identified by the personal information, excluding the operator, have held a meeting in the same closed space in the past, and the judgment step is characterized in that the judgment step reads out the number of times from the storage means, compares the read number of times with a predetermined threshold, and judges whether or not the people have the predetermined relationship. Furthermore, another aspect of the present disclosure is a monitoring method used in a monitoring device that monitors other people's peeking at the content displayed by an image processing device due to the operation of an operator, a medium on which a drawing is output, or a document to be read, the monitoring method including an acquisition step of acquiring personal information indicating people within a predetermined range from the image processing device, a judgment step of judging whether or not any of the people identified by the acquired personal information, excluding the operator, has a predetermined relationship with the operator, and a control step of controlling to output warning information if it is judged that there is even one person among the identified people who does not have the predetermined relationship, and controlling to suppress the output of the warning information if it is judged that all of the identified people have the predetermined relationship, the monitoring device including a storage means that stores the number of times that the operator and the people identified by the personal information, excluding the operator, have looked at the same document in the past, and the judgment step includes reading out the number of times from the storage means, comparing the read number of times with a predetermined threshold, and judging whether or not the people have the predetermined relationship. Furthermore, another aspect of the present disclosure is a monitoring method used in a monitoring device that monitors other people's peeking at the content displayed by an image processing device due to the operation of an operator, a medium on which a drawing is output, or a document to be read, the monitoring method including an acquisition step of acquiring personal information indicating persons present within a predetermined range from the image processing device, a judgment step of judging whether any of the persons identified by the acquired personal information, excluding the operator, have a predetermined relationship with the operator, and a control step of controlling to output warning information if it is judged that there is even one person among the identified persons who does not have the predetermined relationship, and controlling to suppress the output of the warning information if it is judged that all of the persons have the predetermined relationship, characterized in that the judgment step notifies the image processing device to request the operator to make the judgment if it cannot be judged whether any of the persons identified by the personal information, excluding the operator, have the predetermined relationship. Furthermore, another aspect of the present disclosure is a monitoring method used in a monitoring device that monitors other people looking at content displayed by an image processing device due to an operation by an operator, a medium on which a drawing is output, or a document to be read, the monitoring method including an acquisition step of acquiring personal information indicating people present within a predetermined range from the image processing device, a determination step of determining whether or not people other than the operator, among the people identified by the acquired personal information, have a predetermined relationship with the operator, and controlling to output warning information if it is determined that there is even one person among the identified people who does not have the predetermined relationship, and controlling to output the warning information if it is determined that all people have the predetermined relationship. The method includes a control step of controlling the image processing device to suppress output of warning information, and a notification step of notifying the image processing device of control information indicating whether to output warning information or suppress output of the warning information through the control of the control step, wherein the image processing device is an image reading device that reads an original and generates image data, or a printing device that produces printed matter, and the notification step is characterized in that, if the judgment step determines that the person does not have a specified relationship, when the image processing device detects that the operator has forgotten to remove the original or printed matter, the image processing device is notified to issue a more emphasized warning than when the person is determined to have a specified relationship. Furthermore, another aspect of the present disclosure is a monitoring method used in a monitoring device that monitors other people's peeking at the content displayed by an image processing device due to the operation of an operator, a medium on which a drawing is output, or a document to be read, the monitoring method including an acquisition step of acquiring personal information indicating persons present within a predetermined range from the image processing device, a judgment step of judging whether any person identified by the acquired personal information, excluding the operator, has a predetermined relationship with the operator, a control step of controlling to output warning information if it is judged that there is even one person among the identified persons who does not have the predetermined relationship, and controlling to suppress the output of the warning information if it is judged that all of the identified persons have the predetermined relationship, and a notification step of notifying the image processing device of control information indicating whether to output warning information or suppress the output of the warning information through the control of the control step, and the notification step is characterized in that, if it is judged by the judgment step that none of the persons have the predetermined relationship, when security-related settings are made in the image processing device, the notification step notifies the image processing device to issue a warning.

[0035] In addition, one aspect of the present disclosure is a method for detecting an image by an image processing device in response to an operation by an operator. On the display Displayed content, drawing output and placed on the output tray. Media, or Placed on the original tray The document to be scanned Or documents placed on the document output tray after scanning A control computer program used in a monitoring device that monitors other people's peeking into a room, the control computer program being recorded on a computer-readable recording medium, the control computer program including an acquisition step of acquiring from the image processing device personal information indicating people present within a predetermined range, and a determination step of determining whether or not people other than the operator, among the people identified by the acquired personal information, have a predetermined relationship with the operator. A calculation step of calculating a viewpoint position of a person other than the operator; If it is determined that even one of the identified persons does not have the specified relationship, and it is determined that the calculated viewpoint position for the person is on the display surface of the display unit, the paper output tray, or the document tray or the document output tray. In this case, the system controls to output warning information, and if it is determined that all persons have a specific relationship, And, in any case where it is determined that the calculated position of the viewpoint is different from the display surface of the display unit, the paper output tray, the document tray, or the document output tray, and a control step of suppressing the output of warning information. Furthermore, another aspect of the present disclosure is a computer program for control recorded on a computer-readable recording medium, which is used in a monitoring device that monitors other people's peeking at the content displayed by an image processing device due to the operation of an operator, a medium on which a drawing is output, or a manuscript to be read, and causes the monitoring device, which is a computer, to execute an acquisition step of acquiring personal information indicating people within a predetermined range from the image processing device, a judgment step of judging whether or not any of the people identified by the acquired personal information, excluding the operator, has a predetermined relationship with the operator, and a control step of controlling to output warning information if it is judged that there is even one person among the identified people who does not have the predetermined relationship, and controlling to suppress the output of the warning information if it is judged that all of the people have the predetermined relationship, wherein the monitoring device is provided with a storage means that stores the number of times that the operator and the people identified by the personal information, excluding the operator, have held a meeting in the same closed space in the past, and the judgment step is characterized in that the number of times is read from the storage means, the number of times read is compared with a predetermined threshold, and the judgment step is characterized in that the number of times is read from the storage means, the number of times read is compared with a predetermined threshold, and the judgment step is characterized in that the people have the predetermined relationship. Furthermore, another aspect of the present disclosure is a computer program for control recorded on a computer-readable recording medium, which is used in a monitoring device that monitors other people's peeking at the content displayed by an image processing device due to the operation of an operator, a medium on which a drawing is output, or a manuscript to be read, and causes the monitoring device, which is a computer, to execute an acquisition step of acquiring personal information indicating people within a predetermined range from the image processing device, a judgment step of judging whether or not any of the people identified by the acquired personal information, excluding the operator, has a predetermined relationship with the operator, and a control step of controlling to output warning information if it is judged that there is even one person among the identified people who does not have the predetermined relationship, and controlling to suppress the output of the warning information if it is judged that all of the people have the predetermined relationship, wherein the monitoring device is provided with a storage means that stores the number of times that the operator and the people identified by the personal information, excluding the operator, have looked at the same document in the past, and the judgment step is characterized in that the number of times is read from the storage means, the number of times read is compared with a predetermined threshold, and the judgment step is characterized in that the number of times is read from the storage means, the number of times read is compared with a predetermined threshold, and the judgment step is characterized in that the people have the predetermined relationship. Furthermore, another aspect of the present disclosure is a computer program for control recorded on a computer-readable recording medium, which is used in a monitoring device that monitors other people's peeking at the content displayed by an image processing device due to the operation of an operator, the medium on which a drawing is output, or the original document to be read, and causes the monitoring device, which is a computer, to execute an acquisition step of acquiring personal information indicating people within a predetermined range from the image processing device, a judgment step of judging whether or not all of the people identified by the acquired personal information, excluding the operator, have a predetermined relationship with the operator, and a control step of controlling to output warning information if it is judged that there is even one person among the identified people who does not have the predetermined relationship, and controlling to suppress the output of the warning information if it is judged that all of the people have the predetermined relationship, characterized in that the judgment step notifies the image processing device to request the operator to make the judgment if it cannot be judged whether or not all of the people identified by the personal information, excluding the operator, have the predetermined relationship. Furthermore, another aspect of the present disclosure is a computer program for control recorded in a computer-readable recording medium, which is used in a monitoring device that monitors other people's peeking into the content displayed by an image processing device, a medium on which a drawing is output, or a document to be read due to an operation by an operator, and the computer program includes the following steps in the monitoring device, which is a computer: acquiring personal information indicating people present within a predetermined range from the image processing device; judging whether or not people other than the operator, among the people identified by the acquired personal information, have a predetermined relationship with the operator; and, if it is judged that even one person does not have the predetermined relationship among the identified people, controlling the monitoring device to output warning information, and outputting warning information indicating that all people have the predetermined relationship. The method executes a control step of controlling to suppress output of warning information when it is determined that the person is a person having a specified relationship, and a notification step of notifying the image processing device of control information indicating whether to output warning information or suppress the output of warning information through the control of the control step, wherein the image processing device is an image reading device that reads an original and generates image data, or a printing device that produces printed matter, and the notification step is characterized in that, when it is determined by the judgment step that the person does not have a specified relationship, when the image processing device detects that the operator has forgotten to remove the original or printed matter, the image processing device is notified to issue a more emphasized warning than when it is determined that the person has a specified relationship. Furthermore, another aspect of the present disclosure is a computer program for control recorded on a computer-readable recording medium, which is used in a monitoring device that monitors other people's peeking at the content displayed by an image processing device due to the operation of an operator, a medium on which a drawing is output, or a document to be read, and causes the monitoring device, which is a computer, to execute an acquisition step of acquiring personal information indicating persons present within a predetermined range from the image processing device, a judgment step of judging whether any person identified by the acquired personal information, excluding the operator, has a predetermined relationship with the operator, a control step of controlling to output warning information if it is judged that there is even one person among the identified persons who does not have the predetermined relationship, and a control step of controlling to suppress the output of the warning information if it is judged that all of the identified persons have the predetermined relationship, and a notification step of notifying the image processing device of control information indicating whether to output warning information or suppress the output of the warning information through the control of the control step, and the notification step is characterized in that, if it is judged by the judgment step that the person does not have the predetermined relationship, when security-related settings are made in the image processing device, the notification step notifies the image processing device to issue a warning. Effect of the Invention

[0036] According to this aspect, when a person looking into the image processing device being operated by the operator is related to the operator, no warning is issued, which has the effect of reducing the annoyance caused by the warning. [Brief description of the drawings]

[0037] [Figure 1] 1 shows a configuration of a security system 1 as a first embodiment of the present invention. [Diagram 2] 1A is a diagram showing a schematic configuration of an image forming apparatus 5. FIG. 1B is a block diagram showing a configuration of a control circuit 500. FIG. [Diagram 3] 13A shows, as an example, a screen 581 including a warning message displayed on the operation panel 519 of the image forming apparatus 5. 13B shows, as another example, a screen 583 including a warning message displayed on the operation panel 519 of the image forming apparatus 5. [Figure 4] As an example, an image 601 captured by the camera 6 is shown. [Diagram 5] FIG. 2 is a block diagram showing a configuration of a server device 4. [Figure 6] 1 shows an example of the data structure of a user table 410 stored in the memory circuit 460 of the server device 4. [Figure 7] As an example, facial image data 610b extracted from image 601 is shown. [Figure 8] As an example, part data 137a to 137f extracted from face image 610 are shown. [Figure 9] (a) A schematic diagram of a user's head in three orientations is shown. (b) A schematic diagram of a user's head in the horizontal direction is shown. (c) A schematic diagram of a user's head with three inclinations relative to the vertical direction is shown. (d) A schematic diagram of a user's head inclinations relative to the vertical direction is shown. [Figure 10] 1 is a sequence diagram (part 1) showing the operation of the security system 1. FIG. [Figure 11] 12 is a sequence diagram (part 2) showing the operation of the security system 1. Continued from FIG. [Figure 12] 13 is a sequence diagram (part 3) showing the operation of the security system 1. Continued from FIG. [Figure 13] 13 is a flowchart showing an operation of viewpoint processing in the server device 4. [Figure 14] 15A shows an employee ID image 610c representing an employee ID card held by a user as a first modification of the first embodiment, and FIG. [Figure 15] FIG. 11 is a sequence diagram showing an operation in the first modified example. [Figure 16] 2 shows the configuration of a security system 1a according to a second embodiment. [Figure 17] FIG. 11 is a block diagram showing a configuration of a control circuit 500a according to a second embodiment. [Figure 18] FIG. 11 is a block diagram showing a configuration of an identification card 7a according to a second embodiment. [Figure 19] FIG. 11 is a block diagram showing a configuration of a server device 4a according to a second embodiment. [Figure 20] 21 is a sequence diagram (part 1) showing the operation of the security system 1a. Continued in FIG. [Figure 21] FIG. 2 is a sequence diagram (part 2) showing the operation of the security system 1a. [Figure 22] 13 shows the connection between a server device 4 and a camera 9 in the third embodiment. [Diagram 23] 13 shows a configuration of a memory circuit 460 according to a third embodiment. [Figure 24] 13 shows an example of a data structure of a user table 410a in the third embodiment. [Diagram 25] 26 is a sequence diagram (part 1) showing the operation in the embodiment 3. Continued in FIG. [Figure 26] FIG. 13 is a sequence diagram (part 2) showing the operation in the third embodiment. [Figure 27] 10 is a sequence diagram showing an operation in the third embodiment. [Figure 28] 13A shows the connection between the server device 4 and the camera 9a in the fourth embodiment. [Figure 29] 13 shows a configuration of a memory circuit 460 according to a fourth embodiment. [Diagram 30] 13 shows an example of a data structure of a user table 410b in the fourth embodiment. [Diagram 31] 32 is a sequence diagram (part 1) showing the operation in the fourth embodiment. Continued in FIG. [Diagram 32] 33 is a sequence diagram (part 2) showing the operation in the fourth embodiment. Continued in FIG. [Diagram 33] FIG. 13 is a sequence diagram (part 3) showing the operation in the fourth embodiment. [Diagram 34] 6 shows a screen 651 including a warning message displayed on the operation panel 519 of the image forming apparatus 5 in the fifth embodiment. [Diagram 35] FIG. 13 is a sequence diagram showing the operation in the fifth embodiment. [Diagram 36] 13 shows a screen 653 including a warning message displayed on the operation panel 519 of the image forming apparatus 5 in the sixth embodiment. [Figure 37] FIG. 23 is a sequence diagram showing the operation in the sixth embodiment. [Figure 38] FIG. 23 is a sequence diagram showing an operation in the seventh embodiment. [Figure 39] FIG. 23 is a sequence diagram showing an operation in the eighth embodiment. [Diagram 40] FIG. 13 is a sequence diagram showing the operation in the ninth embodiment. [Diagram 41] FIG. 13 is a sequence diagram showing the operation in the ninth embodiment. [Diagram 42] 13 is a flowchart showing an operation in a modified example of the ninth embodiment. [Diagram 43] FIG. 23 is a sequence diagram showing the operation in the tenth embodiment. [Diagram 44] FIG. 23 is a sequence diagram showing the operation in the tenth embodiment. [Diagram 45] FIG. 23 is a sequence diagram showing the operation in a modified example of the tenth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0038] [Embodiment Mode] 1 Example 1 A security system 1 as a first embodiment of the present invention will be described.

[0039] 1.1 Security System 1 As shown in FIG. 1, the security system 1 is made up of an image forming device 5, a camera 6, and a server device 4 (monitoring device).

[0040] The camera 6 (imaging device) is connected to the image forming device 5 via a signal line. The image forming device 5 and the server device 4 are connected to each other via a network 2.

[0041] The image forming apparatus 5 is installed in a shared office that is used jointly by a plurality of users from a plurality of different companies and a plurality of users from a plurality of departments within a single company.

[0042] The image forming device 5 reads an original document and produces a printed copy of the original document, in response to an operation by an operator. The image forming device 5 also receives a print job from an information processing device (not shown; one example is a personal computer) connected to the network 2, and produces a printed copy in accordance with the received print job. The image forming device 5 also reads an original document and generates original image data in response to an operation by an operator, and transmits the generated original image data to another information processing device via the network 2.

[0043] The camera 6 is installed above and behind the image forming device 5 with the optical axis of the lens fixed and facing towards the image forming device 5. The camera 6 captures an image of an operator 10 who operates the image forming device 5 and people 11-13 standing around the operator 10, and generates image data. Since the camera 6 is installed with the optical axis of the lens fixed and facing towards the image forming device 5, a certain range (predetermined range) from the image forming device 5 is captured in the image data. Therefore, the image forming device 5 can capture only people who are within the certain range from the image forming device 5. The camera 6 transmits the generated image data to the server device 4 via the image forming device 5 and the network 2.

[0044] The server device 4 monitors whether or not another person is looking at the contents displayed by the image forming device 5 due to the operation of the operator, the medium on which the image is output, or the document to be read.

[0045] The server device 4 receives the image data generated by the camera 6, and uses the received image data to determine whether a person standing near the operator who appears in the image represented by the image data is looking at a manuscript, a printed matter, an operation panel 519 (described later) provided on the image forming device 5, etc.

[0046] When the image forming device 5 determines from the operator's point of view that a person standing around the operator who is not permitted to peek is peeking, the image forming device 5 outputs a warning to the operator by display or sound, etc., and further outputs a warning to the surrounding people by sound, etc. On the other hand, when the image forming device 5 determines from the operator's point of view that a person standing around the operator who is permitted to peek is peeking, the image forming device 5 does not output a warning.

[0047] In this way, among people standing around the operator, those related to the operator are permitted to peek, but those who are not related to the operator are not permitted to peek.

[0048] For example, a person who belongs to the same department as the operator belongs to is a person (a related person) who has a certain relationship with the operator, and can be allowed to peek in. On the other hand, for example, a person who belongs to a department different from the department to which the operator belongs is not a related person of the operator, and cannot be allowed to peek in. Also, for example, a person who belongs to a company different from the company to which the operator belongs is not a related person of the operator, and cannot be allowed to peek in.

[0049] 1.2 Image forming device 5 As shown in FIGS. 1 and 2(a), the image forming apparatus 5 (image processing apparatus) is a tandem-type color multifunction peripheral (MFP: Multifunction Peripheral) having the functions of a scanner, a printer, and a copier.

[0050] 2(a), the image forming apparatus 5 is configured with a paper feed unit 513 that stores and feeds recording sheets provided at the bottom of the housing. A printer 512 that forms images by electrophotography is provided above the paper feed unit 513. Further above the printer 512, an image reader 511 that reads an original and generates image data, and an operation panel 519 (output means) that displays an operation screen and accepts input operations from an operator are provided.

[0051] Image reader 511 has an automatic document feeder. Automatic document feeder feeds documents set on document tray 517 one by one through a feed path to a predetermined reading position. Image reader 511 reads documents fed to the reading position by automatic document feeder or documents placed on a document glass plate by an operator using a scanner, and obtains image data consisting of multi-value digital signals of red (R), green (G), and blue (B). Image reader 511 feeds documents fed from document tray 517 and that have been read to document discharge tray 518.

[0052] Original tray 517 is provided with an original detection sensor (not shown) that detects that an original is set on original tray 517. Also, original discharge tray 518 is provided with an original detection sensor (not shown) that detects that an original that has been read has been transported and placed on original discharge tray 518.

[0053] The image data of each color component obtained by the image reader 511 undergoes various data processing in the control circuit 500 and is further converted into image data of each reproduction color of yellow (Y), magenta (M), cyan (C), and black (K).

[0054] The printer 512 comprises an intermediate transfer belt stretched between a drive roller, a driven roller and a backup roller, a secondary transfer roller, four image creating units arranged at predetermined intervals along the running direction of the intermediate transfer belt facing the intermediate transfer belt, a fixing unit, a control circuit 500, etc.

[0055] The four image forming units respectively form toner images of the colors Y, M, C, and K. Specifically, each image forming unit is made up of a photosensitive drum which is an image carrier, an LED array for exposing and scanning the surface of the photosensitive drum, an electrostatic charger, a developing unit, a cleaner, and a primary transfer roller.

[0056] The paper feed section 513 is made up of a plurality of paper feed cassettes that accommodate recording sheets of different sizes, and a plurality of pickup rollers for feeding the recording sheets from each paper feed cassette onto a conveying path.

[0057] In each of the multiple image forming units, each photoconductor drum is uniformly charged by a charging charger and exposed to an LED array, forming an electrostatic latent image on the surface of the photoconductor drum. Each electrostatic latent image is developed by a developer of the corresponding color, forming a toner image of Y to K colors on the surface of each photoconductor drum, and the toner images are transferred sequentially onto the surface of the intermediate transfer belt by the electrostatic action of each primary transfer roller disposed on the back side of the intermediate transfer belt.

[0058] The image formation timing of each color is shifted so that the Y, B, and K toner images are transferred in a multi-layer manner onto the intermediate transfer belt.

[0059] On the other hand, a recording sheet is fed from one of the paper feed cassettes in the paper feed unit 513 in accordance with the image forming operations by the multiple image forming units.

[0060] The recording sheet is transported on the transport path to a secondary transfer position where a secondary transfer roller and a backup roller face each other with the intermediate transfer belt in between, and at the secondary transfer position, the Y, K toner images that have been multiply transferred onto the intermediate transfer belt are secondarily transferred onto the recording sheet by the electrostatic action of the secondary transfer roller. The recording sheet onto which the Y, K toner images have been secondarily transferred is then transported to a fixing unit.

[0061] As the toner image on the surface of the recording sheet passes through the fixing nip formed between the heating roller of the fixing section and the pressure roller pressed against it, the toner image is fused and fixed to the surface of the recording sheet by heat and pressure, and after passing through the fixing section, the recording sheet is sent to the printed matter output tray 516.

[0062] The printed matter discharge tray 516 is provided with a printed matter detection sensor (not shown) that detects that a recording sheet (printed matter) sent from the fixing unit is placed on the printed matter discharge tray 516 .

[0063] Operation panel 519 is provided with a display unit consisting of a liquid crystal display panel or the like, which displays the contents set by the operator and various messages. Operation panel 519 accepts from the operator an instruction to start copying, a setting of the number of copies, a setting of copy conditions, a setting of a data output destination, and the like, and notifies control circuit 500 of the accepted contents. Operation panel 519 also includes a speaker 515 (output means) which converts analog audio signals into sound and outputs the sound.

[0064] 1.3 Control circuit 500 As shown in FIG. 2(b), the control circuit 500 is composed of a CPU (Central Processing Unit) 551, a ROM (Read Only Memory) 552, a RAM (Random Access Memory) 553, an image memory 554, an image processing circuit 555, a network communication circuit 556, a scanner control circuit 557, an input / output circuit 558, a printer control circuit 559, a memory circuit 560, an input circuit 561, an output circuit 562, a Box memory circuit 565, an input / output circuit 567, and a bus 566.

[0065] The CPU 551, ROM 552, RAM 553, image memory 554, image processing circuit 555, network communication circuit 556, scanner control circuit 557, input / output circuit 558, printer control circuit 559, input circuit 561, output circuit 562, and input / output circuit 567 are connected to each other via a bus 566. The memory circuit 560 and Box memory circuit 565 are connected to the input / output circuit 567.

[0066] The CPU 551 , the ROM 552 and the RAM 553 constitute a main control unit 541 .

[0067] The RAM 553 temporarily stores various control variables and the number of copies set through the operation panel 519, and also provides a work area when the CPU 551 executes a program.

[0068] A control program for executing various jobs such as a copy operation is stored in the ROM 552. The control program may be stored in the storage circuitry 560.

[0069] The CPU 551 operates according to a control program stored in the ROM 552 .

[0070] Image memory 554 temporarily stores image data contained in a print job or the like.

[0071] The image processing circuit 555 performs various data processing on the image data of the R, G, and B color components obtained by the image reader 511, and converts them into image data of the Y, M, C, and K reproduction colors.

[0072] The network communication circuit 556 receives print jobs from external terminal devices via the network 2. In addition, the network communication circuit 556 (receiving means) receives processing code (0) through processing code (n) (control information) from the server device 4 via the network 2. The processing codes (0) through (n) will be described later. In addition, the network communication circuit 556 transmits emails via the network 2.

[0073] The scanner control circuit 557 controls the image reader 511 to execute the operation of reading the document.

[0074] The storage circuit 560 is composed of a non-volatile semiconductor memory. Alternatively, the storage circuit 560 may be composed of a hard disk.

[0075] The memory circuit 560 stores a user table 531 .

[0076] 2B, the user table 531 is a data table including a plurality of pieces of user information. Each piece of user information corresponds to a user of the image forming apparatus 5.

[0077] Each piece of user information includes a user identifier and a password. The user identifier is an identifier for uniquely identifying a user of the image forming device 5. The password is a password set corresponding to the user identifier. The user identifier and password are used at the time of logging in, as described later.

[0078] The box storage circuit 565 is configured from a non-volatile semiconductor memory. The storage circuit 560 may be configured from a hard disk.

[0079] The Box storage circuit 565 has a Box for storing data for each user. The Box storage circuit 565 may have a Box for storing data for each group made up of multiple users.

[0080] A Box, for example, has an area for storing print jobs.

[0081] The input circuit 561 is connected to the camera 6 via a signal line. The input circuit 561 receives image data from the camera 6.

[0082] The output circuit 562 converts the digital audio signal into an analog audio signal, and outputs the generated analog audio signal to the speaker 515 .

[0083] 1.4 Screen examples As an example, screens 581 and 583 displayed on the operation panel 519 of the image forming apparatus 5 are shown in FIGS. 3(a) and 3(b), respectively.

[0084] Screen 581 includes a message area 582. In message area 582, a warning message for the operator is displayed. Message area 582 includes, for example, the message "Warning: A message is being played for nearby outsiders. [Yes]". This message indicates that an audio warning message is being output to people standing around the operator. The audio warning message is, for example, "Important documents are being handled. Please keep a certain distance away."

[0085] Screen 583 also includes a message area 584. In message area 584, a warning message for the operator is displayed. Message area 584 includes, for example, the message "Warning: Outsider Nearby: Be Careful [Yes]". This message indicates that a person who is not a related party (outsider) is standing near the operator.

[0086] 1.5 Image examples FIG. 4 shows, by way of example, an image 601 represented by image data received from the camera 6 via the input circuitry 561 .

[0087] The image 601 includes person images 610a, 611a, 612a, and 613a each showing a plurality of people 10 to 13 standing around the image forming device 5, and a back image 614a from the upper back of the image forming device 5. Since the shooting direction of the camera 6 is fixed, the back image 614a of the same part of the image forming device 5 is always captured at the same position (center, bottom) in the image 601. Therefore, from the positional relationship between the back image 614a and each of the person images 610a, 611a, 612a, and 613a in the image 601, the standing positions of the people represented by the person images 610a, 611a, 612a, and 613a in the three-dimensional space in which the image forming device 5 exists can be known. In addition, the person image 610a arranged in the center of the image 601 is located in front of the image forming device 5, so it can be known that it is an operator who operates the image forming device 5.

[0088] Furthermore, the image 601 shows face images 610, 611, 612, and 613, which are the face portions of the respective persons, out of person images 610a, 611a, 612a, and 613a. The shape of the face images 610, 611, 612, and 613 is rectangular.

[0089] Here, each user of the image forming apparatus 5 may have an employee ID card hanging from his / her neck or attached to his / her chest to indicate his / her identity. In the image 601, employee ID card images 610c, 611c, 612c, and 613c representing the employee ID cards are displayed on person images 610a, 611a, 612a, and 613a, respectively.

[0090] Although the image 601 represented by the image data received from the camera 6 via the input circuit 561 is a still image, a moving image may also be received from the camera 6 via the input circuit 561.

[0091] 1.6 Main Control Unit 541 As shown in FIG. 2(b), the main control unit 541 includes an overall control unit 542, a judgment unit 543, a message generation unit 544, an alarm sound generation unit 545, and a login control unit 546, as a result of the CPU 551 operating in accordance with the control program.

[0092] (General control unit 542) The CPU 551 operates according to the control program, and the overall control unit 542 comprehensively controls the image memory 554, image processing circuit 555, network communication circuit 556, scanner control circuit 557, input / output circuit 558, printer control circuit 559, input circuit 561, output circuit 562, input / output circuit 567, etc. The overall control unit 542 also comprehensively controls the determination unit 543, message generation unit 544, warning sound generation unit 545, and login control unit 546.

[0093] The central control unit 542 judges whether the received input is login, copy, print, scan to mail, or other. When the received input is login, copy, print, scan to mail, or other, the central control unit 542 controls to execute login processing, copy processing, Box print processing, scan to mail processing, or other processing, respectively.

[0094] For example, the overall control unit 542 receives an operation by an operator from the operation panel 519. If the operation by the operator is an instruction to copy, the overall control unit 542 causes the scanner control circuit 557 to read an original document, and causes the printer control circuit 559 to execute image formation processing using the original image data generated by the reading. If a print job including an instruction to write to a Box is received from the network communication circuit 556, the overall control unit 542 causes the input / output circuit 567 to store the received print job in the Box storage circuit 565. If the operation by the operator is an instruction to execute a print job stored in the Box storage circuit 565, the overall control unit 542 causes the input / output circuit 567 to read the print job from the Box storage circuit 565, and causes the printer control circuit 559 to execute image formation processing according to the read print job. Furthermore, when the operator's operation is a scan to email instruction, the overall control unit 542 causes the scanner control circuit 557 to read the document, generate document image data, and attach the generated document image data to an email and send it through the network communication circuit 556. Furthermore, when the operator's operation is another instruction, the overall control unit 542 causes other processing to be executed.

[0095] When image data is received from the camera 6 by the input circuit 561 , the general control unit 542 controls the network communication circuit 556 to transmit the received image data to the server device 4 via the network 2 .

[0096] (Login control unit 546) The login control unit 546 executes login processing using a user identifier and a password under the control of the overall control unit 542 as described below.

[0097] The login control unit 546 receives a user identifier and a password from the operation panel 519 via the input / output circuit 558 .

[0098] When the login control unit 546 receives the user identifier and password, it refers to the user table 531 and determines whether the pair of the received user identifier and password exists in the user table 531 .

[0099] If the received pair of user identifier and password does not exist in user table 531, login control unit 546 outputs a message indicating that there is an input error to operation panel 519 via input / output circuit 558. Operation panel 519 displays this message. In this case, login has failed.

[0100] If the received pair of user identifier and password exists in user table 531, login control unit 546 temporarily stores the input user identifier. In this case, login is successful. The temporary storage of the input user identifier is valid until logout.

[0101] If the login is successful, the login control unit 546 controls the network communication circuit 556 to transmit the received user identifier to the server device 4 via the network 2 .

[0102] The login method is not limited to the above. For example, login may be performed using image data received from the camera 6. In other words, a person appearing in the center of the image represented by the received image data is considered to be the operator, a facial image is extracted from this person's image, and it is determined whether or not it matches a facial image stored in advance. If they match, login is considered to have been successful, and a user identifier corresponding to the facial image stored in advance may be obtained.

[0103] (Judgment Department 543) When the network communication circuit 556 receives processing code (0) through processing code (n) from the server device 4 via the network 2, the judgment unit 543 (control means) judges whether or not there is even one "1" among the received processing codes (0) through (n). If there is even one "1" among the received processing codes (0) through (n), the judgment unit 543 judges that there is someone in the vicinity of the operator who is not a related person. If all of the received processing codes (0) through (n) are "0", the judgment unit 543 judges that there are only related people in the vicinity of the operator.

[0104] The processing codes (0) through (n) will be described later.

[0105] When it is determined that there is a person who is not a related party near the operator, the judgment unit 543 instructs the message generation unit 544 to generate a warning message indicating that there is a person who is not a related party near the operator, and instructs the warning sound generation unit 545 to generate a warning sound.

[0106] If it is determined that only related persons are present around the operator, the judgment unit 543 instructs the message generation unit 544 to suppress the generation of a warning message, and instructs the warning sound generation unit 545 to suppress the generation of a warning sound.

[0107] (Message generator 544) The message generating unit 544 generates a warning message in response to an instruction from the determining unit 543. The warning message generated by the message generating unit 544 is, for example, as shown in Figs. 3(a) and (b).

[0108] Message generating unit 544 outputs the generated warning message to operation panel 519 via input / output circuit 558. The warning message is displayed on operation panel 519.

[0109] (Warning sound generation unit 545) The warning sound generating unit 545 generates a warning sound in response to an instruction from the determining unit 543. The warning sound generated by the warning sound generating unit 545 is, for example, a digital audio signal that vocalizes "You are handling an important document. Please keep a certain distance away."

[0110] The warning sound generation unit 545 outputs the generated warning sound to the speaker 515 via the output circuit 562. The warning sound is output by the speaker 515 as audio.

[0111] 1.7 Server device 4 As shown in FIG. 5, the server device 4 includes a CPU 451, a ROM 452, a RAM 453, a memory circuit 460, a network communication circuit 456, an input / output circuit 458, an input circuit 459, a bus 461, and the like.

[0112] The CPU 451 , the ROM 452 , the RAM 453 , the network communication circuit 456 , the input / output circuit 458 , and the input circuit 459 are connected to each other via a bus 461 .

[0113] The RAM 453 provides a work area when the CPU 451 executes a program.

[0114] The ROM 452 and the memory circuit 460 store control programs for executing the functions of the server device 4.

[0115] The CPU 451 operates according to a control program stored in the ROM 452 and the storage circuit 460 .

[0116] The CPU 451 , the ROM 452 and the RAM 453 constitute a main control unit 441 .

[0117] As the CPU 451 operates according to the control program, the main control unit 441 constitutes an overall control unit 442, a face recognition processing unit 443, a feature extraction unit 444, a position calculation unit 445, a gaze point calculation unit 446, and a judgment control unit 447.

[0118] The network communication circuit 456 is connected to the image forming apparatus 5 via the network 2. The network communication circuit 456 receives, from the image forming apparatus 5 via the network 2, a user identifier for identifying a user and image data generated by photographing with the camera 6. In addition, the network communication circuit 456 transmits processing code (0) to processing code (n) to the image forming apparatus 5 via the network 2.

[0119] The input circuit 459 is connected to the camera 9 (see FIG. 22) via a signal line. The input circuit 459 is also connected to the camera 9a (see FIG. 28(a)) via another signal line.

[0120] The input / output circuit 458 is connected to the memory circuit 460. The input / output circuit 458 reads data from the memory circuit 460 and writes data to the memory circuit 460. When the network communication circuit 456 receives image data generated by photographing with the camera 6 from the image forming apparatus 5, the input / output circuit 458 writes the received image data to the memory circuit 460 as image data 600.

[0121] (1) Memory circuit 460 The memory circuit 460 (memory means) is composed of, for example, a hard disk.

[0122] The memory circuit 460 stores a control program for executing the functions of the server device 4.

[0123] 5, the storage circuitry 460 stores in advance a user table 410 and feature amount data 421, 422, 423, 424, . . . The feature amount data 421, 422, 423, 424, . . . constitute a feature amount group 420.

[0124] Furthermore, the memory circuit 460 includes an area for storing image data 600. The image data 600 is image data generated by photographing with the camera 6 and received from the image forming device 5.

[0125] (User table 410) The user table 410 is a data table that stores information about the users who use the image forming apparatus 5.

[0126] 6, the user table 410 includes a plurality of pieces of user information 411. Each of the plurality of pieces of user information corresponds to a respective one of a plurality of users who use the image forming apparatus 5.

[0127] Each piece of user information 411 is made up of a user identifier 412 , a company name 413 , and a department name 414 .

[0128] The user identifier 412 is identification information for uniquely identifying a user.

[0129] The company name 413 is the name of the company or the like to which the user belongs.

[0130] The department name 414 is the name of the department or the like to which the user belongs.

[0131] Here, the company name 413 and the department name 414 are collectively called the organization name.

[0132] (Feature group 420) As described above, the feature amount group 420 includes feature amount data 421, 422, 423, 424, . . .

[0133] Each piece of feature data is data including a feature extracted from a face image of a user who uses the image forming apparatus 5. The feature is, for example, the relative positional relationship between two eyes, a nose, and a mouth in the face image.

[0134] Each of the feature amount data 421 , 422 , 423 , 424 , . . . corresponds to a face image of each of a plurality of users who use the image forming apparatus 5 .

[0135] An identifier is assigned to each of the feature amount data 421, 422, 423, 424, .... Each of the identifiers assigned to the feature amount data 421, 422, 423, 424, ... is the same as a user identifier that identifies each of a plurality of users who use the image forming apparatus 5.

[0136] Note that the storage circuitry 460 may store, instead of the feature amount data 421, 422, 423, 424, ..., face images of each of a plurality of users who use the image forming apparatus 5. In this case, the feature amount extraction unit 444 may generate the feature amount data 421, 422, 423, 424, ... from each face image.

[0137] In this way, the memory circuit 460 (storage means) stores an identifier for identifying a user using the image forming device 5, a facial image (feature data) of the user, and the name of the company and department (organization name) to which the user belongs, in association with each other.

[0138] (2) General Control Unit 442 The overall control unit 442 integrally controls the network communication circuit 456, the input / output circuit 458, the input circuit 459, etc. as a result of the CPU 451 operating according to the control program. Also, the overall control unit 442 integrally controls the face recognition processing unit 443, the feature amount extraction unit 444, the position calculation unit 445, the gaze viewpoint calculation unit 446, the judgment control unit 447, etc. as a result of the CPU 451 operating according to the control program.

[0139] When the network communication circuit 456 receives a user identifier for identifying a user from the image forming apparatus 5 via the network 2, the overall control unit 442 reads out user information corresponding to the received user identifier from the user table 410, and extracts the company name and department name (first organization name) from the read user information. The overall control unit 442 temporarily stores the extracted company name and department name.

[0140] In addition, when image data generated by photographing with the camera 6 is received from the image forming device 5 via the network 2 by the network communication circuit 456, the general control unit 442 controls the face recognition processing unit 443, the feature extraction unit 444, the position calculation unit 445, the gaze point calculation unit 446, the judgment control unit 447, etc. to execute their respective processing.

[0141] (3) Face recognition processing unit 443 The face recognition processing unit 443 (acquisition means) reads out image data 600 from the storage circuit 460 under the control of the general control unit 442, and cuts out face image data including a face image showing the face part of the user from an image 601 (FIG. 4) included in the read image data 600. The face image data is cut out, for example, by recognizing a set of eyes, nose, mouth, and ears in the image 601. As an example, face image data consisting of face images 610, 611, 612, and 613 are cut out from the image 601 shown in FIG. 4. Here, FIG. 7 shows, as an example, face image data 610b cut out by the face recognition processing unit 443.

[0142] Here, the face image data is personal information indicating a person who is within a predetermined range from the image forming apparatus 5, and the face recognition processing unit 443 acquires the face image data as personal information.

[0143] The face recognition processing unit 443 also extracts the eyes, nose, mouth, and ears as part data. Furthermore, the face recognition processing unit 443 determines whether each part data is an eye, a nose, a mouth, or an ear, and identifies the part type. Here, FIG. 8 shows part data 137a, 137b, 137c, 137d, 137e, and 137f extracted by the face recognition processing unit 443 as an example.

[0144] The face recognition processing unit 443 writes the face image data and part data cut out from the image data 600 into the memory circuit 460.

[0145] Furthermore, the face recognition processing unit 443 selects a face image located at the center of the image 601 from the cut-out face image data, thereby specifying face image data of the operator who operates the image forming apparatus 5. As an example, the face recognition processing unit 443 specifies a face image 610 located at the center of the image 601 shown in FIG. 4 as the face image of the operator who operates the image forming apparatus 5.

[0146] (4) Position calculation unit 445 The position calculation unit 445 calculates the standing position of the user of the cut-out face image in the three-dimensional space in which the image forming device 5 exists, based on the positional relationship between the position of the face image represented by the face image data cut out by the face recognition processing unit 443 in the image 601 included in the image data 600 and the rear image of the image forming device 5 captured in the image 601. Here, the standing position of the user is indicated by the position of the center part of the face of the user standing around the image forming device 5.

[0147] It is assumed that the position where image forming apparatus 5 is installed in the above-mentioned three-dimensional space is known. Also, it is assumed that the positions of operation panel 519, document tray 517, document ejection tray 518, and printed matter ejection tray 516 are known in the three-dimensional space.

[0148] (5) Line-of-sight viewpoint calculation unit 446 (Calculating the horizontal orientation of the user's face) The gaze point calculation unit 446 calculates the horizontal direction of the user's face for each user as follows.

[0149] 9(a) shows, as an example, a schematic diagram of a user's head in three orientations captured in a two-dimensional image: a user's head 363a facing left, a user's head 363b facing forward, and a user's head 363c facing right.

[0150] For head 363a facing left, a center line 364a is set as a straight line passing through the vertical direction of head 363a through the midpoint between the left and right eyes, a left contour line 366a is set showing the contour of the left side of head 363a, and a right contour line 365a is set showing the contour of the right side of head 363a. Then, center line 364a is located closer to left contour line 366a than to right contour line 365a.

[0151] Furthermore, for head 363b facing forward, if a center line 364b, a left contour line 366b, and a right contour line 365b are set in the same manner as for head 363a, then center line 364b is located approximately in the middle between right contour line 365b and left contour line 366b.

[0152] Furthermore, for head 363c facing to the right, if a center line 364c, a left contour line 366c, and a right contour line 365c are set in the same manner as for head 363a, center line 364c is located closer to right contour line 365c than left contour line 366c.

[0153] In this way, the gaze viewpoint calculation unit 446 can estimate the horizontal direction of the user's face by determining whether the center line of the head is closer to the left contour line or closer to the right contour line.

[0154] As a result, as shown in Fig. 9(b), horizontal directions 362a, 362b, 362c in which the users' heads 361a, 361b, 361c are facing can be estimated. Here, Fig. 9(b) is a conceptual diagram of a case where a plurality of users standing around the image forming device 5 are looked down from above the image forming device 5. Note that this diagram is a conceptual diagram for explaining a method for estimating the horizontal directions of the users' faces, and it should be noted that this diagram is not an image generated by photographing with the camera 6 installed behind and above the image forming device 5.

[0155] Specifically, the gaze viewpoint calculation unit 446 calculates the center line using the left eye part data, the right eye part data, and the face image data. The gaze viewpoint calculation unit 446 also calculates the left contour line and the right contour line using the face image data. Next, the gaze viewpoint calculation unit 446 calculates the ratio of the left distance between the center line and the left contour line and the right distance between the center line and the right contour line using the face image data. For example, when the center line is in the middle between the left contour line and the right contour line, the ratio is 50%. When the center line coincides with the left contour line, the ratio is left 0%. When the center line coincides with the right contour line, the ratio is right 0%. When the ratio of the left distance between the center line and the left contour line and the right distance between the center line and the right contour line is 1:3, the ratio is left 25%. When the ratio of the left distance between the center line and the left contour line and the right distance between the center line and the right contour line is 3:1, the ratio is right 25%.

[0156] In this way, gaze viewpoint calculation section 446 calculates the horizontal direction in which the user's face is facing, based on the calculated ratio.

[0157] (Calculation of the inclination of the user's face from the vertical direction) The gaze point calculation unit 446 calculates the inclination of the user's face (head) from the vertical direction for each user as follows.

[0158] FIG. 9(c) shows a user's face 375a that is not tilted relative to the vertical direction, a user's face 375b that is slightly tilted relative to the vertical direction, and a user's face 375c that is further tilted relative to the vertical direction.

[0159] For user's face 375a, which is not tilted relative to the vertical, center line 374a maintains a nearly vertical direction. For user's face 375b, which is slightly tilted relative to the vertical, center line 374b is slightly tilted from the vertical. For user's face 375c, which is further tilted relative to the vertical, center line 374c is further tilted from the vertical.

[0160] Therefore, by using the center line of the user's head, the inclination of the user's face from the vertical direction can be known. That is, as shown in FIG. 9(d), the gaze directions 373a, 373b can be known from the inclination of center lines 372a, 372b of the user's faces 371a, 371b from the vertical direction. Here, FIG. 9(d) is a conceptual diagram of a case where a plurality of users standing around the image forming device 5 are viewed from the side of the image forming device 5. Note that this diagram is a conceptual diagram for explaining a method of estimating the inclination of the user's face from the vertical direction, and it should be noted that this diagram is not an image generated by photographing with the camera 6 installed behind and above the image forming device 5.

[0161] Specifically, similar to the case of calculating the horizontal direction of the user's face, gaze viewpoint calculation section 446 calculates the center line using the left eye part data, the right eye part data and the face image data.

[0162] Next, the gaze point calculation unit 446 calculates the angle of the calculated center line with respect to the vertical line, and regards this as the inclination of the user's face from the vertical.

[0163] (Generation of gaze information) The gaze viewpoint calculation unit 446 calculates, for each user, gaze information indicating the user's gaze by combining the user's standing position, the calculated horizontal direction of the user's face, and the inclination of the user's face from the vertical direction.

[0164] (Generation of viewpoint information) The gaze viewpoint calculation unit 446 extends a straight line in the direction of the user's gaze indicated by the calculated gaze information, and calculates the point where the extended straight line intersects with the surface of the housing of the image forming device 5 as the user's viewpoint position.

[0165] (6) Feature extraction unit 444 The feature extraction unit 444 extracts features from each face image data cut out by the face recognition processing unit 443. The feature extraction unit 444 extracts, for example, the arrangement relationship between both eyes, nose, and mouth in the face image as the feature. Specifically, the feature is an index that expresses the distance between the left eye and the right eye, the distance between the right eye and the nose, the distance between the left eye and the nose, and the distance between the nose and the mouth included in the face image as a relative length.

[0166] (7) Judgment control unit 447 The judgment control unit 447 (judgment means, control means) judges whether or not the people, excluding the operator, among the people indicated by the acquired personal information are related to the operator, as shown below, and controls the image forming device 5 to output warning information if it is determined that the people are not related, and controls the image forming device 5 to suppress the output of the warning information if it is determined that the people are related.

[0167] The determination control unit 447 has processing code (0) to processing code (i) as array type variables. If the number of users captured in the image 601 is n, the determination control unit 447 uses processing code (0) to processing code (n). Here, both n and i are integers, and n <iである。

[0168] The processing code (0) corresponds to the operator operating the image forming apparatus 5. The processing codes (1) to (n) correspond to users other than the operator operating the image forming apparatus 5.

[0169] The other users include those related to the operator who is operating the image forming apparatus 5 and those who are not related to the operator who is operating the image forming apparatus 5.

[0170] A person related to the operator performing the operation is permitted to look at the document set in the image forming apparatus 5, the printed matter output from the image forming apparatus 5, the operation panel 519, etc. In this case, the judgment control unit 447 sets the corresponding processing code to "0."

[0171] On the other hand, persons who are not related to the operator performing the operation are prohibited from looking at the document set in the image forming apparatus 5, the printed matter output from the image forming apparatus 5, the operation panel 519, etc. In this case, the judgment control unit 447 sets the corresponding processing code to "1."

[0172] Here, among the processing codes (1) to (n), the processing code set to "1" is warning information.

[0173] The determination control unit 447 will be described in detail below.

[0174] The determination control unit 447 first sets the index n of the processing code to an initial value of "0." The determination control unit 447 also sets the initial value "0" to all of the processing codes (0) to (i).

[0175] Next, if the image data 600 does not contain any other facial image than the facial image of the operator operating the image forming apparatus 5, the determination control unit 447 sets the processing code (0) to "0" and controls the network communication circuit 456 to transmit the processing code (0) to the image forming apparatus 5 via the network 2. In this case, this ends the series of processes.

[0176] If the image data 600 contains facial images other than the facial image of the operator operating the image forming apparatus 5, the judgment control unit 447 controls the process to repeat the following steps (a), (b), (c), (c-1), (c-1-1), (c-1-2), and (c-2) for each of the facial images.

[0177] (a) The determination control unit 447 adds "1" to the index n.

[0178] (b) The determination control unit 447 controls the feature amount extraction unit 444 to extract feature amounts from other face images.

[0179] (c) The judgment control unit 447 (reading means) compares the features extracted from other face images with the features contained in all the feature data in the feature group 420 of the memory circuit 460, and judges whether they match.

[0180] (c-1) When it is determined that a matching feature exists, the determination control unit 447 (reading means) acquires an identifier of feature data including the matching feature in the feature group 420, and reads the company name and department name identified by the acquired identifier from the user table 410. The determination control unit 447 compares the read company name and department name (second organization name) with the company name and department name (first organization name) of the operator operating the image forming apparatus 5, and judges whether they match.

[0181] (c-1-1) If it is determined that the company name and department name match, the judgment control unit 447 sets the processing code (n) to “0” and controls the processing code (n) to be sent to the image forming device 5 via the network 2.

[0182] (c-1-2) If it is determined that the company name and department name do not match, the judgment control unit 447 controls to perform viewpoint processing (see the explanation of Figure 13 described later) and controls to transmit processing code (n) to the image forming device 5 via the network 2.

[0183] (c-2) If it is determined that no matching feature exists, the judgment control unit 447 controls to execute viewpoint processing and controls to transmit processing code (n) to the image forming device 5 via the network 2.

[0184] As described above, the determination control unit 447 acquires the name of the first organization to which the operator belongs and the name of the second organization to which the person indicated by the personal information belongs, and determines whether or not the first organization name and the second organization name match. If the first organization name and the second organization name do not match, the determination control unit 447 determines that the person is not a related person, and if the first organization name and the second organization name match, the determination control unit 447 determines that the person is a related person.

[0185] 1.8 Operational Description (1) Operation of security system 1 The operation of the security system 1 will be described with reference to the sequence diagrams shown in FIGS.

[0186] The login control unit 546 executes login by the operator. As a result, the user identifier is accepted (step S101). The login control unit 546 controls the network communication circuit 556 to transmit the user identifier to the server device 4 via the network 2. The overall control unit 442 receives the user identifier from the image forming device 5 via the network 2 and the network communication circuit 456 (step S102).

[0187] The central control unit 442 reads out the user information corresponding to the received user identifier from the user table 410, and extracts the company name and department name from the read out user information (step S105).

[0188] The camera 6 generates image data by photographing (step S103), and outputs the generated image data to the image forming apparatus 5 (step S104). The input circuitry 561 receives the image data from the camera (step S104). The general control unit 542 controls the network communication circuitry 556 to transmit the received image data to the server apparatus 4 via the network 2. The network communication circuitry 456 receives the image data from the image forming apparatus 5 via the network 2, and writes the received image data in the storage circuitry 460 (step S106).

[0189] The face recognition processing unit 443 detects face images showing the face portions of all users from the image data 600 written in the memory circuit 460 (step S107).

[0190] The face recognition processing unit 443 identifies the face image of the operator who operates the image forming apparatus 5 from the detected face images (step S108).

[0191] The determination control unit 447 sets the index n of the processing code to "0" (step S109), and initializes each of the processing code (0) to processing code (i) to "0" (step S110).

[0192] The face recognition processing unit 443 judges whether or not the detected face images include any face image other than the face image of the operator who operates the image forming apparatus 5 (step S131).

[0193] When it is determined that there is no face image other than the face image of the operator operating the image forming apparatus 5 ("None" in step S131), since only the operator operating the image forming apparatus 5 appears in the image data 600, the determination control unit 447 sets the processing code (0) to "0" (step S132) and controls the network communication circuit 456 to transmit the processing code (0) to the image forming apparatus 5 via the network 2 (step S133). In this case, the series of processes in the server device 4 is thereby completed.

[0194] When it is determined that there are facial images other than the facial image of the operator operating the image forming apparatus 5 (YES in step S131), the determination control unit 447 repeats the following steps S135 to S143 for each of the facial images (steps S134 to S144).

[0195] The determination control unit 447 adds "1" to the index n (step S135).

[0196] The feature amount extraction unit 444 extracts feature amounts from the other face images (step S136).

[0197] The determination control unit 447 compares the feature amounts extracted from the other face images with the feature amounts contained in all the feature amount data in the feature amount group 420 of the storage circuit 460, and determines whether they match (step S137).

[0198] If it is determined that a matching feature exists ("YES" in step S138), the judgment control unit 447 acquires an identifier of the feature data including the matching feature in the feature group 420, reads the company name and department name identified by the acquired identifier from the user table 410, and compares the read company name and department name with the company name and department name of the operator operating the image forming device 5 read in step S105 to determine whether there is a match (step S139).

[0199] If it is determined that there is a match ("YES" in step S140), the judgment control unit 447 sets the processing code (n) to "0" (step S141) and controls the processing code (n) to be sent to the image forming device 5 via the network 2 (step S143).

[0200] If it is determined that no matching feature exists ("NO" in step S138), the judgment control unit 447 controls to execute viewpoint processing (step S145) and to transmit processing code (n) to the image forming device 5 via the network 2 (step S143).

[0201] If it is determined that the company name and department name do not match ("NO" in step S140), the judgment control unit 447 controls to perform viewpoint processing (step S142) and to transmit a processing code (n) to the image forming device 5 via the network 2 (step S143).

[0202] When the repetition of steps S135 to S143 ends, the series of processes in the server device 4 ends.

[0203] The network communication circuit 556 receives the processing code (0) through the processing code (n) from the server device 4 via the network 2 (steps S133, S143).

[0204] The determination unit 543 determines whether or not there is at least one "1" among the received processing codes (0) to (n) (step S161).

[0205] If there is at least one "1" among the received processing codes (0) to (n) ("YES" in step S162), message generation unit 544 generates a warning message in response to an instruction from determination unit 543. Message generation unit 544 outputs the generated warning message to operation panel 519 via input / output circuit 558. Operation panel 519 displays the warning message (step S163). Next, warning sound generation unit 545 generates a warning sound in response to an instruction from determination unit 543. Warning sound generation unit 545 outputs the generated warning sound to speaker 515 via output circuit 562. Speaker 515 outputs the warning sound as audio (step S164).

[0206] If there is no "1" among the received processing codes (0) to (n) ("NO" in step S162), the warning message and warning sound are not generated and are not output.

[0207] Operation panel 519 accepts operations by the operator (step S165).

[0208] The main control unit 541 controls the network communication circuit 556, the scanner control circuit 557, the printer control circuit 559, and the like to operate in accordance with the operation of the operator, and executes the corresponding process (step S166).

[0209] Next, the login control unit 546 performs logging out (step S167), and then the main control unit 541 transfers the control to step S101 to continue the process.

[0210] This completes the series of processes in the security system 1.

[0211] (2) Viewpoint processing operations The operation of the viewpoint processing in the server device 4 will be described with reference to the flowchart shown in FIG.

[0212] The operations described below are details of steps S145 and S142 shown in FIG.

[0213] The position calculation unit 445 calculates the standing positions of other people (step S181).

[0214] Next, gaze viewpoint calculation section 446 calculates the horizontal direction of the face of another person (step S182).

[0215] Next, gaze point calculation section 446 calculates the inclination of the other person's face from the vertical direction (step S183).

[0216] Next, the line of sight viewpoint calculation unit 446 calculates the line of sight (step S184).

[0217] Next, line-of-sight viewpoint calculation unit 446 calculates the viewpoint of another person (step S185).

[0218] Next, determination control unit 447 determines whether the calculated position of the viewpoint is any of document tray 517, document discharge tray 518, printed matter discharge tray 516, and the display surface of operation panel 519 of image forming apparatus 5 (step S186).

[0219] If it is determined that the calculated viewpoint position is any one of the document tray 517, document output tray 518, printed matter output tray 516, and the display surface of the operation panel 519 of the image forming device 5 ("document tray, output tray, or operation panel" in step S186), the judgment control unit 447 sets the processing code (n) to "1" (step S188).

[0220] In other cases (step S186: "other"), the determination control unit 447 sets the process code (n) to "0" (step S187).

[0221] This completes the viewpoint processing operation.

[0222] 1.9 Summary As described above, according to the first embodiment, if a person peeking at a document, printed matter, display screen, etc. is related to the operator, no warning is issued, which has the effect of reducing the annoyance caused by the warning for the operator.

[0223] In the above, when it is determined that the calculated position of the viewpoint is any one of document tray 517, document discharge tray 518, printed matter discharge tray 516, and the display surface of operation panel 519 of image forming apparatus 5, determination control unit 447 sets process code (n) to "1." In other words, when the user's viewpoint is on any one of document discharge tray 518, printed matter discharge tray 516, and the display surface of operation panel 519, it is determined that the user is peeking. However, this is not limited to this.

[0224] Users who are within a certain range from image forming apparatus 5, excluding related parties, may be considered to be peeking or to have the possibility of peeking. In other words, it is not necessarily necessary to determine whether the calculated position of the viewpoint is any of document tray 517, document output tray 518, printed matter output tray 516, and the display surface of operation panel 519 of image forming apparatus 5.

[0225] In addition, when the judgment control unit 447 judges that a person is not a related party, the network communication circuit 456 (notification means) may notify the image forming device 5 to warn the person who is not a related party to move away from the person by a certain distance or more.

[0226] In addition, when the judgment control unit 447 judges that the person is not a related party, the network communication circuit 456 (notification means) may notify the image forming apparatus 5 to issue a warning to the operator that a person who is not a related party is present.

[0227] Also, a speaker connected to the network 2 may be installed near the image forming device 5. When the determination control unit 447 determines that a person is not a related party, the network communication circuit 456 may cause the speaker to output a voice to that effect via the network 2 to warn the operator or a person who is not a related party that a person who is not a related party is present.

[0228] In the above-mentioned first embodiment, the judgment unit 543 of the image forming device 5 judges whether or not there is even one "1" among the received processing codes (0) to (n). If there is even one "1" among the received processing codes (0) to (n), a warning message is displayed and a warning sound is output. On the other hand, if there is not even one "1" among the received processing codes (0) to (n), no warning message or warning sound is generated and output.

[0229] However, the present invention is not limited to this configuration.

[0230] Instead of the judgment unit 543 of the image forming apparatus 5, the judgment control unit 447 (control means) of the server apparatus 4 may judge whether or not there is even one code "1" among the processing codes (0) to (n). The judgment control unit 447 may transmit the judgment result (whether or not there is even one code "1") to the image forming apparatus 5 via the network communication circuit 456 (notification means). That is, when the judgment control unit 447 judges that there is even one code "1" among the processing codes (0) to (n), that is, that there is even one person who is not a related person, it may perform control to output warning information. On the other hand, when there is no code "1" among the processing codes (0) to (n), that is, when it judges that all people are related people, it may perform control to suppress output of warning information. The judgment control unit 447 may transmit the judgment result (whether or not there is even one code "1") to the image forming apparatus 5 via the network communication circuit 456 (notification means). The network communication circuit 556 of the image forming apparatus 5 receives the judgment result, and the judgment unit 543 may determine, based on the received judgment result, whether to display a warning message and output a warning sound, or to not generate or output a warning message or warning sound.

[0231] That is, when the received judgment result indicates that there is even one "1", judgment unit 543 performs control to display a warning message and output a warning sound, so that operation panel 519 displays the warning message and speaker 515 outputs the warning sound. On the other hand, when the received judgment result indicates that there is not even one "1", judgment unit 543 performs control to suppress the display of the warning message and the output of the warning sound.

[0232] 2. Variation 1 The first modification of the first embodiment will be described, focusing on the differences between the first embodiment and the first modification.

[0233] Each user of the image forming device 5 may wear an employee ID card, which displays the user's identity, the name of the organization to which the user belongs, etc., by hanging it around the neck or attaching it to the chest with the display side facing outward.

[0234] As shown in FIG. 4, in an image 601, employee ID card images 610c, 611c, 612c, and 613c showing employee ID cards are displayed on person images 610a, 611a, 612a, and 613a, respectively.

[0235] 14(a) shows details of employee ID image 610c. As shown in this figure, employee ID image 610c includes a company name image 631, a department name image 632, a name image 633, and a face image 634. Employee ID images 611c, 612c, and 613c are similar to employee ID image 610c.

[0236] The main control unit 441 further includes an employee ID image recognition unit 449 (not shown) and an OCR (Optical Character Recognition) processing unit 450 (not shown).

[0237] The employee ID card image recognition unit 449 (acquisition means) recognizes and extracts the employee ID card image from the image 601 as follows.

[0238] The employee ID image recognition unit 449 extracts an area enclosed by a quadrilateral frame from the image 601 .

[0239] Next, the employee ID image recognition unit 449 controls the OCR processing unit 450 so as to extract a character string from the extracted area.

[0240] The OCR processing unit 450 (extraction means) performs OCR processing on the extracted area and attempts to extract a character string under the control of the employee ID image recognition unit 449. If the character string is extracted, the extracted character string is output to the employee ID image recognition unit 449. If the character string cannot be extracted, a message to that effect is output to the employee ID image recognition unit 449.

[0241] When receiving a character string from the OCR processing unit 450, the employee ID image recognition unit 449 judges that the area is an employee ID image. Next, the employee ID image recognition unit 449 (extraction means) recognizes the company name and department name from the received character string by estimation. Specifically, if the received character string contains, for example, the character string "Inc.", the employee ID image recognition unit 449 estimates that it is a company name. Also, if the received character string contains, for example, the character string "department" or "section," the employee ID image recognition unit 449 estimates that it is a department name. The employee ID image recognition unit 449 writes the company name and department name (second organization name) obtained by estimation into the memory circuit 460.

[0242] On the other hand, if a message is received from the OCR processing unit 450 indicating that the character string could not be extracted, the employee ID card image recognition unit 449 determines that the area is not an employee ID card image.

[0243] In this way, employee ID image recognition unit 449 recognizes and extracts one or more employee ID images from image 601. Here, the employee ID image is personal information indicating a person who is in the vicinity (within a specified range) of image forming device 5, and employee ID image recognition unit 449 acquires the employee ID image as personal information.

[0244] Furthermore, there are cases where employee ID image recognition section 449 does not recognize any employee ID image at all from image 601. In this case, the process in modification 1 is not performed.

[0245] When the employee ID image recognition unit 449 recognizes and extracts one or more employee ID images from the image 601, it identifies, among the extracted employee ID images, the employee ID image that is located near the center of the image 601 as the employee ID image of the employee ID worn by the operator.

[0246] The determination control unit 447 (reading means) acquires the user identifier acquired by the operator's login, and reads out the company name and department name identified by the acquired user identifier from the user table 410 as the first organization name.

[0247] The judgment control unit 447 compares the second organization name, which is the company name and department name extracted from the employee ID image of an employee other than the operator, with the first organization name, which is the company name and department name read out based on the user identifier obtained by the operator's login, and determines whether they match.

[0248] In addition, the judgment control unit 447 may compare the company name and department name extracted from the employee ID image of an employee other than the operator with the company name and department name extracted from the employee ID image of the operator to determine whether they match.

[0249] (Operation of Modification Example 1) The operation of variant example 1 is similar to that of embodiment 1, and can be explained by replacing the operations shown in FIGS. 10 and 11 among the operations shown in FIGS. 10 to 12 in embodiment 1 with the operations shown in FIGS. 14(b) and 15 which will be described next.

[0250] Therefore, here, the operation of the first modification will be described, focusing on the difference between the first embodiment and the first modification, that is, the operation shown in FIG. 14(b) and FIG.

[0251] The login control unit 546 executes login by the operator. As a result, the user identifier is accepted (step S101). The login control unit 546 controls the network communication circuit 556 to transmit the user identifier to the server device 4 via the network 2. The overall control unit 442 receives the user identifier from the image forming device 5 via the network 2 and the network communication circuit 456 (step S102).

[0252] The central control unit 442 reads out the user information corresponding to the received user identifier from the user table 410, and extracts the company name and department name from the read out user information (step S105).

[0253] The camera 6 generates image data by photographing (step S103), and outputs the generated image data to the image forming apparatus 5 (step S104). The input circuitry 561 receives the image data from the camera (step S104). The general control unit 542 controls the network communication circuitry 556 to transmit the received image data to the server apparatus 4 via the network 2. The network communication circuitry 456 receives the image data from the image forming apparatus 5 via the network 2, and writes the received image data in the storage circuitry 460 (step S106).

[0254] The employee ID image recognition unit 449 detects employee ID images representing the employee ID cards of all users from the image data 600 written in the memory circuit 460 (step S107a).

[0255] The employee ID image recognition unit 449 identifies the employee ID image of the operator who operates the image forming apparatus 5 from the detected employee ID image (step S108a).

[0256] The determination control unit 447 sets the index n of the processing code to "0" (step S109), and initializes each of the processing code (0) to processing code (i) to "0" (step S110).

[0257] The employee ID image recognition unit 449 determines whether or not there is an employee ID image other than the employee ID image of the operator who operates the image forming apparatus 5 among the detected face images (step S231).

[0258] If it is determined that there are no employee ID images other than that of the user operating image forming device 5 ("None" in step S231), then since only the employee ID of the operator operating image forming device 5 appears in image data 600, employee ID image recognition unit 449 sets processing code (0) to "0" (step S232) and controls network communication circuit 456 to transmit processing code (0) to image forming device 5 via network 2 (step S233). In this case, this marks the end of the series of processes in server device 4.

[0259] If it is determined that there are other employee ID images other than the employee ID image of the operator operating the image forming device 5 (Yes in step S231), the employee ID image recognition unit 449 repeats the following steps S235 to S244 for each of the other employee ID images (steps S234 to S245).

[0260] The determination control unit 447 adds "1" to the index n (step S235).

[0261] The determination control unit 447 reads out the company name and the department name (second organization name) from the memory circuit 460 (step S239).

[0262] The judgment control unit 447 compares the company name and department name (second organization name) of the other employee ID image read from the memory circuit 460 with the company name and department name (first organization name) of the operator operating the image forming device 5 read in step S105, and determines whether they match (step S240).

[0263] If it is determined that there is a match ("YES" in step S241), the judgment control unit 447 sets the processing code (n) to "0" (step S242) and controls the processing code (n) to be sent to the image forming device 5 via the network 2 (step S244).

[0264] If it is determined that there is no match ("NO" in step S241), the judgment control unit 447 sets the processing code (n) to "1" (step S243) and controls the processing code (n) to be sent to the image forming device 5 via the network 2 (step S244).

[0265] When the repetition of steps S235 to S244 ends, the series of processes in the server device 4 ends.

[0266] (summary) As described above, by comparing the company name and department name (second organization name) extracted from the employee ID image of an employee other than the operator with the company name and department name (first organization name) of the operator, it is possible to determine whether the user other than the operator is related to the operator or not.

[0267] 3. Example 2 A security system 1a as a second embodiment of the present invention will be described.

[0268] 3.1 Security System 1a As shown in FIG. 16, the security system 1a comprises an image forming apparatus 5a, a server apparatus 4a, and portable identification cards 7a, 7b, 7c, . . .

[0269] The image forming apparatus 5 a and the server apparatus 4 a are connected via a network 2 .

[0270] The image forming apparatus 5a is installed in a shared office, similar to the image forming apparatus 5 in the first embodiment.

[0271] Similar to image forming device 5, image forming device 5a reads an original document and produces a printed copy of the original document, produces the printed copy in accordance with a print job received from another information processing device, and transmits the original document image data generated by reading the original document to the other information processing device.

[0272] An operator 10 who operates the image forming apparatus 5a, and persons 11-12 standing around the image forming apparatus 5a each possess an identification card 7a, 7b, 7c, ... Each of the identification cards 7a, 7b, 7c, ... stores a user identifier that identifies the persons 10-12 who possess the identification card.

[0273] The image forming apparatus 5a and each of the identification cards 7a, 7b, 7c, ... communicate with each other via short-range wireless communication. Each of the identification cards 7a, 7b, 7c, ... transmits a user identifier stored therein to the image forming apparatus 5a via short-range wireless communication.

[0274] The image forming apparatus 5a receives the user identifiers from each of the identification cards 7a, 7b, 7c, ... by short-range wireless communication. The image forming apparatus 5a transmits all of the received user identifiers to the server apparatus 4a via the network 2.

[0275] Here, the short-distance wireless communication is, for example, a wireless communication method defined by the IEEE 802.11 standard. Since the range of radio waves transmitted by short-distance wireless communication is limited to a certain range (predetermined range), only an identification card present within the certain range can transmit a user identifier to the image forming device 5a from the image forming device 5a. Therefore, the image forming device 5a can capture only people present within the certain range from the image forming device 5a.

[0276] The server device 4a receives the user identifier from the image forming device 5a via the network 2.

[0277] The server device 4a compares the user identifier of the operator of the image forming device 5a with the other user identifiers received, and judges whether the user identifier is that of a related person who is acceptable for peeking from the operator's point of view, or that of a non-related person who is not acceptable for peeking.

[0278] If the server device 4a determines that the user identifier is that of a non-related person who is not permitted to peek, the image forming device 5 outputs a warning to the operator by display or audio, etc., and further outputs a warning by audio, etc. to people in the vicinity.

[0279] 3.2 Image forming device 5a The image forming apparatus 5a has a similar configuration to the image forming apparatus 5 of the embodiment 1. Here, the differences from the image forming apparatus 5 will be mainly described.

[0280] The image forming apparatus 5a includes a control circuit 500a shown in FIG. 17, instead of the control circuit 500 included in the image forming apparatus 5 of the first embodiment.

[0281] The control circuit 500a has a similar configuration to the control circuit 500. The differences between the control circuit 500a and the control circuit 500 are as follows.

[0282] The control circuit 500a does not include the input circuit 561 that the control circuit 500 includes.

[0283] The control circuit 500 a also includes a wireless communication circuit 563 and an antenna 564 .

[0284] The antenna 564 transmits and receives radio signals to and from antennas of other devices by short-distance radio communication. The wireless communication circuit 563 performs frequency selection and frequency conversion of the radio signals transmitted and received by the antenna 564.

[0285] Under the control of the general control unit 542, the wireless communication circuit 563 transmits a request to transmit the user identifier via the antenna 564 by short-range wireless communication.

[0286] Furthermore, the wireless communication circuit 563 receives the user identifier from each of the identification cards 7a, 7b, 7c, ... that has received the request to transmit the user identifier by short-range wireless communication via the antenna 564. Upon receiving the user identifier, the wireless communication circuit 563 outputs the received user identifier to the general control unit 542.

[0287] When the overall control unit 542 accepts login from an operator through the login control unit 546, it generates a transmission request for a user identifier. Next, the overall control unit 542 controls the wireless communication circuit 563 to transmit the transmission request for the generated user identifier via the antenna 564 by short-range wireless communication.

[0288] Upon receiving the user identifier from the wireless communication circuit 563, the general control unit 542 controls the network communication circuit 556 to transmit the received user identifier via the network 2 to the server device 4a.

[0289] 3.3 Identification cards 7a, 7b, 7c, ... Here, the identification card 7a (authentication device) will be described as a representative of the identification cards 7a, 7b, 7c, etc. Note that the identification cards 7b, 7c, etc. have the same configuration as the identification card 7a, and therefore descriptions thereof will be omitted.

[0290] As shown in FIG. 18, the identification card 7a comprises a CPU 751, a ROM 752, a RAM 753, a memory circuit 760, a wireless communication circuit 763, an antenna 764, an input / output circuit 567, a bus 768, and the like.

[0291] The CPU 751, the ROM 752, the RAM 753, the wireless communication circuit 763, and the input / output circuit 767 are connected to each other via a bus 768.

[0292] In addition, the input / output circuit 767 is connected to the memory circuit 760 , and the antenna 764 is connected to the wireless communication circuit 763 .

[0293] The RAM 753 provides a work area when the CPU 751 executes a program.

[0294] The ROM 752 stores a control program for executing the operations of the identification card 7a.

[0295] The CPU 751 operates according to a control program stored in the ROM 752 .

[0296] The CPU 751 , the ROM 752 and the RAM 753 constitute the control unit 741 .

[0297] The memory circuit 760 stores in advance a user identifier for identifying a user who possesses the identification card 7a.

[0298] The antenna 764 transmits and receives radio signals by short-distance radio communication with antennas of other devices. The wireless communication circuit 763 performs frequency selection and frequency conversion of the radio signals transmitted and received by the antenna 764.

[0299] The wireless communication circuit 763 receives a request to transmit a user identifier from the image forming apparatus 5a by short-range wireless communication via the antenna 764. Moreover, the wireless communication circuit 763 transmits the user identifier read from the memory circuit 760 by short-range wireless communication via the antenna 764 under the control of the control unit 741.

[0300] When the wireless communication circuit 763 receives a request to send a user identifier, the control unit 741 reads the user identifier from the memory circuit 760 via the input / output circuit 767, and outputs to the wireless communication circuit 763 to transmit the read user identifier.

[0301] 3.4 Server device 4a The server device 4a has a similar configuration to the server device 4 of the embodiment 1. Here, the differences from the server device 4 will be mainly described.

[0302] As shown in FIG. 19, the server device 4a does not include the input circuit 459 that the server device 4 includes.

[0303] Moreover, the storage circuit 460 included in the server device 4a does not store the feature group 420.

[0304] Furthermore, the main control unit 441 included in the server device 4a does not include the face recognition processing unit 443, the feature extraction unit 444, the position calculation unit 445, and the gaze point calculation unit 446 that constitute the main control unit 441 included in the server device 4.

[0305] The network communication circuit 456 (acquisition means) included in the server device 4a receives the user identifier from the image forming device 5a via the network 2. Here, the user identifier is personal information indicating a person who is in the vicinity (within a predetermined range) of the image forming device 5a, and the network communication circuit 456 acquires the user identifier as the personal information.

[0306] Next, the determination control unit 447 constituting the main control unit 441 will be described.

[0307] At the start of processing, the determination control unit 447 sets the index n of the processing code to "0" and initializes each of the processing code (0) to processing code (i) to "0".

[0308] The determination control unit 447 controls the process to repeat the following steps (a), (b), (b-1), (b-2), (b-2-1), (b-2-1-1), and (b-2-1-2) for each received user identifier.

[0309] (a) The determination control unit 447 adds "1" to the index n.

[0310] (b) The determination control unit 447 searches the user table 410 for a user identifier that matches the received user identifier.

[0311] (b-1) If a user identifier matching the received user identifier does not exist in the user table 410, the judgment control unit 447 sets the processing code (n) to “1” and controls the processing code (n) to be sent to the image forming device 5a via the network 2.

[0312] (b-2) If a user identifier matching the received user identifier exists in the user table 410, the judgment control unit 447 (reading means) reads out the company name and department name (second organization name) corresponding to the user identifier from the user table 410.

[0313] (b-2-1) Next, the judgment control unit 447) compares the read-out company name and department name (second organization name) with the company name and department name (first organization name) of the operator operating the image forming apparatus 5a read in step S303 of FIG. 20 described later, and judges whether they match.

[0314] (b-2-1-1) If it is determined that the company name and department name match, the judgment control unit 447 sets the processing code (n) to “0” and controls the processing code (n) to be sent to the image forming device 5a via the network 2.

[0315] (b-2-1-2) If it is determined that the company name and department name do not match, the judgment control unit 447 sets the processing code (n) to “1” and controls the processing code (n) to be sent to the image forming device 5a via the network 2.

[0316] 3.5 Operation of Security System 1a The operation of the second embodiment is similar to that of the first embodiment, and can be explained by replacing the operations shown in FIGS. 10 to 11 among the operations shown in FIGS. 10 to 12 in the first embodiment with the operations shown in FIGS. 20 to 21 which will be described next.

[0317] Therefore, the operation of the second embodiment will be described here with reference to FIGS. 20 and 21, which are the differences between the first and second embodiments.

[0318] The login control unit 546 executes login by the operator. As a result, the user identifier is accepted (step S301). The login control unit 546 controls the network communication circuit 556 to transmit the user identifier to the server device 4a via the network 2. The overall control unit 442 receives the user identifier from the image forming device 5a via the network 2 and the network communication circuit 456 (step S302).

[0319] The central control unit 442 reads out the user information corresponding to the received user identifier from the user table 410, and extracts the company name and department name from the read out user information (step S303). Next, the control proceeds to step S314.

[0320] When the login control unit 546 accepts the login from the operator, the overall control unit 542 generates a transmission request for the user identifier (step S304). Next, the overall control unit 542 controls the wireless communication circuit 563 to transmit the transmission request for the generated user identifier by short-range wireless communication via the antenna 564, and the wireless communication circuit 563 transmits the transmission request (step S305).

[0321] The wireless communication circuit 563 of each identification card receives a request to transmit the user identifier from the image forming apparatus 5a by short-range wireless communication via the antenna 564 (step S305).

[0322] When the wireless communication circuit 563 of each identification card receives a request to transmit the user identifier, the control unit 741 of each identification card reads the user identifier from the memory circuit 760 via the input / output circuit 767 (steps S306, S308, S310) and outputs a command to the wireless communication circuit 763 to transmit the read user identifier. The wireless communication circuit 763 of each identification card transmits the user identifier read from the memory circuit 760 by short-range wireless communication via the antenna 764 (steps S307, S309, S311).

[0323] If the wireless communication circuit 563 does not receive the user identifier ("None" in step S312), no identification card is present near the operator, so the general control unit 542 sets the processing code (0) to "0" (step S313) and then transfers control to step S161 shown in FIG. 12.

[0324] When the overall control unit 542 receives user identifiers from the wireless communication circuit 563 ("Yes" in step S312), it controls the network communication circuit 556 to transmit all of the received user identifiers to the server device 4a via the network 2, and the network communication circuit 556 transmits all of the received user identifiers to the server device 4a via the network 2 (steps S314, S315, S316).

[0325] The network communication circuit 456 included in the server device 4a receives the user identifier from the image forming device 5a via the network 2 (steps S314, S315, S316).

[0326] The determination control unit 447 sets the index n of the processing code to "0" (step S317), and initializes each of the processing codes (0) to (i) to "0" (step S318).

[0327] The determination control unit 447 repeats the following steps S331 to S340 for each received user identifier (steps S330 to S341).

[0328] The determination control unit 447 adds "1" to the index n (step S331).

[0329] The determination control unit 447 searches the user table 410 for a user identifier that matches the received user identifier (step S332).

[0330] If a user identifier matching the received user identifier does not exist in user table 410 ("None" in step S333), judgment control unit 447 sets processing code (n) to "1" (step S334) and controls the transmission of processing code (n) to image forming device 5a via network 2 (step S340).

[0331] If a user identifier matching the received user identifier exists in the user table 410 (YES in step S333), the determination control unit 447 reads out the company name and department name (second organization name) corresponding to the user identifier from the user table 410 (step S335). Next, the determination control unit 447 compares the read out company name and department name (second organization name) with the company name and department name (first organization name) of the operator operating the image forming device 5a read out in step S303, and determines whether they match (step S336).

[0332] If it is determined that the company name and department name match ("YES" in step S337), the judgment control unit 447 sets the processing code (n) to "0" (step S338) and controls the processing code (n) to be sent to the image forming device 5a via the network 2 (step S340).

[0333] If it is determined that the company name and department name do not match ("NO" in step S337), the judgment control unit 447 sets the processing code (n) to "1" (step S339) and controls the processing code (n) to be sent to the image forming device 5a via the network 2 (step S340).

[0334] When the repetition of steps S331 to S340 ends, the series of processes in the server device 4a ends.

[0335] 3.6 Summary As described above, even when a user identifier is obtained from an authentication card held by a user, the company name and department name (second organization name) obtained based on the user identifier obtained from an authentication card other than the operator can be compared with the operator's company name and department name (first organization name) to determine whether the user other than the operator is related to the operator or not.

[0336] 4. Example 3 A third embodiment of the present invention will be described.

[0337] 4.1 System Configuration In the third embodiment, as shown in FIG. 22, a camera 9 is installed on the upper part of the inner wall of a conference room 471 (closed space) so that the optical axis of the lens faces into the conference room 471.

[0338] A plurality of users gather in a conference room 471 to hold a conference, and a camera 9 captures images of the users gathered in the conference room 471 and generates image data. In the image (still image) represented by the generated image data, the users gathered in the conference room 471 are captured together with their respective faces.

[0339] The camera 9 is connected to the server device 4 of the security system 1 of the first embodiment via a signal line, and transmits the generated image data to the server device 4 via the signal line.

[0340] The camera 9 may generate video data by capturing images. Also, multiple cameras may be installed in the conference room 471. Each of the multiple cameras generates image data by capturing images, and transmits the generated image data to the server device 4.

[0341] 4.2 Server device 4 The input circuit 459 of the server device 4 is connected to the camera 9 .

[0342] The input circuit 459 receives image data from the camera 9 .

[0343] The memory circuit 460 of the server device 4 stores the feature amount group 420 as shown in Fig. 5 and Fig. 23. The memory circuit 460 also includes an area for storing the conference room group 430 as shown in Fig. 23. The conference room group 430 stores feature amount data 431, 432, 433, ....

[0344] 6. Moreover, the storage circuit 460 holds a user table 410a shown in FIG. 24, instead of the user table 410 shown in FIG.

[0345] User table 410a has a data structure similar to that of user table 410. User table 410a differs from user table 410 in that user information 411 included in user table 410a further includes related party number 415 in addition to user identifier 412, company name 413, and department name 414. Here, when first user information and second user information in user table 410a include the same related party number, it indicates that the user indicated by the first user information and the user indicated by the second user information are related parties who were captured in one image data captured and generated by camera 9.

[0346] For example, in the user table 410a shown in Fig. 24, three pieces of user information including user identifiers "ID001", "ID003", and "ID004" include the same related party number "010". Therefore, the three users identified by the user identifiers "ID001", "ID003", and "ID004" are related parties.

[0347] The face recognition processing unit 443 detects a plurality of face images from the image represented in the image data received from the camera 9 .

[0348] The feature extraction unit 444 extracts features from each of the detected face images, generates feature data, and writes the generated feature data to the storage circuit 460. When feature extraction from each of all detected face images is completed, the feature extraction unit 444 associates all the generated feature data as a conference room group 430 in the storage circuit 460.

[0349] 23, a conference room group 430 includes feature amount data 431, 432, 433, etc. These feature amount data 431, 432, 433, etc. are extracted from face images of users who have gathered in a conference room 471.

[0350] The determination control unit 447 determines whether or not the feature amount of all the feature data in the conference room group 430 matches the feature amount represented by any of the feature amount data in the feature amount group 420 in the storage circuit 460 .

[0351] If there is a matching feature, the determination control unit 447 associates the feature data in the conference room group 430 with the feature data in the feature group 420 as being the same.

[0352] 23, feature amount data 431 in conference room group 430 is the same as feature amount data 421 in feature group 420. Similarly, feature amount data 432 in conference room group 430 is the same as feature amount data 423 in feature group 420, and feature amount data 433 in conference room group 430 is the same as feature amount data 424 in feature group 420.

[0353] In this way, the feature amount data 421, 423, and 424 in the feature amount group 420 are associated with any of the feature amount data in the conference room group 430 as being the same as each other.

[0354] Next, the determination control unit 447 generates and assigns the same related party number to a plurality of feature amount data in the feature amount group 420 associated with any of the feature amount data in the conference room group 430 .

[0355] As an example, the same party number "010" is generated and assigned to the feature amount data 421, 423, and 424 in the feature amount group 420 shown in FIG.

[0356] Next, if the feature data is identical to any of the feature data in the conference room group 430, the judgment control unit 447 acquires an identifier that identifies the feature data in the associated feature group 420, and reads out user information including a user identifier identical to the acquired identifier from the user table 410a.

[0357] Next, the determination control unit 447 assigns the generated related party number to the read user information, and writes the user information with the assigned related party number back to the user table 410a.

[0358] 24, in the user table 410a, three pieces of user information including user identifiers "ID001", "ID003", and "ID004" include the same related person number "010". Therefore, it can be seen that the three users identified by the user identifiers "ID001", "ID003", and "ID004" are related people who appeared in one image data captured and generated by the camera 9.

[0359] In this manner, in the third embodiment, a plurality of users who are captured in one image data captured and generated by the camera 9 are regarded as related persons.

[0360] 4.3 Operation in the Third Embodiment (1) The operation of assigning a related party number The operation of assigning a related party number will be described with reference to the sequence diagrams shown in FIGS.

[0361] The camera 9 captures images of users gathered in the conference room 471 and generates image data (step S401). The camera 9 transmits the generated image data to the server device 4 via a signal line. The input circuit 459 receives the image data from the camera 9 (step S402).

[0362] The face recognition processing unit 443 detects a plurality of face images from the image represented in the received image data (step S403).

[0363] The feature amount extraction unit 444 repeats the following step S405 for each of the multiple face images (steps S404 to S406).

[0364] The feature amount extraction unit 444 extracts feature amounts from each detected face image, generates feature amount data, and writes the generated feature amount data into the storage circuit 460 (step S405).

[0365] When feature amounts have been extracted from all the detected face images, feature amount extracting section 444 associates all the generated feature amount data as conference room group 430 in memory circuit 460 (step S407).

[0366] The determination control unit 447 repeats steps S412 to S414 for all feature amount data in the conference room group 430 (steps S411 to S415).

[0367] The determination control unit 447 determines whether or not the feature amount of each piece of feature amount data in the conference room group 430 matches the feature amount represented by any of the feature amount data in the feature amount group 420 of the memory circuit 460 (step S412).

[0368] If a matching feature exists ("YES" in step S413), the determination control unit 447 associates the feature data in the conference room group 430 with the feature data in the feature group 420 (step S414).

[0369] When the repetition of steps S412 to S414 has been completed for all feature data in conference room group 430, next, judgment control unit 447 generates and assigns the same party number to multiple feature data in feature group 420 that are associated with any of the feature data in conference room group 430 (step S416).

[0370] Next, the judgment control unit 447 acquires an identifier that identifies the feature data in the feature group 420 associated with any of the feature data in the conference room group 430, reads user information including a user identifier identical to the acquired identifier from the user table 410a, assigns the generated related party number to the read user information, and writes the user information with the related party number assigned back to the user table 410a (step S417).

[0371] This completes the operation of generating and assigning a related party number.

[0372] (2) Actions to determine whether or not someone is a related party The operation of determining whether or not a person is a related party in the third embodiment is similar to the operation of determining whether or not a person is a related party in the first embodiment, and can be explained by replacing the operation shown in FIG. 11, among the operations shown in FIGS. 10 to 12 in the first embodiment, with the operation shown in FIG. 27 which will be described next.

[0373] Therefore, the operation of determining whether or not a person is a related person in the third embodiment will be described here with reference to a sequence diagram shown in FIG.

[0374] The general control unit 442 reads out user information including the user identifier of the operator from the user table 410a, and extracts the related party number from the read out user information (step S130).

[0375] The face recognition processing unit 443 determines whether or not any face image other than the face image of the operator operating the image forming apparatus 5 is present among the detected face images in the image data received from the camera 6 (step S131).

[0376] When it is determined that there is no face image other than the face image of the operator operating the image forming apparatus 5 ("None" in step S131), since only the operator operating the image forming apparatus 5 appears in the image data 600, the determination control unit 447 sets "0" to the processing code (0) (step S132) and controls the network communication circuit 456 to transmit the processing code (0) to the image forming apparatus 5 via the network 2 (step S133). This ends the series of processes in the server device 4.

[0377] When it is determined that there are facial images other than the facial image of the operator operating the image forming apparatus 5 (YES in step S131), the determination control unit 447 repeats the following steps S135 to S143 for each of the facial images (steps S134 to S144).

[0378] The determination control unit 447 adds "1" to the index n (step S135).

[0379] The feature amount extraction unit 444 extracts feature amounts from the other face images (step S136).

[0380] The determination control unit 447 compares the feature amounts extracted from the other face images with the feature amounts contained in all the feature amount data in the feature amount group 420 of the storage circuit 460, and determines whether they match (step S137).

[0381] If it is determined that a matching feature exists ("YES" in step S138), the judgment control unit 447 acquires an identifier of the feature data including the matching feature in the feature group 420, reads the related party number identified by the acquired identifier from the user table 410a, and compares the read related party number with the related party number acquired in step S130 to determine whether they match (step S139a).

[0382] If it is determined that the related party numbers match ("YES" in step S140a), the judgment control unit 447 sets the processing code (n) to "0" (step S141) and controls the processing code (n) to be sent to the image forming device 5 via the network 2 (step S143).

[0383] If it is determined that no matching feature exists ("NO" in step S138), the judgment control unit 447 sets the processing code (n) to "1" (step S145a) and controls the processing code (n) to be transmitted to the image forming device 5 via the network 2 (step S143).

[0384] If it is determined that the related party numbers do not match ("NO" in step S140a), the judgment control unit 447 sets the processing code (n) to "1" (step S142a) and controls the processing code (n) to be sent to the image forming device 5 via the network 2 (step S143).

[0385] When the repetition of steps S135 to S143 ends, the series of processes in the server device 4 ends.

[0386] 4.4 Summary When multiple users gather in a conference room to hold a conference, image data generated by photographing the users gathered in the conference room with the camera 9 is used, and the users whose faces appear in the image data are set as related parties. Using the relationships set in this way, it is possible to determine whether users other than the operator are related parties to the operator.

[0387] The memory circuit 460 (storage means) may store the number of times that the operator and a person other than the operator have held a conference in the same conference room in the past. That is, in the user table 410a shown in Fig. 24, each piece of user information further includes the number of times. This number is the number of times that multiple people have held a conference in the same conference room in the past.

[0388] The determination control unit 447 may determine whether or not the person is a related person by reading the number of times from the user table 410a in the storage circuit 460 and determining whether or not the number of times read is equal to or greater than a predetermined threshold. If the number of times read is equal to or greater than the predetermined threshold, it determines that the person is a related person, and if the number of times read is less than the predetermined threshold, it determines that the person is not a related person.

[0389] 5. Example 4 A fourth embodiment of the present invention will be described.

[0390] 5.1 System Configuration In the fourth embodiment, as shown in FIG. 28(a), a camera 9a is installed on the upper part of the inner wall of an office 471a such that the optical axis of the lens faces into the office 471a.

[0391] In the office 471a, multiple users are gathered and looking at the same document. The camera 9a captures the users gathered in the office 471a and generates image data. In the image (still image) represented by the generated image data, the users gathered in the office 471a are captured together with their respective faces.

[0392] The camera 9a is connected to the server device 4 of the security system 1 of the first embodiment via a signal line, and transmits the generated image data to the server device 4 via the signal line.

[0393] The camera 9a may generate video data by capturing images. Also, a plurality of cameras may be installed in the office 471a. Each of the plurality of cameras captures images to generate image data, and transmits the generated image data to the server device 4.

[0394] 5.2 Server device 4 The input circuit 459 of the server device 4 is connected to the camera 9a.

[0395] The input circuit 459 receives image data from the camera 9a.

[0396] 28(b), the main control unit 441 of the server device 4 further includes a viewpoint processing unit 448. The viewpoint processing unit 448 will be described later.

[0397] The memory circuit 460 of the server device 4 stores a feature group 420 as shown in Fig. 5 and Fig. 29. The memory circuit 460 also has an area for storing a viewpoint group 440 as shown in Fig. 29. The viewpoint group 440 stores feature data 481, 482, 483, 484, ..., and stores viewpoints 486, 487, 488, 489, ... in association with the feature data 481, 482, 483, 484, ..., respectively. In the example shown in this figure, the viewpoints 486 and 489 are assigned related party numbers 491 and 492, respectively.

[0398] Feature amount data 481, 482, 483, 484, . . . are extracted from face images of users who have gathered in office 471a.

[0399] Also, viewpoints 486, 487, 488, 489, . . . corresponding to feature amount data 481, 482, 483, 484, . . . are extracted from face images of users who have gathered in office 471a, and indicate the positions of the gaze points of each face image.

[0400] Moreover, among viewpoints 486, 487, 488, 489, ..., those that match (or exist within a predetermined viewpoint range) are assigned the same related party number. Here, related party numbers 491 and 492 are both "020", which indicates that viewpoints 486 and 489 match. In other words, the viewpoints of the two users looking at viewpoints 486 and 489 are in the same position, indicating that the two users are looking at the same position.

[0401] 6. Moreover, the storage circuit 460 holds a user table 410b shown in FIG. 30, instead of the user table 410 shown in FIG.

[0402] The user table 410b has a data structure similar to that of the user table 410. The user table 410b differs from the user table 410 in that the user information 411 included in the user table 410b further includes a related party number 415 and a number of times 416 in addition to a user identifier 412, a company name 413, and a department name 414. Here, when the first user information and the second user information in the user table 410b include the same related party number, the user indicated by the first user information and the user indicated by the second user information are shown in one image data captured and generated by the camera 9a and are related parties who were looking at one document.

[0403] For example, in the user table 410b shown in FIG. 30, two pieces of user information including the user identifiers "ID001" and "ID003" each include the same related party number "020."

[0404] In other words, the two users identified by the user identifiers "ID001" and "ID003" appear in a single image data captured and generated by camera 9a, indicating that they are related parties who were looking at a single document.

[0405] Also, for example, in the user table 410b shown in FIG. 30, two pieces of user information including the user identifiers "ID001" and "ID003" include the number of times "012".

[0406] Here, the number of times indicates the number of times that two users identified by user identifiers "ID001" and "ID003" appeared in one image data and were looking at one document when the inside of office 471a was photographed multiple times by camera 9a.

[0407] Therefore, the more times the two users meet, the stronger the relationship between them is.

[0408] If the number of times exceeds a threshold, for example, if the number of times exceeds a threshold "10", it may be determined that the two users identified by the user identifiers "ID001" and "ID003" are related parties having a strong relationship.

[0409] The face recognition processing unit 443 detects a plurality of face images from the image represented in the image data received from the camera 9a.

[0410] For each of the multiple face images detected by face recognition processing unit 443, the following processes (a) to (f) are executed.

[0411] (a) Under the control of the viewpoint processing unit 448, the feature amount extraction unit 444 extracts feature amounts from the face image and generates feature amount data.

[0412] (b) Under the control of the viewpoint processing unit 448, the position calculation unit 445 calculates the standing position of the user of the face image.

[0413] (c) Under the control of the viewpoint processor 448, the gaze viewpoint calculator 446 calculates the horizontal direction of the face of the user of the facial image.

[0414] (d) Under the control of viewpoint processing unit 448, gaze viewpoint calculation unit 446 calculates the inclination of the user's face in the facial image from the vertical direction.

[0415] (e) Under the control of the viewpoint processing unit 448, the gaze viewpoint calculation unit 446 calculates the viewpoint of the user of the face image.

[0416] (f) The viewpoint processing unit 448 writes the generated feature amount data and the calculated viewpoint into the viewpoint group 440 of the storage circuit 460 in association with each other.

[0417] When the repetition of the above steps (a) to (f) is completed, the viewpoint processing unit 448 executes the following steps (g) to (i) for each of the multiple face images detected by the face recognition processing unit 443.

[0418] (g) The viewpoint processor 448 reads one viewpoint from the viewpoint group 440 .

[0419] (h) The viewpoint processing unit 448 searches the viewpoint group 440 for other viewpoints that match the read viewpoint (or exist within a predetermined viewpoint range).

[0420] (i) If a matching viewpoint exists, the viewpoint processing unit 448 associates the read viewpoint with other viewpoints within the viewpoint group 440.

[0421] When the repetition of the above steps (g) to (i) is completed, the viewpoint processing unit 448 generates the same related party number for the multiple associated viewpoints in the viewpoint group 440, and assigns the generated related party number.

[0422] Next, the viewpoint processing unit 448 executes the following processes (j) to (m) for each of the multiple face images detected by the face recognition processing unit 443.

[0423] (j) The viewpoint processing unit 448 reads one piece of feature data from the viewpoint group 440.

[0424] (k) The viewpoint processing unit 448 judges whether or not a related party number has been added to the read feature amount data.

[0425] (l) If it is determined that a related party number has been assigned, the viewpoint processing unit 448 searches the feature amount group 420 for feature amount data that is the same as the read feature amount data.

[0426] (m) If feature data identical to the read feature data is present in the feature group 420, the viewpoint processing unit 448 assigns a related party number to the user information in the user table 410b, including a user identifier that identifies the identical feature data, adds "1" to the number of times, and writes the user information with the assigned related party number back to the user table 410b.

[0427] When the repetition of the above steps (j) to (m) is completed, the series of processes is completed.

[0428] 5.3 Operation in the Fourth Embodiment (1) The operation of assigning a related party number The operation of assigning a related party number will be described with reference to the sequence diagrams shown in FIGS.

[0429] The camera 9a takes an image of users who have gathered in the office 471a and generates image data (step S501). The camera 9a transmits the generated image data to the server device 4 via a signal line. The input circuit 459 receives the image data from the camera 9a (step S502).

[0430] The face recognition processing unit 443 detects a plurality of face images from the image represented in the received image data (step S503).

[0431] The viewpoint processing unit 448 exercises control so as to repeatedly execute the following steps S505 to S510 for each of the multiple face images detected by the face recognition processing unit 443 (steps S504 to S511).

[0432] Under the control of the viewpoint processing unit 448, the feature amount extraction unit 444 extracts feature amounts from the face image and generates feature amount data (step S505).

[0433] Under the control of viewpoint processing unit 448, position calculation unit 445 calculates the standing position of the user of the face image (step S506).

[0434] Under the control of viewpoint processing unit 448, gaze viewpoint calculation unit 446 calculates the horizontal direction of the face of the user of the facial image (step S507).

[0435] Under the control of viewpoint processing unit 448, gaze viewpoint calculation unit 446 calculates the inclination of the user's face in the facial image from the vertical direction (step S508).

[0436] Under the control of viewpoint processing unit 448, gaze viewpoint calculation unit 446 calculates the viewpoint of the user of the face image (step S509).

[0437] The viewpoint processing unit 448 writes the feature amount data and the viewpoint in association with each other in the viewpoint group 440 of the storage circuit 460 (step S510).

[0438] When the repetition of the above steps S505 to S510 ends, the viewpoint processing unit 448 exercises control so that the following steps S522 to S525 are repeatedly executed for each of the multiple face images detected by the face recognition processing unit 443 (steps S521 to S526).

[0439] The viewpoint processing unit 448 reads one viewpoint from the viewpoint group 440 (step S522).

[0440] The viewpoint processing unit 448 searches the viewpoint group 440 for other viewpoints that match the read viewpoint (or exist within a predetermined viewpoint range) (step S523).

[0441] If a matching viewpoint exists (YES in step S524), the viewpoint processing unit 448 associates the read viewpoint with other viewpoints within the viewpoint group 440 (step S525).

[0442] When the repetition of the above steps S522 to S525 is completed, the viewpoint processing unit 448 generates the same related party number for the multiple associated viewpoints in the viewpoint group 440, and assigns the generated related party number (step S527).

[0443] Next, the viewpoint processing unit 448 exercises control so as to repeatedly execute the processes of the following steps S532 to S538 for each of the multiple face images detected by the face recognition processing unit 443 (steps S531 to S539).

[0444] The viewpoint processing unit 448 reads one piece of feature amount data from the viewpoint group 440 (step S532).

[0445] The viewpoint processing unit 448 determines whether or not a related person number has been assigned to the read feature amount data (step S533).

[0446] When it is determined that a related party number has been assigned (YES in step S533), viewpoint processing unit 448 searches feature amount data identical to the read feature amount data from feature amount group 420 (step S534).

[0447] If feature data identical to the read feature data exists in feature group 420 (step S535: Yes), viewpoint processing unit 448 assigns a related party number to the user information in user table 410b including a user identifier that identifies the identical feature data (step S536), adds "1" to the number of times (step S537), and writes the user information with the related party number assigned back to user table 410b (step S538).

[0448] When the repetition of the above steps S532 to S538 is completed, the series of processes ends.

[0449] (2) Actions to determine whether or not someone is a related party The operation of determining whether or not a person is a related person in the fourth embodiment can be performed similarly to the operation of determining whether or not a person is a related person in the third embodiment. In this case, if the number of times included in the user information in the user table 410b is equal to or greater than a predetermined threshold, the person may be determined to be a related person, and if the number of times included in the user information is less than the predetermined threshold, the person may be determined not to be a related person.

[0450] 5.4 Summary As described above, the memory circuit 460 (storage means) stores the number of times the operator and a person other than the operator have looked at the same document in the past, and the judgment control unit 447 reads the number of times from the memory circuit 460 and judges whether the number of times read is equal to or greater than a predetermined threshold to judge whether the person is a related person. If the number of times read is equal to or greater than the predetermined threshold, it is determined that the person is a related person, and if the number of times read is less than the predetermined threshold, it is determined that the person is not a related person.

[0451] In this way, when multiple users gather in an office and are looking at one document, the camera 9a uses image data generated by photographing the users who have gathered in the office, and among the multiple face images of users appearing in the image data, the user looking at one document is deemed to be a related party, and it is possible to determine whether a user other than the operator is a related party of the operator. Also, by counting the number of occurrences of such a situation, if the number of occurrences is equal to or exceeds a threshold, it is possible to determine that the user is a related party with a stronger relationship.

[0452] 6 Example 5 A fifth embodiment of the present invention will be described.

[0453] In the fifth embodiment, if the facial image of another user in the image data generated by the camera 6 or an identifier identifying its features is not stored in the user table 410, or if there is a partial image in the image data that cannot be clearly recognized as a facial image, the operator is notified of this and is left to make a judgment.

[0454] Here, for example, when the image of a person captured in the image data is extremely small and the image of the part corresponding to the face is also extremely small, when the image corresponding to the face is hidden behind other images such as hair, or when the person is facing to the side, diagonally backwards or backwards, a phenomenon occurs in which it is not possible to clearly recognize whether or not it is a face image.

[0455] 6.1 System Configuration The fifth embodiment has a similar configuration to that of the first embodiment. The following description will focus on the differences from the first embodiment.

[0456] (1) When cutting out facial image data including a facial image showing the face of a user from an image 601 included in image data 600, the facial recognition processing unit 443 of the main control unit 441 of the server device 4 determines whether or not there is a partial image that cannot be clearly recognized as a facial image.

[0457] If face recognition processing section 443 determines that there is no partial image that cannot be recognized as a face image, general control section 442 transfers control to step S108 in FIG. 10 and continues the process.

[0458] When the face recognition processing unit 443 determines that there is a partial image that cannot be recognized as a face image, the overall control unit 442 generates warning information. Here, the generated warning information indicates a message asking the operator whether or not people in the vicinity of the operator pose a security problem. The overall control unit 442 controls the network communication circuit 456 to transmit the generated warning information to the image forming apparatus 5 via the network 2. The network communication circuit 456 transmits the warning information to the image forming apparatus 5.

[0459] As an example, FIG. 34 shows a screen 651 that is displayed on operation panel 519 of image forming apparatus 5 in response to warning information.

[0460] The screen 651 includes a message area 652. In the message area 652, a warning message for the operator is displayed. For example, the message area 652 includes a message such as "Warning: Is there a security problem with people nearby? [YES] [NO]." This message is for asking the operator whether or not people standing around the operator pose a security problem when an identifier for identifying the feature amount of a face image does not exist in the user table 410, or when it is not possible to clearly recognize whether the image is a face image.

[0461] (2) As shown in step S137 of FIG. 11, the judgment control unit 447 included in the main control unit 441 of the server device 4 compares the features extracted from other face images with the features contained in all the feature data in the feature group 420 of the memory circuit 460, and judges whether they match.

[0462] Here, if the judgment control unit 447 judges that no matching feature exists, the overall control unit 442 generates the same warning information as above. The overall control unit 442 controls the network communication circuit 456 to transmit the generated warning information to the image forming apparatus 5 via the network 2. The network communication circuit 456 transmits the warning information to the image forming apparatus 5.

[0463] (3) The network communication circuit 556 of the image forming apparatus 5 receives the warning information from the server apparatus 4 via the network 2. The network communication circuit 556 outputs the received warning information to the integrated control unit 542.

[0464] When the warning information is received, if an operation is being accepted from the operator, the overall control unit 542 performs control to temporarily stop the operation. For example, if a copy operation is being accepted from the operator, the overall control unit 542 controls the scanner control circuit 557 and the printer control circuit 559 to temporarily stop the copy operation.

[0465] Next, general control unit 542 controls message generation unit 544 to generate a message in accordance with the received warning information. Message generation unit 544 outputs the generated message to operation panel 519. Operation panel 519 displays, as an example, a screen 651 shown in FIG.

[0466] The operation panel 519 receives a response from the operator. The response from the operator is either “YES” or “NO.” The operation panel 519 outputs the response received from the operator to the general control unit 542.

[0467] The general control unit 542 receives a response from the operator via the operation panel 519 .

[0468] If the received response is "YES", the overall control unit 542 performs control to continue the paused operation. For example, if a copy operation is received from the operator, the overall control unit 542 controls the scanner control circuit 557 and the printer control circuit 559 to continue the copy operation.

[0469] On the other hand, if the received response is "NO", the overall control unit 542 performs control to cancel the paused operation. For example, if a copy operation has been received from the operator, the overall control unit 542 cancels the copy job, and controls the scanner control circuit 557 and the printer control circuit 559 to cancel the copy operation.

[0470] As described above, when the determination control unit 447 cannot determine whether the person indicated by the person information is a related person, it notifies the image forming apparatus 5 to request the operator to determine whether the person is a related person of the operator. Furthermore, when the determination control unit 447 cannot recognize the face image of the person from the acquired person image, it determines that it cannot determine whether the person is a related person.

[0471] Furthermore, the face recognition processing unit 443 (search means) searches the memory circuit 460 for a face image that matches the acquired face image. If the determination control unit 447 cannot find a face image that matches the acquired face image from the memory circuit 460, it determines that it is not possible to determine whether the person is a related person.

[0472] 6.2 Operation in the Fifth Embodiment The operation in the fifth embodiment will be described with reference to the sequence diagram shown in FIG.

[0473] In step S101 shown in FIG. 10, after the login by the user is executed by the login control unit 546, the operation panel 519 of the image forming apparatus 5 accepts, for example, a copy operation from the operator (step S551).

[0474] Also, in step S107 shown in FIG. 10, after the face recognition processing unit 443 detects face images representing all of the user's facial parts from the image data 600 written to the memory circuit 460, the face recognition processing unit 443 determines whether or not there is a partial image that cannot be clearly recognized as a facial image when cutting out face image data including face images representing the user's facial parts from the image 601 included in the image data 600 (step S552).

[0475] If the face recognition processing unit 443 determines that there is no partial image that cannot be clearly recognized as a face image ("not present" in step S553), the overall control unit 442 transfers control to step S108 in FIG. 10 and continues the processing.

[0476] On the other hand, if face recognition processing unit 443 determines that there is a partial image that cannot be clearly recognized as a face image ("Exists" in step S553), central control unit 442 generates warning information. Central control unit 442 controls so that the generated warning information is transmitted to image forming apparatus 5 via network 2 (step S554).

[0477] 11, the judgment control unit 447 compares the feature extracted from another face image with the feature included in all the feature data in the feature group 420 of the storage circuit 460 to judge whether they match, and if it is judged that there is no matching feature ("NO" in step S138), the general control unit 442 generates warning information. The general control unit 442 controls so as to transmit the generated warning information to the image forming apparatus 5 via the network 2 (step S555).

[0478] The general control unit 542 receives the warning information from the server device 4 via the network 2 (steps S554 and S555).

[0479] When the warning information is received, for example, if a copy operation is being accepted, the overall control unit 542 performs control so as to temporarily stop the copy operation (step S561).

[0480] Next, message generating unit 544 generates a message in accordance with the received warning information, and operation panel 519 displays a screen including the message (step S562).

[0481] The operation panel 519 receives a response from the operator (step S563).

[0482] If the received response is "YES" (step S564: "YES"), the overall control unit 542 controls the scanner control circuit 557 and the printer control circuit 559 to, for example, continue executing the copy operation (step S565).

[0483] On the other hand, if the received response is "NO" ("NO" in step S564), the central control unit 542, for example, cancels the copy job and controls the scanner control circuit 557 and the printer control circuit 559 to, for example, cancel the copy operation (step S566).

[0484] This completes the series of operations.

[0485] 6.3 Summary If the face image of another user in the image data or an identifier that identifies its feature amount is not stored in the user table 410, or if it cannot be clearly determined whether the image data is a face image, the operator is notified of this, and depending on the operator's response, the operation is continued or stopped. This makes it possible to maintain security.

[0486] 7 Example 6 Example 6 of the embodiment will be described.

[0487] In the sixth embodiment, when it is determined using image data generated by photographing with the camera 6 that a person who is not involved is present near the operator, the operation of the operator on the image forming device 5 is disabled until the person who is not involved moves away from the operator by a certain distance or more.

[0488] 7.1 System Configuration The sixth embodiment has a similar configuration to the first embodiment. Here, the differences from the first embodiment will be mainly described.

[0489] The judgment unit 543 of the image forming apparatus 5 judges whether or not there is even one "1" among the received processing codes (0) to (n). In other words, it judges whether or not there is a person who is not a related person around the operator.

[0490] If there is even one "1" among the received processing codes (0) to (n), that is, if there is a person who is not a related party, the central control unit 542 disables the operation keys of the operation panel 519 of the image forming apparatus 5. For example, the operation panel 519 has a copy key as an operation key, and the central control unit 542 disables the copy key. That is, even if the copy key is pressed, a copy operation is not started. In addition, the central control unit 542 controls the message generating unit 544 to generate a message indicating that a person who is not a related party is nearby and therefore the operation keys are to be disabled until the person who is not a related party leaves. The message generating unit 544 outputs the generated message to the operation panel 519 and controls to display a screen including the generated message. As an example, a screen 653 displayed on the operation panel 519 of the image forming apparatus 5 is shown in FIG. 36.

[0491] The screen 653 includes a message area 654. A warning message for the operator is displayed in the message area 654. For example, the message area 654 includes a message such as "Warning: An outsider is nearby. The copy key will be disabled until an outsider moves a certain distance away [Yes]."

[0492] If there is no "1" among the received processing codes (0) to (n), that is, if there is no person who is not a related party, the overall control unit 542 activates the operation keys. The operation panel 519 receives the operation keys, and the overall control unit 542 exercises control so as to execute a process corresponding to the received operation key. For example, the overall control unit 542 activates the copy key, and the operation panel 519 receives the copy key, and the overall control unit 542 exercises control so as to execute a copy operation corresponding to the received copy key. Next, the user is logged out.

[0493] Next, the camera 6 generates image data by capturing an image, and transmits the generated image data to the image forming device 5, which then transmits the image data to the server device 4.

[0494] The server device 4 executes steps S107 to S144 shown in Figures 10 and 11, and then transfers control to the judgment unit 543 to judge whether or not there is even one "1" among the received processing codes (0) to (n).

[0495] As described above, when the determination control unit 447 determines that the person is not a related person, the network communication circuit 456 (notification means) notifies the image forming apparatus 5 to disable the operation keys.

[0496] 7.2 Operation of Example 6 The operation of the sixth embodiment will be described with reference to the sequence diagram shown in FIG.

[0497] 37 is used in place of the sequence diagram shown in FIG. 12 among FIGS. 10 to 12 showing the operation of the first embodiment.

[0498] In steps S133 and S143 shown in FIG. 11, after the image forming device 5 receives processing code (0) to processing code (n) from the server device 4, the judgment unit 543 judges whether or not there is at least one "1" among the received processing code (0) to processing code (n) (step S161).

[0499] If any one of the received processing codes (0) to (n) is "1" ("YES" in step S162), the central control unit 542 disables the operation keys, such as the copy key, of the operation panel 519 of the image forming apparatus 5 (step S601). Next, the message generating unit 544 generates a message indicating that a person who is not involved is nearby and therefore the operation keys will be disabled until the person who is not involved leaves. The operation panel 519 displays a screen including the generated message (step S602).

[0500] If there is no "1" among the received processing codes (0) to (n) ("NO" in step S162), the central control unit 542 activates an operation key (e.g., copy key) (step S603). Next, the operation panel 519 accepts an operation key (e.g., copy key) (step S604), and the central control unit 542 controls the execution of a process corresponding to the accepted operation key (e.g., copy key) (step S605). Next, the user is logged out (step S606).

[0501] Next, the camera 6 generates image data by shooting (step S102a) and transmits the generated image data to the image forming apparatus 5 (step S103a), and the image forming apparatus 5 transmits the image data to the server apparatus 4 (step S106a). Next, the server apparatus 4 executes steps S107 to S144 shown in Fig. 10 to Fig. 11. Next, control is transferred to step S161, and the process is repeated.

[0502] This completes the series of operations.

[0503] 7.3 Summary As described above, when it is determined that a person other than the person involved is present near the operator using image data generated by photographing with the camera 6, the operation of the operator on the image forming device 5 is invalidated until the person other than the person involved moves away from the operator by a certain distance or more. This makes it possible to maintain security.

[0504] 8 Example 7 A seventh embodiment of the present invention will be described.

[0505] In the seventh embodiment, when a document that has been read is left on document ejection tray 518 due to the operator forgetting to pick it up, or when a printed matter is left on printed matter ejection tray 516 due to the operator forgetting to pick it up, image forming apparatus 5 issues an alert to the operator. At that time, when there are people who are not related persons around the operator, image forming apparatus 5 issues a more emphasized alert compared to when only related persons are around the operator.

[0506] 8.1 System Configuration Example 7 has a similar configuration to Example 1. Here, the differences from Example 1 will be mainly described.

[0507] Operation panel 519 further includes a sound source that generates a beep sound and a speaker that outputs the beep sound. When a document that has been read is left on document discharge tray 518 because the operator forgets to pick it up, or when a printed matter is left on printed matter discharge tray 516 because the operator forgets to pick it up, the speaker outputs the beep sound generated by the sound source as an alert sound under the control of general control unit 542. The speaker can change the volume of the beep sound to be output to a large volume, a medium volume, a small volume, or the like under the control of general control unit 542.

[0508] The judgment unit 543 of the image forming apparatus 5 judges whether or not there is even one "1" among the received processing codes (0) to (n). In other words, it judges whether or not there is a person who is not a related person around the operator.

[0509] If there is even one "1" among the received processing codes (0) to (n), that is, if there is a person who is not a related party near the operator, the operation panel 519 displays a message to the effect that a person who is not a related party is near the operator, and the speaker 515 provided on the operation panel 519 outputs a warning sound. Next, the operation panel 519 accepts operations such as a copy job and a print job from the operator, and the general control unit 542 controls the scanner control circuit 557, the printer control circuit 559, etc. so that they operate in accordance with the accepted operation.

[0510] Among the processing codes (0) to (n), the processing code "1" also means that when it is detected that a document that has been read and left on document output tray 518 of image forming apparatus 5 because the operator forgot to pick it up, or when it is detected that a printed matter has been left on printed matter output tray 516 because the operator forgot to pick it up, the processing code "1" output by server apparatus 4 is synonymous with a notification of a warning output from server apparatus 4 to image forming apparatus 5.

[0511] After the operations of image reader 511, printer 512, etc. are completed, general control unit 542 measures the passage of a predetermined time. Here, in the case of a copy job, the predetermined time is the time normally required for the operator to remove the document from document discharge tray 518 after reading of the document is completed. Also, in the case of a print job, the predetermined time is the time normally required for the operator to remove the printed matter from print discharge tray 516 after printing is completed.

[0512] After a predetermined time has elapsed, if the document detection sensor provided in document output tray 518 detects a document, or if the printed matter detection sensor provided in printed matter output tray 516 detects a printed matter, the general control unit 542 controls the speaker to output a beep sound generated by a sound source as an alert sound.

[0513] In this case, when it is determined that a person other than the related person is present around the operator, the overall control unit 542 controls the speaker to output a loud beep sound as an alert sound, whereas when it is determined that only the related person is present around the operator, the overall control unit 542 controls the speaker to output a low volume beep sound as an alert sound.

[0514] 8.2 Operation of Example 7 The operation of the seventh embodiment will be described with reference to the sequence diagram shown in FIG.

[0515] The sequence diagram shown in FIG. 38 is used in place of the sequence diagram shown in FIG. 12 among FIGS. 10 to 12 showing the operation of the first embodiment.

[0516] In steps S133 and S143 shown in FIG. 11, after the image forming device 5 receives processing code (0) to processing code (n) from the server device 4, the judgment unit 543 judges whether or not there is at least one "1" among the received processing code (0) to processing code (n) (step S161).

[0517] If there is even one "1" among the received processing codes (0) to (n) ("YES" in step S162), message generating unit 544 generates a warning message, such as a warning that someone who is not a related person is nearby, in response to an instruction from determination unit 543. Message generating unit 544 outputs the generated warning message to operation panel 519 via input / output circuit 558. Operation panel 519 displays the warning message (step S621). Next, warning sound generating unit 545 generates a warning sound in response to an instruction from determination unit 543. Warning sound generating unit 545 outputs the generated warning sound to speaker 515 via output circuit 562. Speaker 515 outputs the warning sound as audio (step S622).

[0518] The operation panel 519 receives operations such as a copy job and a print job from the operator (step S623), and the general control unit 542 controls the scanner control circuit 557, the printer control circuit 559, etc. so that they operate in accordance with the received operations (step S624).

[0519] After the operations of the image reader 511, the printer 512, etc. are completed, the general control unit 542 measures the passage of a predetermined time, and waits until the predetermined time has elapsed ("NO" in step S625).

[0520] When a predetermined time has elapsed ("YES" in step S625), if the document detection sensor provided in document ejection tray 518 detects a document, or if the print detection sensor provided in print ejection tray 516 detects a print ("YES" in step S626), the overall control unit 542 controls the speaker to output a beep sound generated by the sound source at a high volume as an alert sound (step S627). Next, the overall control unit 542 transfers control to step S626 and repeats the process.

[0521] If the document detection sensor provided in document ejection tray 518 does not detect a document, and if the printed matter detection sensor provided in printed matter ejection tray 516 does not detect a printed matter ("NO" in step S626), login control unit 546 logs out (step S628). Next, overall control unit 542 transfers control to step S101 and repeats the process.

[0522] On the other hand, if there is no "1" among the received processing codes (0) through (n) ("NO" in step S162), the operation panel 519 accepts an operation such as a copy job or a print job from the operator (step S165), and the general control unit 542 controls the scanner control circuit 557, the printer control circuit 559, etc. to operate in accordance with the accepted operation (step S166).

[0523] After the operations of the image reader 511, the printer 512, etc. are completed, the general control unit 542 measures the passage of a predetermined time, and waits until the predetermined time has elapsed ("NO" in step S631).

[0524] When a predetermined time has elapsed ("YES" in step S631), if the document detection sensor provided in document ejection tray 518 detects a document, or if the print detection sensor provided in print ejection tray 516 detects a print ("YES" in step S632), the central control unit 542 controls the speaker to output a beep sound generated by the sound source at a low volume as an alert sound (step S633). Next, the central control unit 542 transfers control to step S632 and repeats the process.

[0525] If the document detection sensor provided in document ejection tray 518 does not detect a document, and if the printed matter detection sensor provided in printed matter ejection tray 516 does not detect a printed matter ("NO" in step S632), login control unit 546 logs out (step S634). Next, overall control unit 542 transfers control to step S101 and repeats the process.

[0526] 8.3 Summary As described above, when the judgment control unit 447 judges that the person is not a related party, the network communication circuit 456 (notification means) notifies the image forming device 5 to issue a more emphasized warning than usual when the operator forgets to remove the original document or printed material from the image forming device 5.

[0527] Here, the image forming device 5 may be an image reading device that reads an original and generates image data, or a printing device that produces printed matter.

[0528] As described above, when a document that has been read is left on document output tray 518, or when a printed matter is left on printed matter output tray 516, image forming apparatus 5 issues an alert to the operator. At that time, when there are people who are not related parties around the operator, image forming apparatus 5 issues a more emphasized alert compared to when only related parties are around the operator. This makes it possible to maintain security.

[0529] As described above, operation panel 519 includes a sound source that generates a beep sound and a speaker that outputs the beep sound, and when a document that has been read is left on document output tray 518 or a printed material is left on printed material output tray 516, the speaker outputs a beep sound as an alert sound. However, the present invention is not limited to this.

[0530] Operation panel 519 may further include an LED lamp, and when a document that has been read is left on document ejection tray 518, or when a printed matter is left on printed matter ejection tray 516, the LED lamp may emit light as an alert under the control of central control unit 542. At that time, when there is a person who is not a related person around the operator, the LED lamp may emit a stronger light under the control of central control unit 542, compared to when only related people are around the operator.

[0531] Furthermore, when a document that has been read is left on document ejection tray 518, or when a printed matter is left on printed matter ejection tray 516, the LED lamp may repeatedly blink as an alert under the control of central control unit 542. At that time, when there are people who are not related parties around the operator, the LED lamp may be controlled by central control unit 542 to have a shorter blinking cycle than when only related parties are around the operator.

[0532] 9 Example 8 An eighth embodiment of the present invention will now be described.

[0533] In the eighth embodiment, if there is a person who is not a related party near the operator, when the operator leaves the image forming device 5 during processing in the image forming device 5, for example, while a copy job is being executed or while a print job is being executed, for example, to replenish consumables (recording sheets, toner, etc.) in the image forming device 5, the image forming device 5 issues an alert to the operator.

[0534] 9.1 System Configuration Example 8 has a similar configuration to Example 1. Here, the differences from Example 1 will be mainly described.

[0535] (1) Configuration of Image Forming Device 5 and Camera 6 The operation panel 519 further includes a sound source that generates a beep sound and a speaker that outputs the beep sound. If there is a person who is not a related person around the operator, when the operator moves away from the image forming apparatus 5 during processing in the image forming apparatus 5, the speaker outputs the beep sound generated by the sound source as an alert sound under the control of the general control unit 542.

[0536] If there is even one "1" among the processing codes (0) to (n) received from the server device 4, that is, if there is someone who is not a related party in the vicinity of the operator, the judgment unit 543 receives the operator code from the server device 4 via the network 2 and the network communication circuit 556.

[0537] Here, the operator code indicates whether or not the operator is captured in the image 601 captured by the camera 6. When the operator code is "0", it indicates that the operator is captured in the image 601. On the other hand, when the operator code is "1", it indicates that the operator is not captured in the image 601.

[0538] The determination unit 543 determines whether the received operator code is "0" or "1".

[0539] If the received operator code is determined to be "0", no alert is issued.

[0540] On the other hand, if it is determined that the received operator code is "1", the speaker, under the control of the general control unit 542, outputs a beep sound generated by the sound source as an alert sound.

[0541] The camera 6 generates image data by capturing an image at any time, and transmits the generated image data to the image forming apparatus 5.

[0542] The input circuit 561 receives image data from the camera 6 .

[0543] When the network communication circuit 556 receives the image data through the input circuit 561 , the network communication circuit 556 transmits the received image data to the server device 4 via the network 2 .

[0544] (2) Configuration of server device 4 The network communication circuit 456 receives image data from the image forming apparatus 5 via the network 2. The network communication circuit 456 writes the received image data as image data 600 in the storage circuit 460.

[0545] The face recognition processing unit 443 detects face images showing the facial portions of all users from the image data 600 written in the memory circuit 460. In addition, the face recognition processing unit 443 attempts to identify the face image of the operator who operates the image forming apparatus 5 from the detected face images.

[0546] If the facial image of the operator operating the image forming apparatus 5 can be identified, the determination control unit 447 sets the operator code to "0." On the other hand, if the facial image of the operator operating the image forming apparatus 5 cannot be identified, the determination control unit 447 sets the operator code to "1."

[0547] The determination control unit 447 controls the network communication circuit 456 to transmit the operator code to the image forming apparatus 5 via the network 2 .

[0548] The network communication circuit 456 transmits the operator code to the image forming apparatus 5 via the network 2 under the control of the determination control unit 447 .

[0549] 9.2 Operation in Example 8 The operation in the eighth embodiment will be described with reference to the sequence diagram shown in FIG.

[0550] The camera 6 generates image data by capturing an image at any time (step S103a), and transmits the generated image data to the image forming apparatus 5 (step S104a).

[0551] The input circuitry 561 of the image forming apparatus 5 receives the image data from the camera 6 (step S104a).

[0552] When the network communication circuit 556 receives the image data through the input circuit 561, the network communication circuit 556 transmits the received image data to the server device 4 via the network 2 (step S106a).

[0553] The network communication circuit 456 receives the image data from the image forming apparatus 5 via the network 2 (step S106a).

[0554] The face recognition processing unit 443 detects face images showing the facial portions of all users from the image data 600 written in the memory circuit 460 (step S107). In addition, the face recognition processing unit 443 attempts to identify the face image of the operator who operates the image forming apparatus 5 from the detected face images (step S108).

[0555] If the facial image of the operator operating the image forming apparatus 5 can be identified ("YES" in step S641), the determination control unit 447 sets the operator code to "0" (step S643). On the other hand, if the facial image of the operator operating the image forming apparatus 5 cannot be identified ("NO" in step S641), the determination control unit 447 sets the operator code to "1" (step S642).

[0556] The network communication circuit 456, under the control of the determination control unit 447, transmits the operator code to the image forming apparatus 5 via the network 2 (step S644).

[0557] The determination unit 543 determines whether or not there is at least one "1" among the received processing codes (0) to (n) (step S161).

[0558] If there is at least one "1" among the received processing codes (0) to (n) ("YES" in step S162), message generating unit 544 generates a warning message in response to an instruction from determination unit 543, and operation panel 519 displays the warning message (step S163). Next, warning sound generating unit 545 generates a warning sound, and speaker 515 outputs the warning sound (step S164).

[0559] Next, the determination unit 543 receives the operator code from the server device 4 via the network 2 and the network communication circuit 556 (step S644).

[0560] The determination unit 543 determines whether the received operator code is "0" or "1" (step S651).

[0561] If it is determined that the received operator code is "1" ("=1" in step S651), the speaker outputs a beep sound generated by the sound source as an alert sound under the control of the central control unit 542 (step S652). Next, the central control unit 542 transfers control to step S651. That is, the beep sound continues to be output until the operator returns in front of the image forming apparatus 5. In other words, the beep sound continues to be output while the operator code is "1".

[0562] On the other hand, if it is determined that the received operator code is "0" ("=0" in step S651), no beep is output, and the operation panel 519 accepts the operator's operation (step S165). The main control unit 541 controls the network communication circuit 556, the scanner control circuit 557, the printer control circuit 559, etc. to operate in accordance with the operator's operation, and executes the corresponding process (step S166). Next, the login control unit 546 logs out (step S167), and then the main control unit 541 transfers control to step S101 in FIG. 10 to continue the process.

[0563] 9.3 Summary As described above, when the network communication circuit 456 (notification means) detects that the operator has left the image forming device 5 if the judgment control unit 447 determines that the operator is not a related party, it notifies the image forming device 5 to issue a warning.

[0564] As described above, if there is a person who is not a related party near the operator, when the operator leaves the image forming device 5 during processing in the image forming device 5, the image forming device 5 issues an alert to the operator. This makes it possible to maintain security.

[0565] 10 Example 9 A ninth embodiment of the present invention will be described.

[0566] In the ninth embodiment, when processing a document that is important from a security standpoint, for example, when security settings such as printing a confidential stamp, watermark, or copy guard (hereinafter referred to as confidential stamp, etc.) are accepted from an operator and a print job or copy job is executed, if there is a person who is not an involved party near the operator, the image forming device 5 issues a warning to the operator.

[0567] 10.1 System Configuration Example 9 has a similar configuration to Example 1. Here, the differences from Example 1 will be mainly described.

[0568] An operation panel 591 receives settings such as a confidential stamp from a user or an operator.

[0569] The general control unit 542 performs control so as to execute a print job or a copy job based on these settings.

[0570] The determination unit 543 determines whether or not any one of the processing codes (0) to (n) received from the server device 4 is "1".

[0571] If there is even one "1" among the received processing codes (0) to (n), that is, if there is someone around the operator who is not a related party, the judgment unit 543 judges whether or not the user has accepted the setting of a confidential stamp, etc. from the operator.

[0572] When it is determined that a setting such as a confidential stamp has been accepted from the operator, message generation unit 544 generates a warning message in response to an instruction from determination unit 543. Message generation unit 544 outputs the generated warning message to operation panel 519 via input / output circuit 558. Operation panel 519 displays the warning message. Next, warning sound generation unit 545 generates a warning sound in response to an instruction from determination unit 543. Warning sound generation unit 545 outputs the generated warning sound to speaker 515 via output circuit 562. Speaker 515 outputs the warning sound as audio.

[0573] Among the processing codes (0) to (n) received from the server device 4, the processing code "1" also means that when the image forming device 5 accepts settings such as a confidential stamp from the operator, the image forming device 5 instructs the image forming device 5 to display a warning message and output a warning sound. In other words, the output of the processing code "1" among the processing codes (0) to (n) by the server device 4 is synonymous with a notification of the output of a warning from the server device 4 to the image forming device 5.

[0574] 10.2 Operation of Example 9 The operation of the ninth embodiment will be described with reference to the sequence diagrams shown in FIGS.

[0575] The login control unit 546 executes login by a user and accepts a user identifier (step S101). The login control unit 546 controls to transmit the user identifier to the server device 4 via the network 2. The overall control unit 442 receives the user identifier from the image forming device 5 (step S102).

[0576] The central control unit 442 reads out the user information corresponding to the received user identifier from the user table 410, and extracts the company name and department name from the read out user information (step S105).

[0577] The camera 6 generates image data by capturing an image (step S103), and outputs the generated image data to the image forming apparatus 5 (step S104). The general control unit 542 controls the network communication circuit 556 to transmit the received image data to the server device 4 (step S106). The network communication circuit 456 receives the image data from the image forming apparatus 5 (step S106).

[0578] The face recognition processing unit 443 detects face images showing the facial parts of all users from the received image data (step S107).

[0579] Next, the general control unit 442 transfers the control to step S108 shown in FIG.

[0580] On the other hand, after the user logs in, operation panel 591 accepts settings such as a confidential stamp from the user or the operator (step S671).

[0581] Next, after steps S131 to S144 shown in FIG. 11 are executed, as shown in FIG. 41, the judgment unit 543 judges whether or not there is at least one "1" among the processing codes (0) to (n) received from the server device 4 (step S161).

[0582] If there is at least one "1" among the received processing codes (0) to (n) ("YES" in step S162), the judgment unit 543 judges whether or not a setting such as a confidential stamp has been accepted from the user or operator (step S672).

[0583] When it is determined that a setting such as a confidential stamp has been accepted from the operator (YES in step S672), message generating unit 544 generates a warning message in response to an instruction from determining unit 543. Operation panel 519 displays the warning message (step S163). Next, warning sound generating unit 545 generates a warning sound in response to an instruction from determining unit 543. Speaker 515 outputs the warning sound (step S164).

[0584] If there is no "1" among the received processing codes (0) through (n) ("NO" in step S162), or if it is determined that no settings such as a confidential stamp have been accepted from the user or operator ("None" in step S672), then a warning message and warning sound will not be generated and will not be output.

[0585] Operation panel 519 accepts operations by the operator (step S165).

[0586] The main control unit 541 controls the network communication circuit 556, the scanner control circuit 557, the printer control circuit 559, and the like to operate in accordance with the operation of the operator, and executes the corresponding process (step S166).

[0587] Next, the login control unit 546 performs logging out (step S167), and then the main control unit 541 transfers the control to step S101 shown in FIG. 40 and continues the process.

[0588] This completes the series of processes.

[0589] 10.3 Summary As described above, when the judgment control unit 447 judges that the person is not a related party, the network communication circuit 456 (notification means) notifies the image forming device 5 to issue a warning when security settings are made in the image forming device 5.

[0590] As described above, when a print job or copy job is executed after receiving a confidential stamp setting from an operator, if there is a person who is not an interested party in the vicinity of the operator, the image forming device 5 issues a warning to the operator. This makes it possible to maintain security.

[0591] 10.4 Variations A modification of the ninth embodiment will be described.

[0592] The operation panel 519 accepts settings of, for example, "circulation," "important," or "top secret" as a confidential stamp from the operator. The level of importance in terms of security increases in the order of "circulation," "important," and "top secret."

[0593] The determination unit 543 determines whether the setting is "circulation," "important," or "confidential."

[0594] The general control unit 542 controls the speaker 515 so that the volume of the messages output increases in the order of "Circulation," "Important," and "Top Secret."

[0595] That is, when the determination unit 543 determines that the "circulation" setting has been made, the general control unit 542 sets the volume of the speaker 515 to low volume.

[0596] Moreover, when determining unit 543 determines that "important" has been set, general control unit 542 sets the volume of speaker 515 to medium volume.

[0597] Furthermore, if determining unit 543 determines that "top secret" has been set, general control unit 542 sets the volume of speaker 515 to high volume.

[0598] The speaker 515 outputs a warning sound at a set volume.

[0599] (Operation of the modified example of the 9th embodiment) Next, the operation of the modified example of the ninth embodiment will be described with reference to the flowchart shown in FIG.

[0600] The determination unit 543 determines whether the setting is "circulation," "important," or "confidential" (step S701).

[0601] When the determination unit 543 determines that "circulation" has been set ("circulation" in step S701), the general control unit 542 sets the volume of the speaker 515 to low volume (step S702).

[0602] Furthermore, if determining unit 543 determines that "important" has been set ("important" in step S701), general control unit 542 sets the volume of speaker 515 to medium volume (step S703).

[0603] Furthermore, if the determination unit 543 determines that "TOP SECRET" has been set ("TOP SECRET" in step S701), the general control unit 542 sets the volume of the speaker 515 to high volume (step S704).

[0604] Speaker 515 outputs a warning sound at the set volume (step S705).

[0605] This completes the series of operations.

[0606] (summary) As described above, the network communication circuit 456 notifies the user to issue a warning according to the level of security settings.

[0607] As described above, by changing the strength of the warning in accordance with the contents of the security settings, it is possible to give a warning that corresponds to the contents of the security settings.

[0608] 11 Example 10 A tenth embodiment of the present invention will be described.

[0609] In the tenth embodiment, if the document contains strings of characters such as "confidential" or "top secret", the name of the document's destination, or other security-related strings of characters (security information), if there is anyone who is not an involved party near the operator, the image forming device 5 will issue a warning to the operator.

[0610] 11.1 System Configuration The tenth embodiment has a similar configuration to the first embodiment. The following description will focus on the differences from the first embodiment.

[0611] In the image forming apparatus 5, for example, a copy job is executed.

[0612] As described above, the document contains character strings such as "confidential" and "top secret," the name of the document's destination, and other security-related character strings.

[0613] The main control unit 441 of the image forming apparatus 5 further includes an OCR processing unit 450 (not shown). The OCR processing unit 450 performs OCR processing on an image acquired from a document surface to extract a character string.

[0614] The determination unit 543 determines whether or not any one of the processing codes (0) to (n) received from the server device 4 is "1".

[0615] If there is even one "1" among the received processing codes (0) to (n), that is, if there is someone who is not a related party around the operator, the judgment unit 543 judges whether the character string extracted by the OCR processing unit 450 is a security-related character string.

[0616] When it is determined that the character string is related to security, message generation unit 544 generates a warning message in response to an instruction from determination unit 543. Message generation unit 544 outputs the generated warning message to operation panel 519 via input / output circuit 558. Operation panel 519 displays the warning message. Next, warning sound generation unit 545 generates a warning sound in response to an instruction from determination unit 543. Warning sound generation unit 545 outputs the generated warning sound to speaker 515 via output circuit 562. Speaker 515 outputs the warning sound as audio.

[0617] Among the processing codes (0) to (n) received from the server device 4, a processing code of "1" also means that when security information is detected from a document in the image forming device 5, the image forming device 5 is instructed to display a warning message and output a warning sound. In other words, the output of a processing code of "1" among the processing codes (0) to (n) by the server device 4 is synonymous with a notification of the output of a warning from the server device 4 to the image forming device 5.

[0618] 11.2 Operation of Example 10 The operation of the tenth embodiment will be described with reference to the sequence diagrams shown in FIGS.

[0619] The login control unit 546 executes login by a user and accepts a user identifier (step S101). The login control unit 546 controls to transmit the user identifier to the server device 4 via the network 2. The overall control unit 442 receives the user identifier from the image forming device 5 (step S102).

[0620] The central control unit 442 reads out the user information corresponding to the received user identifier from the user table 410, and extracts the company name and department name from the read out user information (step S105).

[0621] The camera 6 generates image data by capturing an image (step S103), and outputs the generated image data to the image forming apparatus 5 (step S104). The general control unit 542 controls the network communication circuit 556 to transmit the received image data to the server device 4 (step S106). The network communication circuit 456 receives the image data from the image forming apparatus 5 (step S106).

[0622] The face recognition processing unit 443 detects face images showing the facial parts of all users from the received image data (step S107).

[0623] Next, the general control unit 442 transfers the control to step S108 shown in FIG.

[0624] On the other hand, after the user logs in, operation panel 591 accepts a copy operation (step S681).

[0625] The general control unit 542 controls the scanner control circuit 557 to execute scanning (step S682).

[0626] OCR processing unit 450 performs OCR processing on the image of the document surface to extract character strings (step S683).

[0627] Next, after steps S131 to S144 shown in FIG. 11 are executed, the judgment unit 543 judges whether or not there is at least one "1" among the processing codes (0) to (n) received from the server device 4 (step S161).

[0628] If there is at least one "1" among the received processing codes (0) to (n) ("YES" in step S162), the judgment unit 543 judges whether or not a security-related character string, for example, "confidential," has been extracted by the OCR processing unit 450 (step S685).

[0629] If it is determined that a character string related to security has been extracted (YES in step S685), message generating unit 544 generates a warning message in response to an instruction from determining unit 543. Operation panel 519 displays the warning message (step S163). Next, warning sound generating unit 545 generates a warning sound in response to an instruction from determining unit 543. Speaker 515 outputs the warning sound (step S164).

[0630] If there is not even one "1" among the received processing codes (0) through (n) ("NO" in step S162), or if it is determined that no security-related character string has been extracted ("NONE" in step S685), then no warning message or warning sound is generated or output.

[0631] The main control unit 541 controls the printer control circuit 559 to operate in accordance with the operator's operation, and executes printing (step S166a).

[0632] Next, the login control unit 546 performs logging out (step S167), and then the main control unit 541 transfers the control to step S101 shown in FIG. 43 and continues the process.

[0633] This completes the series of processes.

[0634] 11.3 Summary As described above, when the judgment control unit 447 judges that the person is not a related party, the network communication circuit 456 (notification means) notifies the image forming device 5 to issue a warning when the image forming device 5 reads the document surface and detects security information.

[0635] Here, the image forming device 5 may be an image reading device that reads the surface of a document and generates image data.

[0636] As described above, when a document contains security-related character strings, such as character strings such as "confidential" or "top secret," the name of the document's destination, or the like, and there is a person who is not involved in the process near the operator when the operator executes a copy job, the image forming device 5 issues a warning to the operator. This makes it possible to maintain security.

[0637] In addition to copy jobs, the present invention can also be applied to jobs in which an original document is scanned and then the resulting scanned image data is stored in a Box, a job in which the resulting scanned image data is attached to an email and sent, and a job in which the resulting scanned image data is sent to another information processing device.

[0638] 11.4 Variations A modification of the tenth embodiment will be described.

[0639] Among the security-related character strings written in the document, the character strings such as "confidential" and "top secret" have the highest security importance. The document destination name has a medium level of importance, and other security-related character strings have a low level of importance.

[0640] General control unit 542 controls speaker 515 so that the higher the level of importance in terms of security, the louder the output volume becomes.

[0641] If the extracted character string is a character string such as "confidential" or "top secret", the general control unit 542 sets the volume of the speaker 515 to a large volume.

[0642] Furthermore, if the extracted character string is the name of a destination of the document, then the general control unit 542 sets the volume of the speaker 515 to a medium volume.

[0643] Moreover, if the extracted character string is a character string related to other security, the general control unit 542 sets the volume of the speaker 515 to low volume.

[0644] The speaker 515 outputs a warning sound at a set volume.

[0645] (Operation of the modified example of the tenth embodiment) Next, the operation of the modified example of the tenth embodiment will be described with reference to the flowchart shown in FIG.

[0646] The determination unit 543 determines whether the extracted character string is a character string such as "confidential" or "top secret," a destination name of a document, or other character string related to security (step S711).

[0647] If the judgment unit 543 judges that the extracted character string is a character string such as "confidential" or "top secret" (confidential, top secret in step S711), the overall control unit 542 sets the volume of the speaker 515 to high volume (step S712).

[0648] Furthermore, if the determination unit 543 determines that the extracted character string is the name of a destination of the document ("destination name" in step S711), the general control unit 542 sets the volume of the speaker 515 to medium volume (step S713).

[0649] Furthermore, if the determination unit 543 determines that the extracted character string is a character string related to other security ("other" in step S711), the overall control unit 542 sets the volume of the speaker 515 to low volume (step S714).

[0650] Speaker 515 outputs a warning sound at the set volume (step S715).

[0651] This completes the series of operations.

[0652] (summary) As described above, the network communication circuit 456 (notification means) issues a notification so as to issue the warning according to the level of the security information.

[0653] As described above, by changing the strength of the warning in accordance with the contents of the character string related to security written in the document, it is possible to give a warning according to the contents of the character string.

[0654] 12 Other Modifications In the above, several embodiments and several modified examples have been described. However, the present invention is not limited to these embodiments and modified examples. The following may be performed.

[0655] (1) In the above, the image forming device 5 is a tandem-type color multifunction device having the functions of a scanner, a printer, and a copier. However, this is not limited to this. Instead of the image forming device 5, a printing device having the functions of a printer may be used. Also, instead of the image forming device 5, an image reading device having the functions of a scanner may be used.

[0656] (2) In the above, it is stated that the camera 6 is provided on the wall and the image forming device 5 and the camera 6 are connected by a signal line. However, this is not limited to this.

[0657] The imaging device may include a camera.

[0658] A camera installed with the optical axis of the lens facing the image forming device may be connected to the network 2. This camera may transmit image data generated by photographing to the server device 4.

[0659] The security system 1 is configured from an image forming device 5, a camera 6, and a server device 4. However, the present invention is not limited to this.

[0660] The image forming apparatus and the server apparatus may be configured as one body. In this case, the main control unit 541 of the image forming apparatus 5 may have the face recognition processing unit 443, the feature amount extraction unit 444, the position calculation unit 445, the gaze point calculation unit 446, and the judgment control unit 447 included in the main control unit 441 of the server apparatus 4.

[0661] (3) The examples and modifications of the above-described embodiments may be combined. [Industrial Applicability]

[0662] The monitoring device of the present disclosure does not issue a warning if the person peering into an image processing device being operated by the operator is a related person of the operator, thereby reducing the inconvenience for the operator and being useful as a technology for monitoring someone peering into an image processing device being operated by an operator. [Explanation of symbols]

[0663] 1, 1a Security System 2 Network 4, 4a Server equipment 5, 5a Image forming device 5a Image forming device 6. Camera 7a, 7b, 7c Identification cards 9, 9a Camera 441 Main control unit 442 General Control Unit 443 Face Recognition Processing Unit 444 Feature Extraction Unit 445 Position calculation part 446 Gaze Point Calculation Unit 447 Judgment Control Unit 448 Viewpoint Processing Unit 449 Employee ID Image Recognition Unit 450 OCR processing section 451 CPU 452 ROM 453 RAM 456 Network Communication Circuit 458 Input / Output Circuit 459 Input Circuit 460 Memory circuit 461 Bus 500, 500a Control circuit 511 Image Reader 512 Printer 513 Paper feed section 515 Speaker 516 Print output tray 517 Document Tray 518 Document output tray 519 Operation Panel 531 User Table 541 Main control unit 542 General Control Unit 543 Judgment Department 544 Message Generation Unit 545 Warning sound generator 546 Login control section 551 CPU 552 ROM 553 RAM 554 Image Memory 555 Image Processing Circuit 556 Network Communication Circuit 557 Scanner control circuit 558 Input / Output Circuit 559 Printer control circuit 560 Memory circuit 561 Input Circuit 562 Output circuit 563 Wireless communication circuits 564 Antenna 565 Box memory circuit 566 Bus 567 Input / Output Circuit 591 Operation Panel 741 Control Unit 751 CPU 752 ROM 753 RAM 760 Memory circuit 763 Wireless communication circuits 764 Antenna 767 Input / Output Circuit 768 Bus

Claims

1. A monitoring device that monitors whether or not another person is looking at contents displayed on a display unit by an image processing device due to an operation by an operator, a medium that has been drawn and output and placed on a paper output tray, or a document to be read that has been set on a document tray or a document that has been read and placed on the document output tray, An acquisition means for acquiring person information indicating people present within a predetermined range from the image processing device; A determination means for determining whether or not a person other than the operator among the persons identified by the acquired personal information has a predetermined relationship with the operator; A calculation means for calculating a viewpoint position of a person other than the operator; a control means for controlling to output warning information when it is determined that there is even one person among the specified persons who does not have a predetermined relationship and that the calculated viewpoint position for that person is on the display surface of the display unit, the paper output tray, or the document tray or the document output tray, and for controlling to suppress output of warning information in either case when it is determined that all persons are people who have a specific relationship and when it is determined that the calculated viewpoint position is different from the display surface of the display unit, the paper output tray, the document tray, or the document output tray; A monitoring device comprising:

2. A monitoring device that monitors whether another person is looking into the contents displayed by an image processing device due to an operation by an operator, a medium on which a drawing is output, or a document to be read, An acquisition means for acquiring person information indicating people present within a predetermined range from the image processing device; A determination means for determining whether or not a person other than the operator among the persons identified by the acquired personal information has a predetermined relationship with the operator; a control means for controlling to output warning information when it is determined that there is even one person among the identified persons who does not have a predetermined relationship, and for controlling to suppress the output of warning information when it is determined that all the identified persons have the predetermined relationship; a storage means for storing the number of times that the operator and any person identified by the personal information, excluding the operator, have held a conference in the same closed space in the past; The determination means reads out the number of times from the storage means, compares the read number of times with a predetermined threshold value, and determines whether or not the person has a predetermined relationship. A monitoring device comprising:

3. A monitoring device that monitors whether another person is looking into the contents displayed by an image processing device due to an operation by an operator, a medium on which a drawing is output, or a document to be read, An acquisition means for acquiring person information indicating people present within a predetermined range from the image processing device; A determination means for determining whether or not a person other than the operator among the persons identified by the acquired personal information has a predetermined relationship with the operator; a control means for controlling to output warning information when it is determined that there is even one person among the identified persons who does not have a predetermined relationship, and for controlling to suppress the output of warning information when it is determined that all the identified persons have the predetermined relationship; a storage means for storing the number of times that the operator and the persons specified by the personal information, excluding the operator, have viewed the same document in the past; The determination means reads out the number of times from the storage means, compares the read number of times with a predetermined threshold value, and determines whether or not the person has a predetermined relationship. A monitoring device comprising:

4. The determination means determines that the person has a predetermined relationship when the number of times is equal to or greater than a predetermined threshold, and determines that the person does not have a predetermined relationship when the number of times is less than the predetermined threshold.

4. The monitoring device according to claim 2 or 3.

5. A monitoring device that monitors whether another person is looking into the contents displayed by an image processing device due to an operation by an operator, a medium on which a drawing is output, or a document to be read, An acquisition means for acquiring person information indicating people present within a predetermined range from the image processing device; A determination means for determining whether or not a person other than the operator among the persons identified by the acquired personal information has a predetermined relationship with the operator; a control means for controlling to output warning information when it is determined that there is even one person among the specified persons who does not have a predetermined relationship, and for controlling to suppress output of warning information when it is determined that all the persons have the specified relationship; Equipped with When the determining means cannot determine whether or not a person other than the operator among the persons specified by the person information has a predetermined relationship, the determining means notifies the image processing device to request the operator to make such a determination. A monitoring device comprising:

6. an image capturing device that captures an image of a predetermined range of an area including the image processing device is installed in the vicinity of the image processing device; the acquiring means acquires, as the personal information, a person image representing a person other than the operator from an image acquired by shooting, When the determination means cannot recognize a face image of the person from the acquired person image, the determination means determines that it is not possible to determine whether the person has a predetermined relationship.

6. The monitoring device according to claim 5.

7. an image capturing device that captures an image of a predetermined range of an area including the image processing device is installed in the vicinity of the image processing device; the acquiring means acquires, as the personal information, a face image representing a person other than the operator from an image acquired by photographing; moreover, a storage means for storing an identifier for identifying a user who uses the image processing device, a face image of the user, and a name of an organization to which the user belongs in association with each other; a search means for searching the storage means for a face image that matches the face image acquired by the acquisition means, The determination means determines that it is not possible to determine whether or not the person has a predetermined relationship when the search means cannot find a face image that matches the acquired face image from the storage means.

6. The monitoring device according to claim 5.

8. a notification means for notifying the image processing device of control information indicating whether to output warning information or to suppress output of warning information under the control of the control means.

6. The monitoring device according to claim 1, further comprising:

9. When the determining means determines that the person does not have a predetermined relationship, the notifying means notifies the image processing device to issue a warning to the person to move away from the person by a certain distance or more.

9. The monitoring device according to claim 8.

10. The notification means notifies the image processing device to issue a warning to the operator that there is a person who does not have a predetermined relationship when the determination means determines that the person does not have a predetermined relationship.

9. The monitoring device according to claim 8.

11. The notification means notifies the image processing device to disable an operation key when the determination means determines that the person does not have a predetermined relationship.

9. The monitoring device according to claim 8.

12. A monitoring device that monitors whether another person is looking into the contents displayed by an image processing device due to an operation by an operator, a medium on which a drawing is output, or a document to be read, An acquisition means for acquiring person information indicating people present within a predetermined range from the image processing device; A determination means for determining whether or not a person other than the operator among the persons identified by the acquired personal information has a predetermined relationship with the operator; a control means for controlling to output warning information when it is determined that there is even one person among the identified persons who does not have a predetermined relationship, and for controlling to suppress the output of warning information when it is determined that all the identified persons have the predetermined relationship; a notification means for notifying the image processing device of control information indicating whether to output warning information or to suppress output of warning information under the control of the control means; Equipped with The image processing device is an image reading device that reads an original and generates image data, or a printing device that produces printed matter, When the determining means determines that the person does not have a predetermined relationship, and when the image processing device detects that the operator has forgotten to take in an original or a printed matter, the notifying means notifies the image processing device to issue a more emphasized warning than when the person is determined to have a predetermined relationship. A monitoring device comprising:

13. When the determination means determines that the operator does not have a predetermined relationship with the image processing device, the notification means notifies the image processing device to issue a warning when it detects that the operator has left the image processing device.

9. The monitoring device according to claim 8.

14. A monitoring device that monitors whether another person is looking into the contents displayed by an image processing device due to an operation by an operator, a medium on which a drawing is output, or a document to be read, An acquisition means for acquiring person information indicating people present within a predetermined range from the image processing device; A determination means for determining whether or not a person other than the operator among the persons identified by the acquired personal information has a predetermined relationship with the operator; a control means for controlling to output warning information when it is determined that there is even one person among the identified persons who does not have a predetermined relationship, and for controlling to suppress the output of warning information when it is determined that all the identified persons have the predetermined relationship; a notification means for notifying the image processing device of control information indicating whether to output warning information or to suppress output of warning information under the control of the control means; Equipped with When the determining means determines that the person does not have a predetermined relationship with the image processing device, the notifying means notifies the image processing device to issue a warning when security settings are made in the image processing device. A monitoring device comprising:

15. The notification means notifies the user to issue the warning corresponding to the level of the security setting.

15. The monitoring device according to claim 14.

16. the image processing device is an image reading device that reads a document surface and generates image data; When the determining means determines that the person does not have a predetermined relationship, the notifying means notifies the image processing device to issue a warning when security information is detected by reading a document surface in the image processing device.

9. The monitoring device according to claim 8.

17. The notification means issues a notification to issue the warning according to the level of the security information.

17. The monitoring device according to claim 16.

18. The determination means obtains a name of a first organization to which the operator belongs and a name of a second organization to which people other than the operator belong among people specified by the personal information, and determines whether the first organization name and the second organization name match each other to determine whether the people have a predetermined relationship.

18. The monitoring device according to claim 1, 5, or 8 to 17.

19. The determination means determines that the person does not have a predetermined relationship when the first organization name and the second organization name do not match, and determines that the person has a predetermined relationship when the first organization name and the second organization name match.

20. The monitoring device according to claim 18.

20. A predetermined range of an area including the image processing device is photographed, the acquiring means acquires a face image of a person as the personal information from an image acquired by photographing; moreover, a storage means for storing an identifier for identifying a user who uses the image processing device, a face image of the user, and a name of an organization to which the user belongs in association with each other; a readout means for acquiring an operator identifier for identifying the operator, reading out an organization name corresponding to the operator identifier from the storage means as a first organization name, searching the storage means for a face image that matches a face image of a person other than the operator among the face images acquired by the acquisition means, and reading out the organization name corresponding to the face image as a second organization name, The determination means performs the determination using the read first organization name and second organization name.

20. The monitoring device according to claim 18.

21. A person in the vicinity of the image processing device is wearing an employee ID card with the display side showing the name of the organization to which the person belongs facing outward, A predetermined range of an area including the image processing device is photographed, The acquisition means acquires, from the image acquired by photographing, as the personal information, an employee ID image showing a display surface of an employee ID card worn by the person, moreover, a storage means for storing an identifier for identifying a user who uses the image processing device and a name of an organization to which the user belongs in association with each other; a readout means for acquiring an operator identifier for identifying the operator, and reading out an organization name corresponding to the operator identifier from the storage means as a first organization name; an extraction means for extracting an organization name as a second organization name from the employee ID images of persons other than the operator among the employee ID images acquired by the acquisition means; The determination means performs the determination using the read first organization name and the extracted second organization name.

20. The monitoring device according to claim 18.

22. an image capturing device that captures an image of a predetermined range of an area including the image processing device is installed in the vicinity of the image processing device; The acquisition means receives the image from the photographing device.

22. A monitoring device according to claim 20 or 21.

23. A person who is within a predetermined range of the image processing device has a portable authentication device that performs short-distance wireless communication and stores an identifier that identifies the person, The authentication device transmits the identifier by short-range wireless communication, The acquiring means acquires the identifier as the personal information, moreover, a storage means for storing an identifier for identifying a user who uses the image processing device and a name of an organization to which the user belongs in association with each other; a readout means for acquiring an operator identifier for identifying the operator, reading out an organization name corresponding to the operator identifier from the storage means as a first organization name, and reading out an organization name corresponding to an identifier of a person other than the operator from among the identifiers acquired by the acquisition means from the storage means as a second organization name; The determination means performs the determination using the read first organization name and second organization name.

20. The monitoring device according to claim 18.

24. the image processing device includes a communication circuit for wirelessly communicating with the authentication device by short-distance wireless communication; The acquiring means acquires the identifier from the authentication device via the image processing device.

24. The monitoring device according to claim 23.

25. A monitoring system including an image processing device and a monitoring device for monitoring other people's viewing of contents displayed on a display unit by the image processing device due to an operation by an operator, a medium that has been drawn and output and placed on a paper output tray, or a document to be read that has been set on a document tray or a document that has been read and placed on the document output tray, The monitoring device includes: An acquisition means for acquiring person information indicating people present within a predetermined range from the image processing device; A determination means for determining whether or not a person other than the operator among the persons identified by the acquired personal information has a predetermined relationship with the operator; A calculation means for calculating a viewpoint position of a person other than the operator; a control means for controlling to output warning information when it is determined that there is even one person among the specified persons who does not have a predetermined relationship and that the calculated viewpoint position for that person is on the display surface of the display unit, the paper output tray, or the document tray or the document output tray, and for controlling to suppress output of warning information in either case when it is determined that all persons are people who have a specific relationship and when it is determined that the calculated viewpoint position is different from the display surface of the display unit, the paper output tray, the document tray, or the document output tray; a notification unit that notifies the image processing device of control information indicating whether to output warning information or to suppress output of warning information under the control of the control unit, The image processing device includes: A receiving means for receiving the control information; a control means for controlling to output the warning information when the received control information indicates to output the warning information, and for suppressing the output of the warning information when the received control information indicates to suppress the output of the warning information; and an output unit that outputs the warning information only when the received control information indicates that the warning information is to be output. A monitoring system comprising:

26. A monitoring method used in a monitoring device that monitors whether another person is looking at content displayed on a display unit by an image processing device due to an operation by an operator, a medium that has been drawn and output and placed on a paper output tray, or a document to be read that is set on a document tray or a document that has been read and placed on the document output tray, comprising: an acquisition step of acquiring person information indicating people present within a predetermined range from the image processing device; a determination step of determining whether or not a person other than the operator among the people identified by the acquired personal information has a predetermined relationship with the operator; A calculation step of calculating a viewpoint position of a person other than the operator; a control step of controlling to output warning information when it is determined that there is even one person among the specified persons who does not have a predetermined relationship and that the calculated viewpoint position for that person is on the display surface of the display unit, the paper output tray, or the document tray or the document output tray, and controlling to suppress output of warning information in either case when it is determined that all persons have a specific relationship and when it is determined that the calculated viewpoint position is different from the display surface of the display unit, the paper output tray, the document tray, or the document output tray; A monitoring method comprising:

27. ​​A monitoring method for use in a monitoring device that monitors other people's viewing of content displayed by an image processing device due to an operation by an operator, a medium on which a drawing is output, or a document to be read, comprising: an acquisition step of acquiring person information indicating people present within a predetermined range from the image processing device; a determination step of determining whether or not a person other than the operator among the people identified by the acquired personal information has a predetermined relationship with the operator; a control step of controlling to output warning information when it is determined that there is even one person who does not have a predetermined relationship among the identified persons, and controlling to suppress the output of warning information when it is determined that all the persons have the specific relationship; Including, the monitoring device includes a storage means for storing the number of times that the operator and any person identified by the personal information, excluding the operator, have held a conference in the same closed space in the past; The determination step includes reading the number of times from the storage means, comparing the read number of times with a predetermined threshold value, and determining whether or not the person has a predetermined relationship. A monitoring method comprising:

28. A monitoring method used in a monitoring device for monitoring other people's viewing of content displayed by an image processing device due to an operation by an operator, a medium on which a drawing is output, or a document to be read, comprising: an acquisition step of acquiring person information indicating people present within a predetermined range from the image processing device; a determination step of determining whether or not a person other than the operator among the people identified by the acquired personal information has a predetermined relationship with the operator; a control step of controlling to output warning information when it is determined that there is even one person who does not have a predetermined relationship among the identified persons, and controlling to suppress the output of warning information when it is determined that all the persons have the specific relationship; Including, the monitoring device includes a storage means for storing the number of times that the operator and the persons identified by the personal information, excluding the operator, have viewed the same document in the past; The determination step includes reading the number of times from the storage means, comparing the read number of times with a predetermined threshold value, and determining whether or not the person has a predetermined relationship. A monitoring method comprising:

29. A monitoring method for use in a monitoring device that monitors other people's viewing of content displayed by an image processing device due to an operator's operation, a medium on which a drawing is output, or a document to be read, comprising: an acquisition step of acquiring person information indicating people present within a predetermined range from the image processing device; a determination step of determining whether or not a person other than the operator among the people identified by the acquired personal information has a predetermined relationship with the operator; a control step of controlling to output warning information when it is determined that there is even one person who does not have a predetermined relationship among the identified persons, and controlling to suppress the output of warning information when it is determined that all the persons have the specific relationship; Including, When it is not possible to determine whether or not a person other than the operator among the persons specified by the person information has a predetermined relationship, the determining step notifies the image processing device to request the operator to make such a determination. A monitoring method comprising:

30. A monitoring method used in a monitoring device for monitoring other people's viewing of content displayed by an image processing device due to an operation by an operator, a medium on which a drawing is output, or a document to be read, comprising: an acquisition step of acquiring person information indicating people present within a predetermined range from the image processing device; a determination step of determining whether or not a person other than the operator among the people identified by the acquired personal information has a predetermined relationship with the operator; a control step of controlling to output warning information when it is determined that there is even one person who does not have a predetermined relationship among the identified persons, and controlling to suppress the output of the warning information when it is determined that all the persons have the specific relationship; a notification step of notifying the image processing device of control information indicating whether to output warning information or to suppress output of warning information under control of the control step; Including, The image processing device is an image reading device that reads an original and generates image data, or a printing device that produces printed matter, In the notification step, when it is determined in the determination step that the person does not have a predetermined relationship, and when it is detected in the image processing device that the operator has forgotten to take in a document or a printed matter, the image processing device is notified to issue a more emphasized warning than when it is determined that the person has a predetermined relationship. A monitoring method comprising:

31. A monitoring method used in a monitoring device for monitoring other people's viewing of content displayed by an image processing device due to an operator's operation, a medium on which a drawing is output, or a document to be read, comprising: an acquisition step of acquiring person information indicating people present within a predetermined range from the image processing device; a determination step of determining whether or not a person other than the operator among the people identified by the acquired personal information has a predetermined relationship with the operator; a control step of controlling to output warning information when it is determined that there is even one person who does not have a predetermined relationship among the identified persons, and controlling to suppress the output of the warning information when it is determined that all the persons have the specific relationship; a notification step of notifying the image processing device of control information indicating whether to output warning information or to suppress output of warning information under control of the control step; Including, The notification step notifies the image processing device to issue a warning when security settings are made in the image processing device if it is determined in the determination step that the person does not have a predetermined relationship. A monitoring method comprising:

32. A control computer program recorded in a computer-readable recording medium is used in a monitoring device that monitors other people looking at content displayed on a display unit by an image processing device due to an operation by an operator, a medium that has been drawn and output and placed on a paper output tray, or a document to be read that is set on a document tray or a document that has been read and placed on the document output tray, the control computer program being ... recorded in a computer-readable recording medium The surveillance device is a computer. an acquisition step of acquiring person information indicating people present within a predetermined range from the image processing device; a determination step of determining whether or not a person other than the operator among the people identified by the acquired personal information has a predetermined relationship with the operator; A calculation step of calculating a viewpoint position of a person other than the operator; a control step of controlling to output warning information when it is determined that there is even one person among the specified persons who does not have a predetermined relationship and that the calculated viewpoint position for that person is on the display surface of the display unit, the paper output tray, or the document tray or the document output tray, and controlling to suppress output of warning information in either case when it is determined that all persons have a specific relationship and when it is determined that the calculated viewpoint position is different from the display surface of the display unit, the paper output tray, the document tray, or the document output tray; A computer program for executing.

33. A control computer program recorded on a computer-readable recording medium for use in a monitoring device that monitors other people's viewing of content displayed by an image processing device due to an operator's operation, a medium on which a drawing is output, or a document to be read, comprising: The surveillance device is a computer. an acquisition step of acquiring person information indicating people present within a predetermined range from the image processing device; a determination step of determining whether or not a person other than the operator among the people identified by the acquired personal information has a predetermined relationship with the operator; a control step of controlling to output warning information when it is determined that there is even one person who does not have a predetermined relationship among the identified persons, and controlling to suppress the output of warning information when it is determined that all the persons have the specific relationship; The purpose is to execute the following: the monitoring device includes a storage means for storing the number of times that the operator and any person identified by the personal information, excluding the operator, have held a conference in the same closed space in the past; The determination step includes reading the number of times from the storage means, comparing the read number of times with a predetermined threshold value, and determining whether or not the person has a predetermined relationship. A computer program characterized by:

34. A control computer program recorded on a computer-readable recording medium for use in a monitoring device that monitors other people's viewing of content displayed by an image processing device due to an operator's operation, a medium on which a drawing is output, or a document to be read, comprising: The surveillance device is a computer. an acquisition step of acquiring person information indicating people present within a predetermined range from the image processing device; a determination step of determining whether or not a person other than the operator among the people identified by the acquired personal information has a predetermined relationship with the operator; a control step of controlling to output warning information when it is determined that there is even one person who does not have a predetermined relationship among the identified persons, and controlling to suppress the output of warning information when it is determined that all the persons have the specific relationship; The purpose is to execute the following: the monitoring device includes a storage means for storing the number of times that the operator and the persons identified by the personal information, excluding the operator, have viewed the same document in the past; The determination step includes reading the number of times from the storage means, comparing the read number of times with a predetermined threshold value, and determining whether or not the person has a predetermined relationship. A computer program characterized by:

35. A control computer program recorded on a computer-readable recording medium for use in a monitoring device that monitors other people's viewing of content displayed by an image processing device due to an operator's operation, a medium on which a drawing is output, or a document to be read, comprising: The surveillance device is a computer. an acquisition step of acquiring person information indicating people present within a predetermined range from the image processing device; a determination step of determining whether or not a person other than the operator among the people identified by the acquired personal information has a predetermined relationship with the operator; a control step of controlling to output warning information when it is determined that there is even one person who does not have a predetermined relationship among the identified persons, and controlling to suppress the output of warning information when it is determined that all the persons have the specific relationship; The purpose is to execute the following: When it is not possible to determine whether or not a person other than the operator among the persons specified by the person information has a predetermined relationship, the determining step notifies the image processing device to request the operator to make such a determination. A computer program characterized by:

36. A control computer program recorded on a computer-readable recording medium for use in a monitoring device that monitors other people's viewing of content displayed by an image processing device due to an operator's operation, a medium on which a drawing is output, or a document to be read, comprising: The surveillance device is a computer. an acquisition step of acquiring person information indicating people present within a predetermined range from the image processing device; a determination step of determining whether or not a person other than the operator among the people identified by the acquired personal information has a predetermined relationship with the operator; a control step of controlling to output warning information when it is determined that there is even one person who does not have a predetermined relationship among the identified persons, and controlling to suppress the output of the warning information when it is determined that all the persons have the specific relationship; a notification step of notifying the image processing device of control information indicating whether to output warning information or to suppress output of warning information under control of the control step; The purpose is to execute the following: The image processing device is an image reading device that reads an original and generates image data, or a printing device that produces printed matter, In the notification step, when it is determined in the determination step that the person does not have a predetermined relationship, and when it is detected in the image processing device that the operator has forgotten to take in a document or a printed matter, the image processing device is notified to issue a more emphasized warning than when it is determined that the person has a predetermined relationship. A computer program characterized by:

37. A control computer program recorded on a computer-readable recording medium for use in a monitoring device that monitors other people's viewing of content displayed by an image processing device due to an operator's operation, a medium on which a drawing is output, or a document to be read, comprising: The surveillance device is a computer. an acquisition step of acquiring person information indicating people present within a predetermined range from the image processing device; a determination step of determining whether or not a person other than the operator among the people identified by the acquired personal information has a predetermined relationship with the operator; a control step of controlling to output warning information when it is determined that there is even one person who does not have a predetermined relationship among the identified persons, and controlling to suppress the output of the warning information when it is determined that all the persons have the specific relationship; a notification step of notifying the image processing device of control information indicating whether to output warning information or to suppress output of warning information under control of the control step; The purpose is to execute the following: The notification step notifies the image processing device to issue a warning when security settings are made in the image processing device if it is determined in the determination step that the person does not have a predetermined relationship. A computer program characterized by:

Images