Information processing device and program

Abstract

To enable the conversions of people present around an apparatus to be more reliably recorded by sound recording means that records the operating sound of the apparatus, as compared with the case where the presence state of people around the apparatus is not taken into account.SOLUTION: A multifunction machine 10 includes a sound recording control unit 33 that controls sound recording by a microphone 21 that collects operating sounds in order to detect abnormality from the operating sounds. The sound recording control unit 33 starts recording sounds when job execution begins. When a plurality of people are detected around the multifunction machine 10 by a human presence sensor or a camera, the sound recording control unit 33 stops recording upon estimating that there is the possibility of a conversion taking place due to the fact that the presence state of the plurality of people matches a conversion estimation condition. When the number of people detected around the multifunctional machine 10 decreases to one or less, the sound recording control unit 33 estimates that there is no possibility of a conversion taking place around the multifunctional machine 10 and causes recording to restart.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an information processing apparatus and a program.

Background Art

[0002] For example, as one method for detecting an abnormality in a device such as a multifunction device, there is a method that uses a sound collection means such as a microphone. Specifically, the sound collection means is attached inside the device, the sound inside the device is collected, and the abnormality is detected by analyzing the sound. By determining in advance the attachment position of the sound collection means, it is also possible to specify the abnormal location.

[0003] By the way, if the device is used by a customer with the sound collection means attached, it is difficult to be accepted by the customer due to the risk of wiretapping. Therefore, conventionally, a method has been proposed in which recording is being performed or the state of being able to record is notified by a warning sound (for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] By the way, when a plurality of people talk around the device, there is a possibility that the sound collection means that collects the operation sound of the device also collects and records the content of the conversation together with the operation sound of the device. In other words, it is also considered that by checking the location status of the people around the device and if there are no plurality of people, there is a situation where no conversation is being made, that is, there is no possibility of recording the content of the conversation.

[0006] The present invention aims to more reliably prevent the recording of conversations of people in the vicinity of a device by using sound-collecting means to capture the operating sounds of the device, compared to cases where the presence of people in the vicinity of the device is not taken into consideration. [Means for solving the problem]

[0007] The information processing device according to the present invention includes a processor, and the processor has a function to record sounds picked up by a sound-collecting means for picking up the operating sounds of a device. The device controls the recording of the operating sounds according to the presence of people around the device, and detects people around the device from data obtained from a detection means for detecting the surrounding conditions of the device, and controls the recording of the operating sounds according to the result of the detection. death , If multiple people are detected from the aforementioned data, the system will control the recording of the aforementioned operating sounds. It is characterized by the following:

[0009] Furthermore, the processor is characterized in that, even if multiple people are detected, if the situation of the people around the device does not match the conversation estimation conditions for estimating that they are having a conversation, it will record the operating sound.

[0010] Furthermore, the conversation estimation condition is characterized in that the distance between people around the device is within a predetermined distance.

[0011] Furthermore, the conversation estimation condition is characterized by the fact that multiple people are continuously present in the vicinity of the device for a predetermined period of time or longer.

[0012] In another aspect, the information processing device according to the present invention comprises a processor, and the processor has a function of recording sounds picked up by a sound-collecting means for picking up the operating sounds of a device, and controls the recording of the operating sounds according to the presence of people around the device, and performs detection of people around the device from data obtained from a detection means for detecting the surrounding conditions of the device, and controls the recording of the operating sounds according to the result of the detection, The system is characterized by estimating the number of people around the device from data acquired from a detection means that detects the surrounding conditions of the device, and controlling the system to record the operating sound if the number of detected people is one or less.

[0013] The present invention provides a program for a computer that has a function to record sounds picked up by a sound-collecting means for picking up the operating sounds of a device. The program controls the recording of the operating sounds according to the presence of people around the device, and also detects people around the device from data obtained from a detection means for detecting the surrounding conditions of the device, and controls the recording of the operating sounds according to the result of the detection. Furthermore, if multiple people are detected from the aforementioned data, the system will control itself to not record the aforementioned operating sound. To implement the function. In another respect, the program according to the present invention provides a function in which, when a computer has a function to record sounds picked up by a sound-collecting means for picking up the operating sounds of a device, the program controls the recording of the operating sounds according to the presence of people around the device, detects people around the device from data obtained from a detection means for detecting the surrounding conditions of the device, controls the recording of the operating sounds according to the result of the detection, estimates the number of people around the device from data obtained from the detection means for detecting the surrounding conditions of the device, and controls the program to record the operating sounds if one or fewer people are detected. [Effects of the Invention]

[0014] According to the invention described in claim 1, compared to the case where the location of people around the device is not taken into consideration, the sound-collecting means for collecting the operating sound of the device can more reliably prevent the recording of conversations of people around the device.

[0015] According to the invention described in claim 2, the possibility of recording conversations of people in the surrounding area can be eliminated.

[0016] According to the invention described in claim 3, if it is not presumed that people in the vicinity of the device are having a conversation, the operating sound of the device can be recorded.

[0017] According to the invention described in claim 4, if it can be presumed that people are not conversing because they are in different locations, the operating sounds of the device can be recorded.

[0018] According to the invention described in claim 5, the operating sound of the device can be recorded when it is presumed that no conversation is taking place, such as when simply passing near the device.

[0019] According to the invention described in claim 6, when no conversation occurs, the operating sounds of the device can be recorded.

[0020] According to the invention described in claim 7, compared with the case where the location status of a person around the device is not considered, it is possible to more surely prevent the conversation of a person around the device from being recorded by the sound collection means that collects the operating sound of the device.

Brief Description of Drawings

[0021] [Figure 1] FIG. 8 is a diagram showing an example of the hardware configuration of the multifunction device in the present embodiment. [Figure 2] FIG. 11 is a diagram showing an example of the block configuration of the multifunction device in the present embodiment. [Figure 3] FIG. 14 is a flowchart showing the recording process in the present embodiment.

Embodiments for Carrying Out the Invention

[0022] Hereinafter, preferred embodiments of the present invention will be described based on the drawings.

[0023] FIG. 1 is a diagram showing an example of the hardware configuration of the multifunction device in the present embodiment. In the present embodiment, a multifunction device equipped with a plurality of functions will be described as an example of the device.

[0024] The multifunction device 10 is an image forming device equipped with various functions such as printing, copying, and scanning, and is a device that incorporates a computer, which is an information processing device. In Figure 1, the ROM 12 stores various programs related to the control of this device, encryption of electronic data, and transmission and reception of electronic data, and the CPU 11 controls the operation of various mechanisms installed in this device, such as the scanner 16 and printer 17, according to the programs stored in the ROM 12. The RAM 13 is used for programs to be executed and as work memory when programs are executed. The HDD (Hard Disk Drive) 14 stores electronic documents read using the scanner 16. The operation panel 15 receives instructions from the user and displays information. The scanner 16 reads the original document set by the user and stores it as electronic data in the HDD 14, etc. The printer 17 prints images on output paper according to instructions from the control program executed by the CPU 11. The network interface (IF) 18 connects to a network and is used for data exchange with external devices and access to this device via a browser. The input / output interface (IF) 19 is used to connect input or output devices. The address data bus 20 connects to various mechanisms controlled by the CPU 11 to communicate data.

[0025] In this embodiment, a microphone (hereinafter referred to as "microphone") 21, a human presence sensor 22, and a camera 23 are connected to the input / output interface 19. The microphone 21 is a sound-collecting means for capturing sound, and in this embodiment, it is provided for the purpose of capturing sounds generated by the operation of the multifunction device 10 (hereinafter referred to as "operational sounds"). However, depending on the performance of the microphone 21 and the mounting position of the microphone 21, the microphone 21 may also capture ambient sounds in the vicinity of the multifunction device 10, such as conversations taking place near the multifunction device 10, in addition to the operating sounds of the multifunction device 10.

[0026] In this embodiment, a human presence sensor 22 and a camera 23 are provided on the multifunction printer 10 as detection means for detecting the surrounding environment. In particular, the detection means are designed to detect people and the number of people around the multifunction printer 10. The human presence sensor 22 is a sensor means that detects and reacts to people, and for example, uses infrared light to detect people around the multifunction printer 10. The camera 23 is a shooting means that takes pictures of the area around the multifunction printer 10. In this embodiment, the camera 23 takes moving images. The multifunction printer 10 has the function of detecting people and the number of people around the multifunction printer 10 by analyzing the captured data from the camera 23. Furthermore, it may also have a function to identify the direction of faces and the movement of mouths.

[0027] Figure 2 shows an example of the block configuration of the multifunction device 10 in this embodiment. The multifunction device 10 includes a control unit 31, a job execution unit 32, a recording control unit 33, a display control unit 34, a data transmission unit 35, and a storage unit 36. Components not used in the description of this embodiment have been omitted from the figure.

[0028] The control unit 31 works in conjunction with other components to control the entire multifunction printer 10. The job execution unit 32 executes jobs in response to user operations from the operation panel 15 or instructions via the network. The recording control unit 33 has the function of recording sounds picked up by the microphone 21, and in particular controls the recording of the operating sounds of the multifunction printer 10 according to the presence of people around the multifunction printer 10. The display control unit 34 controls the display on the operation panel 15. The data transmission unit 35, under the control of the recording control unit 33, transmits the recorded data (hereinafter referred to as "recorded data") to a server (not shown) via the network interface 18. The server has the function of detecting abnormalities in the multifunction printer 10 by analyzing the recorded data and identifying the location of the abnormality.

[0029] The memory unit 36 ​​stores the recorded data. The recorded data is formed by adding additional information such as time-related information (recording date and time or recording period) and information about the executed job to signal data representing the sound picked up by the microphone 21. However, in this embodiment, there is no need to treat signal data and recorded data as particularly distinct, so signal data and recorded data are used synonymously.

[0030] Each component 31-35 in the multifunction printer 10 is realized through the coordinated operation of the computer installed in the multifunction printer 10 and the program running on the CPU 11 included in the computer. The storage unit 36 ​​is realized by the HDD 14 installed in the multifunction printer 10. Alternatively, RAM 13 or external storage means may be used via a network.

[0031] Furthermore, the program used in this embodiment can be provided not only by communication means, but also by being stored on a computer-readable recording medium such as a USB memory stick. The program provided by the communication means or recording medium is installed on the computer, and various processes are realized by the computer's CPU 11 sequentially executing the program.

[0032] By the way, in order to detect abnormalities in the multifunction printer 10 from its operating sounds, it is preferable for the service provider to attach the microphone 21 to the multifunction printer 10 and keep it in a state where it can constantly record the operating sounds. On the other hand, from the customer's perspective, if the microphone 21 is attached to the multifunction printer 10, there is a possibility that the microphone 21 will pick up the content of conversations around the multifunction printer 10, i.e., there is a risk of eavesdropping, so they may be reluctant to accept it and would like to avoid leaving the microphone 21 attached if possible. However, in other words, if the content of conversations taking place around the multifunction printer 10 is not recorded, the customer is more likely to accept having the multifunction printer 10 with the microphone 21 attached installed in their company.

[0033] Therefore, in this embodiment, the recording of operating sounds of the multifunction printer 10 is controlled according to the presence of people around the multifunction printer 10. In other words, since conversations take place between multiple people, recording is disabled if there is a possibility that multiple people are around the multifunction printer 10 and are having a conversation. The recording process in this embodiment will be explained below using the flowchart shown in Figure 3.

[0034] The user instructs the execution of a job by performing a predetermined operation on the operation panel 15 of the multifunction device 10 in order to use the functions provided by the multifunction device 10, such as copying. The job uses a function provided by the multifunction device 10, such as copying. When the control unit 31 detects that the user has instructed the execution of a job (step 101), it activates the microphone 21 and starts recording sound (step 102).

[0035] Here, "sound collection" generally refers to receiving sound in a sound field with a microphone and capturing it as an electrical signal. In this embodiment, "sound collection" has basically the same meaning, but it refers to temporarily storing the signal data generated by capturing the sound as an electrical signal in the work memory (also called a "buffer") within the RAM 13. Also, "recording" generally refers to recording voice, music, sound, etc., on various media, or the sound itself. In this embodiment, "recording" has basically the same meaning, and it refers to recording the signal data from the microphone 21, which has been collected and stored in the work memory, onto a medium (in this embodiment, the "storage unit 36"). In other words, "recording" involves saving the signal data from the microphone 21, while "sound collection" is limited to collecting sound from the microphone 21 and does not include saving the signal data.

[0036] By the way, the motion sensor 22 and camera 23, which are provided as means of detecting people, need to be running during job execution, so the control unit 31 starts them if they are not running. If the motion sensor 22 and camera 23 are running for other functions, the sensor data from the motion sensor 22 and the captured data from the camera 23 may be used as is.

[0037] When sound recording begins, the job execution unit 32 executes the job specified by the user (step 103). Generally, malfunctions of the multifunction printer 10 often occur when a job is started, that is, when components such as motors included in the multifunction printer 10 start to operate, due to the load on those components. Therefore, in this embodiment, the operating sounds of the multifunction printer 10 are recorded even before the job is started.

[0038] As the job execution begins, the recording control unit 33 starts controlling the recording. In other words, the recording control unit 33 starts recording when the job execution begins (step 104). More specifically, the recording control unit 33 stores the sound picked up by the microphone 21 in the storage unit 36. However, since the signal data is temporarily stored in the work memory before the job execution starts, the signal data is read sequentially from the work memory and stored in the storage unit 36.

[0039] Here, the recording control unit 33 detects the number of people around the multifunction printer 10 by analyzing at least one of the sensor data from the human presence sensor 22 or the captured data from the camera 23 (step 105). Basically, it is assumed that one person who has instructed the multifunction printer 10 to execute a job by operating the operation panel 15 will be detected. Alternatively, if the execution of a job is instructed via a network, the person who instructed the job will not be detected around the multifunction printer 10. Also, in this embodiment, "around the multifunction printer 10" refers to the vicinity of the multifunction printer 10, as is clear from the above description, and indicates the range in which the microphone 21 may pick up conversations. Therefore, the detection range of the detection means needs to be set so that it can detect only people around the multifunction printer 10. Alternatively, the recording control unit 33 needs to have a function to extract people who can be considered to be around the multifunction printer 10 from among the people detected by the detection means. Conversations may take place between the person who instructed the job standing in front of the operation panel 15 of the multifunction printer 10, or they may take place between people other than the person who instructed the job.

[0040] If the number of detected people is one or less (Y in step 106), the recording control unit 33 determines that there is no possibility of conversation taking place around the multifunction device 10. As will be described later, in this embodiment, recording may be stopped, but otherwise, recording should be continued. In other words, if recording is in progress, no processing is required.

[0041] If the job is currently running (N in step 108), the process returns to step 105. On the other hand, if the job has finished (Y in step 108), the recording control unit 33 stops recording (step 112). Also, since the microphone 21 does not record sound even when it is picking up sound, if the microphone 21 is not being used by other functions, sound pickup may be stopped by stopping the operation of the microphone 21.

[0042] On the other hand, if two or more people are detected (N in step 106), the system verifies whether the location of people around the multifunction printer 10 matches a predetermined conversation estimation condition. For example, a conversation estimation condition refers to a situation in which it can be estimated that two or more people are having a conversation. If the location of people around the multifunction printer 10 matches the predetermined conversation estimation condition, and people are actually having a conversation, the conversation will be picked up by the microphone 21, depending on the volume of their voices. For this reason, the recording control unit 33 in this embodiment controls the system not to record if the location of people around the multifunction printer 10 matches the situation set in the conversation estimation condition. "Not recording" means stopping the recording if it is already in progress, and maintaining the state of not recording if it is not already in progress. On the other hand, "recording" means maintaining the state of recording if it is already in progress, and starting (or resuming) recording if it is not already in progress.

[0043] For example, one condition for estimating a conversation is set to be when two or more people are present around the multifunction printer 10 for a predetermined period of time, for example, two seconds or more. If two or more people are present for two seconds or more, it is considered possible that they are standing still and talking. On the other hand, if the state of two or more people lasts for less than two seconds, it is presumed that, for example, a person simply passed behind the operator and was detected by the detection means. Another condition for estimating a conversation is set to be when the distance between people around the multifunction printer 10 is within a predetermined distance, for example, the distance between people is within 2m. If two or more people are within 2m of each other, it is considered possible that they are approaching each other and talking. On the other hand, if the distance between two people exceeds 2m, it is presumed that a person standing still around the multifunction printer 10 was detected, regardless of whether they are the person giving the job instruction. Note that other situations involving people may also be set as conditions for estimating a conversation. For example, conditions such as when people's mouths are moving or when their faces are facing each other may be combined as appropriate.

[0044] Thus, if, as a result of analyzing the sensor data from the human presence sensor 22 or the captured data from the camera 23, the location of people around the multifunction printer 10 does not match the conversation estimation conditions (N in step 109), that is, if it can be estimated that no conversation is taking place, the recording control unit 33 continues recording the operating sounds of the multifunction printer 10 without stopping the recording. Then, the process proceeds to step 108.

[0045] On the other hand, if the location of people around the multifunction device 10 matches the conversation estimation condition (N in step 109), that is, if it can be estimated that a conversation may be taking place, the recording control unit 33 stops recording (step 110). If recording is already stopped, that state is maintained. Then, the display control unit 34 displays on the operation panel 15 that recording is stopped, in response to instructions from the recording control unit 33 (step 111). If it is already displaying that recording is stopped, that display is maintained. After that, the process proceeds to step 108.

[0046] The process described above (steps 105-111) is repeated until the job is completed. If multiple people are detected while the process is being repeated, and the number of detected people changes from multiple to one or less, the recording control unit 33 resolves the recording stop state and resumes recording (step 107).

[0047] Generally, malfunctions in the multifunction printer 10 can occur when components such as motors in the multifunction printer 10 operate continuously for a long period of time, resulting in prolonged load on those components or the accumulation of load on those components. For example, noise may start to occur when making a large number of copies, such as 100 or more, in succession. Therefore, in this embodiment, even if recording is temporarily stopped, if the number of people around the multifunction printer 10 becomes one or less and recording becomes possible again, recording will resume.

[0048] Furthermore, the sound picked up by the microphone 21 may be temporarily stored in the work memory even when recording is stopped. However, the recording control unit 33 does not store the signal data that was picked up and temporarily stored while the conversation estimation conditions were met in the storage unit 36. The human presence sensor 22 and the camera 23 need to detect people around the multifunction printer 10, so they operate continuously at least while the job is running.

[0049] The data transmission unit 35 sends the recorded data saved during job execution to the server as described above. The server identifies the abnormal location of the multifunction printer 10 by analyzing the recorded data, but as mentioned above, the recorded data referenced when identifying the abnormal location does not contain human conversation.

[0050] According to this embodiment, the location of people around the multifunction printer 10 is taken into consideration, making it possible to more reliably avoid recording conversations of people around the multifunction printer 10.

[0051] As mentioned above, in this embodiment, recording is controlled so as not to record conversations of people around the multifunction printer 10. However, the signal data collected during job execution may be temporarily saved, and the saved signal data may be deleted for the period that matches the conversation estimation conditions. Of course, it is necessary to control access so that the signal data saved for the period that matches the conversation estimation conditions cannot be accessed by third parties.

[0052] In this embodiment, a multifunction printer 10 was used as an example of a device that generates operating noise, but the device is not limited to a multifunction printer 10. For example, this embodiment may be applied to any device installed in a location where the risk of eavesdropping occurs. Examples of such devices include ATMs (Automated Teller Machines).

[0053] In the above embodiment, the term "processor" refers to a processor in a broad sense, and includes general-purpose processors (e.g., CPU: Central Processing Unit, etc.) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.).

[0054] Furthermore, the operation of the processor in the above embodiments may not be performed by a single processor, but may be performed by multiple processors located in physically separate locations working together. Also, the order of each processor operation is not limited to the order described in the above embodiments, but may be changed as appropriate. [Explanation of symbols]

[0055] 10 Multifunction printer, 11 CPU, 12 ROM, 13 RAM, 14 HDD, 15 Control panel, 16 Scanner, 17 Printer, 18 Network interface (IF), 19 Input / output interface (IF), 20 Address data bus, 21 Microphone, 22 Human presence sensor, 23 Camera, 31 Control unit, 32 Job execution unit, 33 Recording control unit, 34 Display control unit, 35 Data transmission unit, 36 Storage unit.

Claims

1. Equipped with a processor, The aforementioned processor, In a case where the device has a function to record sounds picked up by a sound-collecting means for picking up the operating sounds of the device, the recording of the operating sounds is controlled according to the presence of people in the vicinity of the device, Based on data acquired from a detection means that detects the surrounding conditions of the device, the system performs detection of people in the vicinity of the device. Depending on the results of the detection, the recording of the operating sound is controlled. If multiple people are detected from the aforementioned data, the system will control the recording of the aforementioned operating sounds. An information processing device characterized by the following:

2. The information processing apparatus according to claim 1, characterized in that the processor is controlled to record the operating sound if, even when multiple people are detected, the situation of the people around the device does not match the conversation estimation conditions for estimating that they are having a conversation.

3. The information processing apparatus according to claim 2, characterized in that the conversation estimation condition is that the distance between people in the vicinity of the device is within a predetermined distance.

4. The information processing apparatus according to claim 2, characterized in that the conversation estimation condition is when multiple people are continuously present in the vicinity of the device for a predetermined period of time or longer.

5. comprising a processor, The aforementioned processor, In a case where the device has a function to record sounds picked up by a sound-collecting means for picking up the operating sounds of the device, the recording of the operating sounds is controlled according to the presence of people in the vicinity of the device, Based on data acquired from a detection means that detects the surrounding conditions of the device, the system performs detection of people in the vicinity of the device. Depending on the results of the detection, the recording of the operating sound is controlled. The number of people around the device is estimated from the data obtained from the detection means that detects the conditions around the device. If one or fewer people are detected, the system will be controlled to record the aforementioned operating sound. An information processing device characterized by the following:

6. On the computer, In a case where the device has a function to record sounds picked up by a sound-collecting means for picking up the operating sounds of the device, the recording of the operating sounds is controlled according to the presence of people in the vicinity of the device, Based on data acquired from a detection means that detects the surrounding conditions of the device, the system performs detection of people in the vicinity of the device. Depending on the results of the detection, the recording of the operating sound is controlled. If multiple people are detected from the aforementioned data, the system will control the recording of the aforementioned operating sounds. A program to implement a function.

7. A computer, In a case where the device has a function to record sounds picked up by a sound-collecting means for picking up the operating sounds of the device, the recording of the operating sounds is controlled according to the presence of people in the vicinity of the device, Based on data acquired from a detection means that detects the surrounding conditions of the device, the system performs detection of people in the vicinity of the device. Depending on the results of the detection, the recording of the operating sound is controlled. The number of people around the device is estimated from the data obtained from the detection means that detects the conditions around the device. If one or fewer people are detected, the system will be controlled to record the aforementioned operating sound. A program to implement a function.

Images