Information processing apparatus, computer readable medium, and computer program product

Abstract

Provided are an information processing device and a computer-readable medium. The information processing device includes a processor that performs processing of acquiring a situation around a usage site of a user who uses a service, the service being accompanied by communication with another site, judging a level of security of the usage site with respect to image transmission or sound transmission from the other site to the usage site based on the acquired situation, and judging whether or not to perform communication of image or sound between the usage site and the other site based on the judged level.

Description

Technical Field

[0001] This disclosure relates to information processing apparatus and computer-readable media. Background Technology

[0002] Japanese Patent Application Publication No. 2009-110199 discloses an information display method, characterized by comprising the following steps: an acquisition step, acquiring information including disclosure level information indicating the degree of disclosure of the information; a comparison step, comparing the degree of disclosure of the information indicated by the disclosure level information included in the acquisition step with the degree of disclosure of information that can be displayed on the device; a determination step, determining that the display of the information is permitted when the comparison result in the comparison step meets a predetermined permission condition for allowing the display of the information; and a display step, displaying the information acquired in the acquisition step when the determination step determines that the display of the information is permitted.

[0003] Japanese Patent Application Publication No. 2019-117483 discloses an information processing apparatus that controls the display of information on a screen. The apparatus is characterized by comprising: a determining unit that determines its own place of use; a calculation unit that calculates the risk level related to information leakage in the place of use determined by the determining unit; and a display unit that sets the display information to non-display based on the risk level calculated by the calculation unit. Summary of the Invention

[0004] When a service provider offers a user a service that includes video or audio communication over a network, there is concern that the service provider may provide the video or audio without knowing the confidentiality of the location where the user is receiving the service.

[0005] The purpose of this disclosure is to provide an information processing apparatus and a computer-readable medium that, by determining the security of the place of use, can prevent the disclosure of images or sounds that are not intended to be known by a third party to that third party in the place of use.

[0006] According to a first aspect of this disclosure, an information processing apparatus is provided, wherein the information processing apparatus includes a processor that performs the following processing: obtaining the surrounding conditions of a user's location using a service, wherein the service involves communication with other locations; based on the obtained conditions, determining the security level of the location for image or sound transmission from the other locations to the location using the service; and based on the determined level, determining whether to conduct image or sound communication between the location using the service and the other locations.

[0007] According to the second aspect of this disclosure, the processor further determines whether to perform image and sound communication between the utilization site and the other sites based on the security level of the service requirements.

[0008] According to the third aspect of this disclosure, the security level of the service requirements is determined for image transmission and audio transmission respectively.

[0009] According to the fourth aspect of this disclosure, the processor outputs an instruction to obtain information about the surroundings of the user in the location of use, based on the settings of the camera device in the location of use.

[0010] According to the fifth aspect of this disclosure, when no camera is installed at the location of use, the processor outputs an instruction to the user to photograph the surrounding conditions using a device held by the user, as the acquisition instruction.

[0011] According to the sixth aspect of this disclosure, the processor outputs an instruction to obtain information about the surrounding conditions of the user in the usage area based on the configuration of the sound pickup device in the usage area.

[0012] According to the seventh aspect of this disclosure, when no sound pickup device is installed at the place of use, the processor outputs an instruction to the user to pick up sound from the surroundings using the device held by the user, as the acquisition instruction.

[0013] According to the eighth aspect of this disclosure, the processor performs image recognition processing on images sent from the usage site, thereby determining the security level of the image transmission.

[0014] According to the ninth aspect of this disclosure, the processor transmits images from the other location to the location of use when the security level of image transmission is above a predetermined threshold.

[0015] According to the 10th aspect of this disclosure, the processor provides a method for transmitting images from the other location to the location of use when the security level of image transmission is less than a predetermined threshold.

[0016] According to the 11th aspect of this disclosure, the processor performs sound recognition processing on the sound transmitted from the location of use, thereby determining the security level of the image transmission.

[0017] According to the 12th aspect of this disclosure, the processor transmits sound from the other location to the location of use when the security level of the sound transmission is above a predetermined threshold.

[0018] According to the 13th aspect of this disclosure, based on the information processing apparatus of the 12th aspect, the processor provides a method for transmitting sound from the other location to the location of use when the security level of the sound transmission is less than a predetermined threshold.

[0019] According to the 14th aspect of this disclosure, the processor stops the execution of the service if the security level of the location being used is less than a predetermined threshold.

[0020] According to the 15th aspect of this disclosure, even after the service has been started, if the security level of the access site drops below a predetermined threshold, the processor stops the execution of the service.

[0021] According to the 16th aspect of this disclosure, even after the service has been started, the processor issues a warning when the security level of the site being used drops below a predetermined threshold.

[0022] According to the 17th embodiment of this disclosure, based on the information processing apparatus of any one of the 1st to 16th embodiments, the level of security varies depending on the service.

[0023] According to the 18th aspect of this disclosure, a computer-readable medium is provided, which stores a program for causing a computer to perform processing, wherein the processing includes the following steps: obtaining the surrounding conditions of a user's location using a service, wherein the service involves communication with other locations; determining, based on the obtained conditions, a security level of the location for image or sound transmission from the other locations to the location using the service; and determining, based on the determined level, whether to conduct image or sound communication between the location using the service and the other locations.

[0024] (Effect)

[0025] According to the first scheme, by automatically determining the security of the usage location for both sound and image, it is possible to prevent the leakage of images or sounds that are not intended for third parties to know to third parties in the usage location.

[0026] According to the second scheme, it is possible to determine whether to send images or sounds to the location of use based on whether the security level meets the service requirements.

[0027] According to the third scheme, it is possible to determine whether to send images or sounds to the location of use based on whether the security levels determined for image transmission and sound transmission are met respectively.

[0028] According to the fourth scheme, the level of security can be determined by whether a camera is installed at the location being used, thus indicating an instruction to film the location.

[0029] According to the fifth scheme, if no camera is installed at the location of use, the user can be instructed to take a picture in order to determine the level of security.

[0030] According to the sixth scheme, the level of security can be determined by whether a sound pickup device is installed at the location of use to indicate a shooting instruction.

[0031] According to the seventh scheme, if no sound pickup device is installed at the location of use, the user can be instructed to pick up sound to determine the level of security.

[0032] According to the eighth scheme, the security level of image transmission can be determined by analyzing the images transmitted from the location of use.

[0033] According to the ninth scheme, as a result of image analysis, if the security level is above a threshold, the image can be sent to the location of use.

[0034] According to the 10th scheme, as a result of image analysis, a method for sending images to the usage site can be proposed if the security level is less than a threshold.

[0035] According to the 11th scheme, the security level of image transmission can be determined by analyzing the sound transmitted from the location of use.

[0036] According to the 12th scheme, as a result of sound analysis, images can be sent to the location of use if the security level is above a threshold.

[0037] According to the 13th scheme, as a result of sound analysis, a method for sending sound to the place of use can be proposed if the security level is less than a threshold.

[0038] According to the aforementioned Option 14, if the security levels determined for image transmission and sound transmission are not met, the service of transmitting images or sound to the place of use can be suspended.

[0039] According to the 15th scheme, even after the service is performed, if the security levels determined for image transmission and sound transmission respectively are no longer met, the service that includes the transmission of images or sound to the place of use can be stopped.

[0040] According to the 16th scheme, even after the service is performed, if the security levels determined for image transmission and sound transmission are no longer met, a warning can be issued for the security level deterioration.

[0041] According to the 17th scheme, it is possible to determine whether to send images or sounds to the usage location based on whether the security level required for different services is met.

[0042] According to the 18th solution, by automatically determining the security of the usage location for both sound and image, it is possible to prevent the leakage of images or sounds that are not intended for third parties to know to third parties in the usage location.

[0043] According to this disclosure, an information processing apparatus and computer-readable medium can be provided that can prevent the disclosure of images or sounds that are not intended to be known by third parties to third parties in the use area by automatically determining the security of the use area for sound and images respectively. Attached Figure Description

[0044] Figure 1 This is a diagram illustrating the general structure of the service provision system of this embodiment.

[0045] Figure 2 This is a block diagram illustrating the hardware structure of an information processing device.

[0046] Figure 3 This is a block diagram illustrating an example of the functional structure of an information processing device.

[0047] Figure 4 This is a diagram illustrating an example of information related to the level of security associated with image transmission.

[0048] Figure 5 This is a diagram illustrating an example of information related to the level of security associated with sound transmission.

[0049] Figure 6 This is a flowchart illustrating the security determination process performed by an information processing device.

[0050] Figure 7 This is a flowchart illustrating the security determination process performed by an information processing device.

[0051] Figure 8 This is a flowchart illustrating the security determination process performed by an information processing device.

[0052] Figure 9 This is a flowchart illustrating the security determination process performed by an information processing device.

[0053] Figure 10This is a diagram illustrating an example of a user interface provided by an information processing device.

[0054] Figure 11 This is a diagram illustrating an example of a user interface provided by an information processing device.

[0055] Figure 12 This is a diagram illustrating an example of a user interface provided by an information processing device. Detailed Implementation

[0056] Hereinafter, an example of an embodiment of the present disclosure will be described with reference to the accompanying drawings. Furthermore, the same reference numerals are used for the same or equivalent structural elements and parts in each drawing. Additionally, for ease of explanation, the scale of the drawings is exaggerated and sometimes differs from the actual scale.

[0057] Figure 1 This is a diagram illustrating the general structure of the service provision system of this embodiment. Figure 1 The service provision system shown is one in which a service provider 50, which provides certain services via a network, and a user utilizing the services provided by the service provider 50, exchange images or sounds via a network 60. The user uses a user device 20 when receiving services from the service provider. The user device 20 is, for example, a personal computer, a smartphone, a mobile phone, a tablet computer, or other communication device.

[0058] The services provided by service provider 50 can include a variety of services. For example, these could include online meeting services for conducting meetings, online meeting services for meetings that do not contain confidential information, medical consultation services for consulting about medications, travel booking services for making travel proposals and appointments, and online English conversation services for English conversation courses. Meetings conducted through online meeting services may or may not include confidential information. Furthermore, various attributes are considered among the attributes of users utilizing the services provided by service provider 50. For example, attributes such as meeting users, sales representatives, overseas travelers, pregnant women, or parents with children may be considered.

[0059] When using the services provided by service provider 50, users need to connect to the network from any location. This "any location" is not limited to private places such as one's own home or hotel room. For example, meeting rooms in companies, partitioned meeting spaces, restaurants, cafes, fast food restaurants, and other public places can also be used as locations for using the service. Furthermore, in recent years, shared spaces that can be used through advance reservations have become increasingly common, and these shared spaces can also serve as locations for using the service. Figure 1The diagram shows meeting room 30 and shared space 40, but the locations where users receive services are not limited to these.

[0060] When service provider 50 provides services, knowing the user's location is crucial for the secure and smooth delivery of the service. For example, service provider 50 sometimes provides online conferencing services for handling confidential information. Additionally, for instance, in the case of service provider 50 providing medical consultation services, there is information in the medications used that must be strictly managed. When providing such services, it is essential that no one else be present around the user receiving the service, or that their image or sound be seen or heard by others. However, if images or sounds are sent to a user's location without verifying their location when they request the service, it could potentially lead to the leakage of confidential information.

[0061] In this embodiment, the service provision system includes an information processing device 10 that determines the security level of a user's location. The security level indicates the degree to which images or sounds transmitted through the service provided by the service provider 50 can be provided to the user without being leaked to a third party. The information processing device 10 determines the security level of the location based on images captured or sounds collected at the location of the user receiving or currently receiving the service from the service provider 50. The information processing device 10 determines the security level for both image and sound transmission. Based on the determination of the security level of the user's location, the information processing device 10 decides whether the service provider 50 can transmit images or sounds. Because the information processing device 10 determines the security level for both image and sound transmission, the service provider 50 can confidently transmit images or sounds to the user's location.

[0062] Figure 2 This is a block diagram showing the hardware structure of the information processing device 10.

[0063] like Figure 2 As shown, the information processing device 10 includes a CPU (Central Processing Unit) 11, a ROM (Read-Only Memory) 12, a RAM (Random Access Memory) 13, a memory 14, an input unit 15, a display unit 16, and a communication interface (I / F) 17. All components are connected to each other via a bus 19 in a manner enabling communication.

[0064] CPU 11 is a central processing unit used to execute various programs or control various parts. Specifically, CPU 11 reads programs from ROM 12 or memory 14 and executes the programs using RAM 13 as the working area. CPU 11 performs control and various arithmetic operations on the aforementioned structures according to the programs recorded in ROM 12 or memory 14. In this embodiment, a security determination program is stored in ROM 12 or memory 14, which is used to determine the security level of the location of the user utilizing the service.

[0065] ROM 12 stores various programs and data. RAM 13 serves as the operating area to temporarily store programs or data. Memory 14 is composed of storage devices such as HDD (Hard Disk Drive), SSD (Solid State Drive), or flash memory, and stores various programs and data, including the operating system.

[0066] The input unit 15 includes a mouse and other pointing devices and a keyboard for various input operations.

[0067] The display unit 16 is, for example, a liquid crystal display (LCD) used to display various information. The display unit 16 may also be a touch panel, functioning as an input unit 15.

[0068] Communication interface 17 is an interface used to communicate with other devices such as user equipment 20, for example using standards such as Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark).

[0069] When executing the aforementioned security determination procedure, the information processing device 10 uses the aforementioned hardware resources to implement various functions. The functional structure implemented by the information processing device 10 will be described below.

[0070] Next, the functional structure of the information processing device 10 will be explained.

[0071] Figure 3 This is a block diagram illustrating an example of the functional structure of the information processing device 10.

[0072] like Figure 3 As shown, the information processing device 10 has a security determination unit 101, a service matching unit 102, and a reservation unit 103 as its functional structure. The CPU 11 reads and executes the security determination program stored in the ROM 12 or the memory 14, thereby realizing each functional structure.

[0073] The security determination unit 101 determines the security level of the user's service usage location, that is, the location of the user who is using the service provided by service provider 50 or is currently using the service. The security determination unit 101 obtains the surrounding conditions of the user's location and, based on the obtained surrounding conditions, determines the security level for image and sound transmissions to the user's location, respectively. Figure 3 As shown, the security determination unit 101 includes an image determination unit 111, an audio determination unit 112, a security confirmation unit 113, a confirmation result learning unit 114, a learning data recording unit 115, and an avoidance countermeasure prompt unit 116.

[0074] The image determination unit 111 acquires image data transmitted from the user's location as the surrounding conditions of the location where the user uses the service, analyzes the acquired image data, and thereby determines the security level of the image transmission to the user's location. The image data can be a still image or a moving image. When determining the security level, the image determination unit 111 can also make the determination based on machine learning data recorded in the learning data recording unit 115. The image determination unit 111 can also display the determination result of the security level of the image transmission to the user's location. For example, if a display device such as a monitor is installed at the user's location, the image determination unit 111 can also display the determination result on the display device. Alternatively, if a display device such as a monitor is not installed at the user's location, the image determination unit 111 can also display the determination result on the user device 20.

[0075] Specifically, the image determination unit 111 determines the location of the user based on image data sent from the user's location. Specifically, the image determination unit 111 determines whether the user is in a closed space such as a meeting room or shared space, a room with windows, a partitioned area, or an open area without any partitions. Furthermore, the higher the degree of enclosure of the user's location, the higher the security level determined by the image determination unit 111; conversely, the lower the degree of enclosure, the lower the security level determined by the image determination unit 111.

[0076] When the image determination unit 111 has a camera or other shooting device permanently installed at the user's location, it issues a shooting instruction to the shooting device indicating the user's location. Conversely, when no shooting device is permanently installed at the user's location, it issues an instruction to the user to use the user device 20 to shoot the surroundings. The method of instructing the user device 20 to shoot is not limited. For example, the image determination unit 111 can cause the user device 20 to display a message related to the shooting instruction, or it can cause the user device 20 to output an sound related to the shooting instruction.

[0077] If the image determination unit 111 determines, based on image data, that there is a location that would be a blind spot for the camera, it may also provide additional instructions to film the blind spot. Furthermore, if the image determination unit 111 determines, based on image data, that no wall has been filmed, it may also provide additional instructions to film the user's location until the existence of a wall can be detected.

[0078] Furthermore, if the image determination unit 111 is unable to determine the security level based solely on the image data, it can also have the user darken the room before taking the picture, and determine the security level based on the degree of light entering the room. For example, the image determination unit 111 can also determine the structure of the user's location based on the way external light enters, thereby determining the security level.

[0079] The image determination unit 111 may also determine, based on the security level determination result and the service provided by the service provider 50, whether to conduct image communication between the user's location and other locations different from the user's location.

[0080] For example, the security level determined by the image determination unit 111 is compared with the security level required by the service provided by the service provider 50. If the security level determined by the image determination unit 111 is higher than the required security level, then sending the image to the user's location is permitted. On the other hand, if the security level determined by the image determination unit 111 is lower than the required security level, then sending the image to the user's location is restricted.

[0081] The sound determination unit 112 acquires sound data transmitted from the user's location for a predetermined period of time at the user's location, as a representation of the surrounding conditions of the location where the user intends to utilize the service. Then, the sound determination unit 112 analyzes the acquired sound data to determine the security level of the sound transmission to the user's location. The sound determination unit 112 may also make the determination based on machine learning data recorded by the learning data recording unit 115. The sound determination unit 112 may also display the security level determination result to the user's location. For example, if a monitor or other display device is installed at the user's location, the sound determination unit 112 may display the determination result on the display device. Furthermore, for example, if no monitor or other display device is installed at the user's location, the sound determination unit 112 may display the determination result on the user device 20.

[0082] Specifically, the sound determination unit 112 determines the presence or absence of people around the user based on sound data. The fewer people around the user, the higher the safety level is determined by the sound determination unit 112; the more people or noise around the user, the lower the safety level is determined by the sound determination unit 112.

[0083] When the sound determination unit 112 obtains sound data from the user's location, if a microphone or other sound pickup device is fixedly installed at the user's location, it issues an instruction to that device to maximize the sound pickup level. Conversely, if no sound pickup device is fixedly installed at the user's location, it issues an instruction to the user to perform sound pickup based on maximizing the sound pickup level of the sound pickup device equipped on the user equipment 20. The method of instructing the user equipment 20 to perform sound pickup is not limited. For example, the sound determination unit 112 can cause the user equipment 20 to display a message related to the sound pickup instruction, or it can cause the user equipment 20 to output sound related to the sound pickup instruction.

[0084] The voice determination unit 112 can also determine the security level based on whether there is a difference between the number of people who intend to use or are currently using the service at the user's location and the number of people identified based on the voice data. For example, suppose the user's purpose is a meeting, and two people need to attend the meeting at the user's location. If the voice data identifies two people, the voice determination unit 112 determines the security level to be relatively high. On the other hand, if the voice data identifies three people, the voice determination unit 112 determines the security level to be relatively low.

[0085] The sound determination unit 112 can also determine whether to conduct sound communication between the user's location and other locations different from that location based on the security level determination result and the services provided by the service provider.

[0086] For example, the security level determined by the sound determination unit 112 is compared with the security level required by the service provided by the service provider 50. If the security level determined by the sound determination unit 112 is higher than the required security level, then sound transmission to the user's location is permitted. On the other hand, if the security level determined by the sound determination unit 112 is lower than the required security level, then sound transmission to the user's location is restricted.

[0087] Even if the security level of the user's location is at or above the required level when the service provider 50 begins providing services, it is possible that the security level of the user's location may subsequently drop below the required level. Therefore, the image determination unit 111 and the sound determination unit 112 can determine the security level of the user's location after the service provider 50 begins providing services. Furthermore, if the security level of the user's location drops below the required level, the image determination unit 111 or the sound determination unit 112 may restrict the transmission of images or sounds to the user's location. Alternatively, if the security level of the user's location drops below the required level, the image determination unit 111 and the sound determination unit 112 may issue a prescribed warning to the user.

[0088] If the security level cannot be determined by the image determination unit 111 or the sound determination unit 112, the security verification unit 113 shall have the service provider confirm the security level. The security verification unit 113 shall take the security level confirmed by the service provider's manager by sight or sound as the security level of the user's location.

[0089] The confirmation result learning unit 114 uses image or audio data sent from the user's location and the security level confirmed by the service provider through the security confirmation unit 113 as teacher data to perform machine learning for determining the security level. The confirmation result learning unit 114 updates the machine learning data recorded in the learning data recording unit 115 according to the machine learning results. By implementing machine learning for determining the security level, the confirmation result learning unit 114 can improve the determination accuracy of the image determination unit 111 and the audio determination unit 112.

[0090] For example, if the security level cannot be determined by analyzing image or audio data sent from a certain location, the security level can be determined through manager confirmation. By learning the manager's confirmation result through the confirmation result learning unit 114, the security determination unit 101 is more likely to be able to determine the security level by analyzing image or audio data sent from the same location or a location with the same structure.

[0091] Based on the security level determination results of the image determination unit 111 or the sound determination unit 112, it is possible that the security level required for the service provided by the service provider 50 does not meet the requirements. If the security level of the user's location does not meet the security level required for the service provided by the service provider 50, the avoidance countermeasure prompt unit 116 provides a method for providing services to the user's location as an avoidance countermeasure.

[0092] Figure 4 This is a diagram illustrating an example of information related to the security level of image transmission. A higher value indicates a higher level of security for the image transmission.

[0093] For example, if the image data analysis reveals that the user's location is completely surrounded by walls and a ceiling, the image determination unit 111 determines the security level to be level 4. In this case, services can be provided to the user's location while maintaining this status.

[0094] Furthermore, for example, if, as a result of analyzing image data, it is known that part of the wall surrounding the user is a window but is too far away to see the monitor's view, the image determination unit 111 determines the security level to be level 3. In this case, the avoidance countermeasure prompting unit 116 prompts the user device 20 that the image can be transmitted by changing the angle of the monitor. Furthermore, in this case, the service provider could offer services such as company meetings involving confidential information or medical consultations.

[0095] Furthermore, for example, if, as a result of analyzing image data, it is known that part of the user's surroundings consists of windows and the monitor's screen is visible from outside the windows, the image determination unit 111 determines the security level to be Level 2. In this case, the avoidance countermeasure prompting unit 116 prompts the user equipment 20 to reduce the monitor's screen display or lower the resolution to transmit the image. Furthermore, services that the service provider could offer in this case include, for example, meetings within or outside the company that do not involve confidential information, or window services at a travel agency.

[0096] Furthermore, for example, if the image data analysis reveals that the user's location is not surrounded by walls, the image determination unit 111 determines the security level to be Level 1. In this case, the avoidance countermeasure prompting unit 116 prompts the user device 20 that although communication via image transmission is not possible, communication via voice transmission is possible. In this case, the service provider can consider providing voice communication services. Additionally, for example, in this case, the avoidance countermeasure prompting unit 116 prompts the user device 20 that if it is a service that does not handle confidential information, such as an English conversation course, then normal screen display is not a problem.

[0097] The avoidance countermeasures suggested by the avoidance countermeasure prompting unit 116 are not limited to the examples described above. For example, the avoidance countermeasure prompting unit 116 may also cause the user device 20 to display its screen in a prescribed color such as red, notifying the user of the risk that the screen of the user device 20 may be spied on. Furthermore, the avoidance countermeasure prompting unit 116 may also reduce the risk of information being spied on by decreasing the contrast of the screen. Additionally, the avoidance countermeasure prompting unit 116 may control the display to be in blackout when sending confidential documents. The avoidance countermeasure prompting unit 116 may also determine whether a document is confidential based on whether it has been given a prescribed mark.

[0098] Figure 5 This is a diagram illustrating examples of information related to the security level of voice transmission. A higher value indicates a higher level of security for the voice transmission. Here, an example is shown where the user is located in a conference room.

[0099] For example, if no sounds other than those of people in the meeting room can be heard, the sound determination unit 112 determines the security level to be level 3. In this case, the service can be maintained at the user's location. That is, sound can be output from the speaker when the sound transmission security level is 3.

[0100] Furthermore, for example, if the sound is slightly heard from people other than those in the meeting room, the sound determination unit 112 determines the security level to be Level 2. In this case, the avoidance countermeasure prompt unit 116 prompts the user equipment 20 that the service can only be provided when using headphones or a telephone (including telephone call services) in the case of a company meeting involving confidential information or a medical consultation. Furthermore, in the case of a company meeting or a travel agency window service where confidential information is not involved, the avoidance countermeasure prompt unit 116 prompts the user equipment 20 that the service can only be provided when the speaker volume is lowered to a specified level.

[0101] Furthermore, for example, if the sound is loud or frequently heard by people other than those in the meeting room, the sound determination unit 112 determines the security level to be Level 1. In this case, the avoidance countermeasure prompt unit 116 prompts the user device 20 that if it is a service that does not handle confidential information, such as an English conversation course, then the normal screen display is not a problem. Alternatively, for example, the avoidance countermeasure prompt unit 116 may also prompt the user device 20 that the service can be provided if the sound is converted into characters and output to the screen.

[0102] By proposing methods corresponding to the level of security, the information processing device 10 can reduce the risk of information leakage compared to not taking such methods.

[0103] The service matching unit 102 matches the services provided by the service provider 50 with the users. The service matching unit 102 can also perform matching based on the judgment results of the image judgment unit 111 and the sound judgment unit 112. For example, if the service provided by the service provider is an online English conversation system, then the users are English conversation instructors and students listening to English conversation courses.

[0104] The reservation department 103 accepts reservations from users for services provided by service provider 50. The information included in the reservations accepted by the reservation department 103 may include the time slot, location, number of people, and content of the service.

[0105] The information processing device 10 of this embodiment has Figure 3 The structure shown allows for separate determinations regarding image and audio transmission based on images captured or audio collected at the user's location. Then, the information processing device 10 of this embodiment can determine whether the service provider 50 can transmit images or audio based on the security level of the user's location.

[0106] Next, the function of the information processing device 10 will be explained.

[0107] Figure 6 This is a flowchart illustrating the security determination process performed by the information processing device 10. The CPU 11 reads the security determination program from the ROM 12 or the memory 14, expands it in the RAM 13 and executes it, thereby performing the security determination process.

[0108] Initially, CPU 11 obtains a connection request for the service from the location of the user who wants to utilize the service provided by service provider 50 (step S101).

[0109] After step S101, CPU 11 determines the security level of the image sent to the location of the user who wants to use the service provided by service provider 50 (step S102).

[0110] After step S102, CPU 11 determines the security level of the sound transmission to the user's location (step S103). Furthermore, the order of the image transmission security level determination process and the sound transmission security level determination process can be reversed. Additionally, CPU 11 can perform the image transmission security level determination process and the sound transmission security level determination process in parallel.

[0111] After step S103, CPU 11 determines whether the determined security level is higher than the level required by the service provider's service (step S104).

[0112] As a result of the determination in step S104, if the determined security level is higher than or equal to the level required by the service provided by service provider 50 (step S104; Yes), then CPU 11 allows the service provider to connect to the user's location (step S105). CPU 11 determines whether a connection to the user's location is possible for both image and sound.

[0113] On the other hand, as a result of the determination in step S104, if the determined security level is not higher than the level required by the service provided by the service provider 50 (step S104; no), then the CPU 11 determines whether there is an avoidance measure for the user to accept the service provided by the service provider 50 (step S106).

[0114] If it is determined that an avoidance strategy exists (step S106; Yes), the CPU 11 prompts the user with the avoidance strategy (step S107). Then, the CPU 11 allows the service provider to connect to the user's location (step S105).

[0115] CPU 11 can allow the service provider to connect to the user's location if the user has performed an avoidance measure, or it can allow the service provider to connect to the user's location without waiting for the user to perform an avoidance measure, depending on the user's own responsibility. For example, if CPU 11 prompts the user to perform an avoidance measure such as rotating the monitor installed at the user's location by a specified angle to the left or right, it obtains the rotation angle of the monitor installed at the user's location. Then, CPU 11 can also allow the connection to the user's location after the monitor has actually been rotated by more than the specified angle.

[0116] On the other hand, if it is determined that there is no avoidance countermeasure (step S106; no), CPU 11 refuses to connect the service provider to the user's location (step S108).

[0117] Figure 7 This is a flowchart illustrating the security determination process performed by the information processing device 10. Figure 7 What is shown is Figure 6 The details of step S102, which involves determining the security level of image transmission.

[0118] First, the CPU 11 determines whether a camera is permanently installed at the user's location (step S111). The CPU 11 can determine whether a camera is permanently installed based on information provided by the user, or it can determine the location where the user wants to use the service based on pre-registered information.

[0119] If a camera is permanently installed at the user's location (step S111; Yes), the CPU 11 determines whether the user's surroundings can be determined through the camera's image (step S112). The CPU 11 can determine whether the user's surroundings can be determined through the camera's image based on information provided by the user, or it can determine the location where the user intends to use the service based on pre-registered information. For example, the CPU 11 can also determine whether the user's surroundings can be determined based on whether information such as the ability to control the camera's direction from the information processing device 10 is registered.

[0120] If the user's surroundings can be determined by the camera's capture (step S112; Yes), the CPU 11 sends a capture instruction to the camera (step S113).

[0121] If no camera is permanently installed at the user's location (step S111; no), or if the user's surroundings cannot be determined by camera footage (step S112; no), the CPU 11 determines whether the user's surroundings can be captured by the user device 20 (step S114). The CPU 11 determines whether the user's surroundings can be captured by the user device 20, for example, based on information provided by the user.

[0122] If the user's surroundings can be captured by the user equipment 20 (step S114, yes), then the CPU 11 instructs the user on the capture of the user's surroundings (step S115). The method of instructing the user is not limited to a specific method. For example, the CPU 11 can instruct the user by outputting characters, images, or sounds from the user equipment 20, by displaying characters or images on a monitor located at the user's location, or by outputting sounds from a speaker.

[0123] CPU 11 uses image data captured at the user's location to determine the security level of the image to be sent to the user's location (step S116).

[0124] After step S116, CPU 11 determines whether additional footage of the user's location is needed to determine the security level (step S117). For example, CPU 11 determines whether additional footage is needed based on whether there are locations that were not captured at the user's location.

[0125] If it is determined that additional photography of the user's location is necessary (step S117; Yes), the CPU 11 instructs the user to photograph the surrounding environment (step S115). For example, if the CPU 11 can determine multiple security levels from the image data photographed at the user's location, it determines that additional photography of the user's location is necessary. On the other hand, if it is determined that additional photography of the user's location is not necessary (step S117; No), the CPU 11 determines the security level to be sent to the image of the user's location (step S118).

[0126] On the other hand, as a result of the determination in step S114, if it is impossible to capture the user's surroundings using the user equipment 20 (step S114, No), then the CPU 11 determines the security level of transmitting images to the user's location (step S119). If the security level of transmitting images to the user's location cannot be determined, the CPU 11 may also restrict the transmission of images to that location. Furthermore, if the security level of transmitting images to the user's location cannot be determined, the CPU 11 may also ask the user to confirm whether it is acceptable to transmit images to that location.

[0127] Figure 8 This is a flowchart illustrating the security determination process performed by the information processing device 10. Figure 8 What is shown is Figure 6 The details of step S103, which involves determining the security level of the sound transmission.

[0128] First, the CPU 11 determines whether a microphone is permanently installed at the user's location (step S121). The CPU 11 can determine whether a microphone is permanently installed based on information provided by the user, or it can determine the location where the user wants to use the service based on pre-registered information.

[0129] If a microphone is fixedly installed at the user's location (step S121; Yes), the CPU 11 maximizes the microphone's pickup level (step S122).

[0130] If, on the one hand, no microphone is permanently installed at the user's location (step S121; no), the CPU 11 determines whether it is possible to pick up sound about the user's surroundings through the user device 20 (step S123). The CPU 11 determines whether it is possible to pick up sound about the user's surroundings through the user device 20 based on information provided by the user.

[0131] If the user equipment 20 is able to pick up the surrounding conditions of the user (step S123; Yes), the CPU 11 determines whether the pickup level of the microphone of the user equipment 20 can be maximized remotely (step S124). For example, the CPU 11 may also make the determination in step S124 based on whether it is allowed to change the pickup level of the microphone of the user equipment 20 from the outside.

[0132] If it is possible to remotely maximize the pickup level of the microphone of the user equipment 20 (step S124; Yes), the CPU 11 maximizes the pickup level of the microphone (step S122). On the other hand, if it is not possible to remotely maximize the pickup level of the microphone of the user equipment 20 (step S124; No), the CPU 11 instructs the user to maximize the pickup level of the microphone of the user equipment 20 (step S125). For example, the CPU 11 may instruct the user by outputting characters, images, or sounds from the user equipment 20, by displaying characters or images on a monitor located at the user's location, or by outputting sound from a speaker.

[0133] Next, the CPU 11 picks up the user's surroundings through the microphone (step S126).

[0134] After step S126, CPU 11 uses the sound data collected at the user's location to determine the security level of the sound transmission to the user's location (step S127).

[0135] After step S127, CPU 11 determines whether additional sound pickup is needed at the user's location in order to determine the security level (step S128).

[0136] If it is determined that additional sound pickup is needed at the user's location (step S128; Yes), the CPU 11 returns to step S126 and picks up the surrounding environment of the user again using the microphone. For example, if the CPU 11 can determine multiple security levels from the sound data collected at the user's location, it determines that additional sound pickup is needed at the user's location. On the other hand, if it is determined that additional sound pickup is not needed at the user's location (step S128; No), the CPU 11 determines the security level of the sound transmission to the user's location (step S129).

[0137] On the one hand, as a result of the determination in step S123, if the user equipment 20 cannot pick up the surrounding conditions of the user (step S123, No), then the CPU 11 determines the security level of sound transmission to the user's location (step S130). If the security level of sound transmission to the user's location cannot be determined, the CPU 11 may also restrict sound transmission to that location. Furthermore, if the security level of sound transmission to the user's location cannot be determined, the CPU 11 may also ask the user to confirm whether it is acceptable to transmit sound to that location.

[0138] Figure 9 This is a flowchart illustrating the security determination process performed by the information processing device 10. Figure 9 The image shows the details of how the information processing device 10 processes the results of determining the security level of the learning process.

[0139] If the CPU 11 is unable to determine the security level based on the image and sound data of the user's location, it delegates the confirmation of the security level to the service provider (step S131). The method of delegation is not limited to a specific method.

[0140] After step S131, CPU 11 determines whether the service provider deems it necessary to add video recording or audio recording to confirm the level of security (step S132).

[0141] If the service provider determines that additional video recording or audio recording is required (step S132; Yes), the CPU 11 instructs the user's location to perform camera-based video recording or microphone-based audio recording (step S133). After instructing camera-based video recording or microphone-based audio recording and performing the recording or recording, the CPU 11 returns to step S131 and again requests the service provider to confirm the security level.

[0142] On the other hand, if the service provider determines that no additional filming or audio recording is required (step S132; no), the service provider determines the security level. The CPU 11 obtains the security level determination result determined by the service provider and performs machine learning that uses the image data or sound data and the security level determination result as teacher data (step S134).

[0143] After step S134, CPU 11 updates the learning data using the results obtained from machine learning (step S135). By updating the learning data using the results obtained from machine learning, CPU 11 can improve the likelihood of determining the security level when analyzing subsequent image or sound data sent from the same location or a location with the same structure.

[0144] By performing the actions described above, the information processing device 10 of this embodiment can determine whether to transmit images or audio based on images captured or audio collected at the user's location. Furthermore, the information processing device 10 of this embodiment can determine whether the service provider 50 can transmit images or audio based on the security level of the user's location.

[0145] In the example above, CPU 11 determines the security level of the user's location at the moment the user begins using the service. If the location where the user is using the service is known in advance, CPU 11 can also determine the security level of the location before providing the service. By determining the security level of the location in advance, CPU 11 can inform the user of the security level information before providing the service. Furthermore, by determining the security level of the location in advance, CPU 11 can shorten the time until the service is provided to the user.

[0146] For example, if a user reserves shared space 40 to use the service, CPU 11 can pre-determine the security level of shared space 40. Furthermore, if the security level of shared space 40 is lower than the security level required for the service to be provided to the user, CPU 11 can also notify the user that the security level of shared space 40 is low. The user can then decide whether to directly use the reserved shared space 40 or cancel the reservation and change to another location.

[0147] Next, an example of the user interface provided by the information processing device 10 will be described.

[0148] Figure 10 This is an example of a user interface provided by the information processing device 10. Figure 10 What is shown is Figure 7 Step S115 is an example of a message displayed on a monitor located at the user's location or on the user's device 20 when the user takes pictures of their surroundings. The information processing device 10 provides a prompt. Figure 10 The displayed message instructs the user to take a picture in an easy-to-understand way.

[0149] Figure 11 This is an example of a user interface provided by the information processing device 10. Figure 11 What is shown is Figure 7 In step S117, if the CPU 11 determines that additional shooting is needed, for example, a message displayed on a monitor located at the user's location or on the user device 20 used by the user, the information processing device 10 will prompt... Figure 11 The displayed message instructs the user to take a picture in an easy-to-understand way.

[0150] Figure 12 This is an example of a user interface provided by the information processing device 10. Figure 12 What is shown is that, in Figure 6 When prompting avoidance measures in step S107, for example, a message displayed on a monitor located at the user's location or on the user device 20 used by the user. The information processing device 10 prompts... Figure 12 The displayed message can provide users with easily understandable suggestions for avoidance strategies.

[0151] Alternatively, the security determination process performed by the CPU reading the software (program) in the above-described embodiment can also be executed by various processors other than the CPU. Examples of processors in this case include dedicated circuits that are specifically designed for performing specific processes, such as PLDs (Programmable Logic Devices) with circuit structures that can be changed after manufacturing, like FPGAs (Field-Programmable Gate Arrays), and ASICs (Application Specific Integrated Circuits). Furthermore, security determination processing can be performed by one of these various processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs and a combination of a CPU and an FPGA). More specifically, the hardware structure of these various processors is a circuit obtained by combining circuit elements such as semiconductor components.

[0152] Furthermore, while the above embodiments describe pre-storing (installing) the security determination processing program in a ROM or memory, this is not a limitation. The program may also be provided by recording on a recording medium such as a CD-ROM (Compact Disk Read Only Memory), DVD-ROM (Digital Versatile Disk Read Only Memory), or USB (Universal Serial Bus) memory. Alternatively, the program may be downloaded from an external device via a network.

[0153] Furthermore, the actions of the processor in the above embodiments are not performed by a single processor, but can also be performed collaboratively by multiple processors located in physically separate positions. Moreover, the order of the processor's actions is not limited to the order described in the above embodiments and can be appropriately modified.

Claims

1. An information processing device, wherein, The information processing device includes a processor. The processor performs the following processing: Information about the surrounding environment of a user utilizing the service at a user equipment located at a first location is obtained, wherein the service is accompanied by communication with a service provider located at a second location. Based on the surrounding situation information obtained from image data captured at the first location, a first security level is determined when transmitting images from the second location to the first location. Based on the surrounding situation information obtained from the sound data collected at the first location, a second security level is determined when transmitting sound from the second location to the first location. Based on the determined first security level, it is determined whether to initiate image communication between the first location and the second location. Based on the determined second security level, it is determined whether to conduct audio communication between the first location and the second location.

2. The information processing apparatus according to claim 1, wherein, The processor also determines whether to perform image and sound communication between the first location and the second location based on the security level of the service requirements.

3. The information processing apparatus according to claim 2, wherein, The security level required for the service is determined separately for image transmission and audio transmission.

4. The information processing apparatus according to claim 1, wherein, The processor outputs an instruction to obtain information about the surrounding conditions of the user at the first location, based on the settings of the camera device at the first location.

5. The information processing apparatus according to claim 4, wherein, If no camera is set up at the first position, the processor outputs an instruction to the user to take a picture of the surrounding situation using the device held by the user, as the acquisition instruction.

6. The information processing apparatus according to claim 1, wherein, The processor outputs an instruction to obtain information about the surrounding conditions of the user at the first location, based on the settings of the microphone at the first location.

7. The information processing apparatus according to claim 5, wherein, If no microphone is installed at the first position, the processor outputs an instruction to the user to pick up sound from the surroundings using the device held by the user, as the acquisition instruction.

8. The information processing apparatus according to claim 1, wherein, The processor performs image recognition processing on the image sent from the first location to determine the security level of the image transmission.

9. The information processing apparatus according to claim 8, wherein, The processor transmits an image from the second position to the first position when the security level of the image transmission is above a predetermined threshold.

10. The information processing apparatus according to claim 9, wherein, The processor provides a method for transmitting images from the second location to the first location when the security level of image transmission is less than a predetermined threshold.

11. The information processing apparatus according to claim 1, wherein, The processor performs sound recognition processing on the sound transmitted from the first position, thereby determining the security level of the sound transmission.

12. The information processing apparatus according to claim 1, wherein, The processor transmits sound from the second position to the first position when the security level of the sound transmission is above a predetermined threshold.

13. The information processing apparatus according to claim 12, wherein, The processor provides a method for transmitting sound from the second position to the first position when the security level of the sound transmission is less than a predetermined threshold.

14. The information processing apparatus according to claim 1, wherein, If the processor stops the execution of the service if the security level of the first location is less than a predetermined threshold.

15. The information processing apparatus according to claim 1, wherein, Even after the service has started, if the security level of the first location drops below a predetermined threshold, the processor will stop the execution of the service.

16. The information processing apparatus according to claim 1, wherein, Even after the service has started, the processor issues a warning when the security level of the first location drops below a predetermined threshold.

17. The information processing apparatus according to any one of claims 1 to 16, wherein, The required level of security for the service varies depending on the service.

18. A computer-readable medium storing a program that causes a computer to perform processing, wherein, The process includes the following steps: Information about the surrounding environment of a user utilizing the service at a user equipment located at a first location is obtained, wherein the service is accompanied by communication with a service provider located at a second location. Based on the surrounding situation information obtained from image data captured at the first location, a first security level is determined when transmitting images from the second location to the first location. Based on the surrounding situation information obtained from the sound data collected at the first location, a second security level is determined when transmitting sound from the second location to the first location. Based on the determined first security level, it is determined whether to initiate image communication between the first location and the second location. Based on the determined second security level, it is determined whether to conduct audio communication between the first location and the second location.

19. A computer program product comprising a program that causes a computer to perform processing, wherein, The process includes the following steps: Information about the surrounding environment of a user utilizing the service at a user equipment located at a first location is obtained, wherein the service is accompanied by communication with a service provider located at a second location. Based on the surrounding situation information obtained from image data captured at the first location, a first security level is determined when transmitting images from the second location to the first location. Based on the surrounding situation information obtained from the sound data collected at the first location, a second security level is determined when transmitting sound from the second location to the first location. Based on the determined first security level, it is determined whether to initiate image communication between the first location and the second location. Based on the determined second security level, it is determined whether to conduct audio communication between the first location and the second location.

Images