Image display system, shooting device, shooting processing method, and program

Abstract

To solve a problem that it is difficult to provide an other base with a processed screen undergoing prescribed processing treatment as needed when the other base shares a picked-up photographic image of a self-base in a conventional technology.SOLUTION: A photographing device 3 picks up the circumference of an own device to acquire a photographic image of a self-base (S14), receives setting of a prescribed display mode for displaying the photographic image (S40), generates preview screen data representing a preview screen of a processed image obtained by performing a prescribed processing treatment to the photographic image, displayable by another base terminal (a communication terminal 7B) used at another base sharing the photographic image of the self-base on the basis of the prescribed display mode and an image processing setting file for performing the prescribed processing treatment to the photographic image (S41), and transmits the preview screen data to a self-base terminal (a communication terminal 7A) so as to display the preview screen on the other base terminal (the communication terminal 7B) (S42).SELECTED DRAWING: Figure 20

Description

[Technical field]

[0001] The present invention relates to an image management system, an image capturing device, an image processing method, and a program. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there has been known a technique for displaying a captured image to others via a communication network.

[0003] For example, a technology is known in which the system control device 40 pre-defines security priority areas and privacy priority areas for the monitoring areas 91-97, and when it detects an alarm from the monitoring sensor 10, it allows the images within the security priority areas 91-95 to be displayed unprocessed on the display unit 35, but prohibits the images within the privacy priority areas 96, 97 from being displayed unprocessed on the display unit (see Patent Document 1). Summary of the Invention [Problem to be solved by the invention]

[0004] However, with conventional technology, when sharing images taken at one's own location with another location, there is a problem in that it is difficult to provide the other location with an edited screen that has been processed as necessary. [Means for solving the problem]

[0005] In order to solve the above-mentioned problems, the invention of claim 1 provides an image management system having an imaging device, the imaging device having an imaging acquisition means for acquiring an image of the imaging device at its own base by imaging the surroundings of the imaging device, a reception means for receiving a setting of a predetermined display mode for displaying the captured image, a preview screen data generation means for generating preview screen data representing a preview screen of a processed image obtained by subjecting the captured image to the predetermined processing, which can be displayed on a other base terminal used at a other base that shares the captured image of the own base, based on the predetermined display mode and an image processing setting file for subjecting the captured image to the predetermined processing, and a preview screen data transmission means for transmitting the generated preview screen data so that the preview screen is displayed on the other base terminal. Effect of the Invention

[0006] As described above, according to the present invention, when an image taken at one's own location is shared with another location, it is possible to provide the other location with an edited screen that has been subjected to a specified processing as necessary. [Brief description of the drawings]

[0007] [Figure 1] 1 is a diagram showing an example of an overall configuration of an image display system according to an embodiment; [Diagram 2] 1A and 1B are diagrams showing an example of the shooting range of an imaging device according to an embodiment, in which (a) is an example of shooting 360° around the imaging device (device), and (b) is an example of shooting a specified angle above and below the horizontal 0° of the imaging device (device). [Diagram 3] 4A to 4C are diagrams illustrating an example of a panoramic image and a speaker image cut out according to the embodiment. [Figure 4] 1 is a conceptual diagram showing an example of a flow of a predetermined processing process executed on a captured image according to an embodiment. [Diagram 5] 1 is a conceptual diagram showing an example of a relationship between resolution and object distance according to an embodiment. [Figure 6A]6A to 6C are diagrams illustrating examples of display screens for different display modes according to the embodiment. [Figure 6B] 13A and 13B are diagrams illustrating other examples of display screens for different display modes according to the embodiment. [Figure 6C] 13A and 13B are diagrams illustrating other examples of display screens for different display modes according to the embodiment. [Figure 6D] 13A and 13B are diagrams illustrating other examples of display screens for different display modes according to the embodiment. [Figure 6E] 13A and 13B are diagrams illustrating other examples of display screens for different display modes according to the embodiment. [Figure 7] FIG. 2 is a diagram illustrating an example of a hardware configuration of an imaging apparatus according to an embodiment. [Figure 8] FIG. 1 is a diagram illustrating an example of a hardware configuration of an image management device according to an embodiment. [Figure 9] FIG. 2 is a diagram illustrating an example of a hardware configuration of a communication terminal according to an embodiment. [Figure 10A] FIG. 1 is a diagram illustrating an example of a functional configuration of an image display system according to an embodiment. [Figure 10B] FIG. 1 is a diagram illustrating an example of a functional configuration of an image display system according to an embodiment. [Figure 11] 10 is a conceptual diagram illustrating an example of a photographing device information management table according to the embodiment. FIG. [Figure 12] 10 is a conceptual diagram illustrating an example of a captured image information management table according to the embodiment. FIG. [Figure 13] FIG. 4 is a conceptual diagram illustrating an example of a destination information management table according to the embodiment. [Figure 14] 10 is a conceptual diagram illustrating an example of a captured image information management table according to the embodiment. FIG. [Figure 15] 1 is a conceptual diagram illustrating an example of a confidential information management table according to an embodiment. [Figure 16] 11 is a conceptual diagram illustrating an example of a publication permission management table according to the embodiment. FIG. [Figure 17] FIG. 13 is a conceptual diagram illustrating an example of a processing method management table according to the embodiment. [Figure 18] FIG. 4 is an example of a sequence diagram showing an overall process including an image capturing process according to the embodiment. [Figure 19] FIG. 13 is a diagram illustrating an example of a participation request screen according to the embodiment. [Figure 20] FIG. 11 is an example of a sequence diagram showing an overall process including a process for a captured image according to the embodiment; [Figure 21] 11 is a flowchart illustrating an example of a confidential information candidate extraction process according to the embodiment. [Figure 22] 11 is a sequence diagram illustrating an example of a process after a preview screen is displayed according to the embodiment. [Figure 23] 11 is a flowchart illustrating an example of a process after preparation for a Web conference according to an embodiment. [Figure 24A] FIG. 11 is a diagram showing an example of a receiving screen on the other base side according to the embodiment; [Figure 24B] FIG. 11 is a diagram showing another example of a receiving screen on the other base side according to the embodiment; [Figure 24C] FIG. 11 is a diagram showing another example of a receiving screen on the other base side according to the embodiment; [Figure 24D] FIG. 11 is a diagram showing another example of a receiving screen on the other base side according to the embodiment; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the description of the drawings, the same elements are given the same reference numerals, and the description of overlapping parts will be omitted.

[0009] [Embodiment] [Overall configuration of image display system] Fig. 1 is a diagram showing an example of the overall configuration of an image display system. As shown in Fig. 1, the image display system 1 has an image capture device 3, an image management device 5, a communication terminal 7A (an example of a local base terminal) used at the local base, and a communication terminal 7B (an example of a local base terminal) used at another base. The image display system 1 also has an image management system 2, which has an image capture device 3. The image management system 2 also has an image management device 5. The image management system 2 further has a communication terminal 7A used at the local base.

[0010] In the image display system 1 (image management system 2), the communication terminal 7A, the image management device 5, and the communication terminal 7B are connected to each other via the communication network 100.

[0011] The communication network 100 is a wired or wireless communication network in which an unspecified number of communications are carried out, and is constructed by the Internet, an intranet, a LAN (Local Area Network), etc. The communication network 100 may include a communication network using wireless communication such as 4G (4th Generation), 5G (5th Generation), WiMAX (Worldwide Interoperability for Microwave Access), LTE (Long Term Evolution), etc. The communication terminal 7A, the image management device 5, and the communication terminal 7B may be connected to each other by a dedicated in-house network, etc.

[0012] The image capturing device 3 and the communication terminal 7A are connected by a dedicated wired cable, and data (information) is exchanged using, for example, the UVC (USB Video Class) protocol.

[0013] <Photographic equipment> The photographing device 3 is a device (apparatus) capable of photographing 360° around (periphery) the photographing device 3 to obtain a photographed image (panoramic image) of its own base. The photographing device 3 may be realized by an information processing device (computer system) for communication equipped with a general OS or the like, and is one of the components that configure the image display system 1 (image management system 2). The photographing device 3 also installs and stores in a storage means a communication application for communicating with the communication terminal 7A that is capable of communicating according to the above-mentioned UVC protocol.

[0014] The imaging device 3 may be a wide-angle camera capable of acquiring a panoramic image, or a 360° camera (omnidirectional camera) capable of capturing a 360° spherical image.

[0015] <Image management device> The image management device 5 is a device that manages captured image related information related to images captured at the own base by the photographing device 3. The image management device 5 is realized by an information processing terminal (computer system) for communication equipped with a general OS or the like, and is one of the components that configure the image display system 1 (image management system 2). The image management device 5 also installs and stores in a storage means a communication application or the like for communicating with the communication terminal 7A connected to the photographing device 3.

[0016] Incidentally, image management device 5 may be a commonly used PC (Personal Computer) or a server device.

[0017] Furthermore, the image management device 5 may notify (transmit) data (information) by push notification (transmission) to the communication terminal 7 connected via the communication network 100. In this case, the image management device 5 may perform the push notification via, for example, FCM (Firebase Cloud Messaging), which is an example of a push notification server.

[0018] <Communication terminal> The communication terminal 7 is a communication terminal used at a base where the photographing device 3 is used or at another base where images captured at the base are shared. Among the communication terminals 7, the communication terminal 7A is a terminal that transmits and receives image data related to images captured at the base by the photographing device 3 between the photographing device 3 and the communication terminal 7A. In addition, the communication terminal 7A connected to the photographing device 3 using, for example, a UVC protocol displays images captured at the base. However, the communication terminal 7 and the photographing device 3 may communicate with each other using a short-range wireless communication technology such as BLE (Bluetooth Low Energy, registered trademark) or NFC instead of the UVC protocol.

[0019] On the other hand, among the communication terminals 7, communication terminal 7B is a communication terminal used at another base, and an image of the user's own base captured by the photographing device 3 is displayed on communication terminal 7B. Note that an image (photographed image) of the other base is displayed on communication terminal 7 of another user who uses the user's own base. In this way, in image display system 1, images (photographed images) of the other bases are displayed on communication terminals 7 other than communication terminal 7 connected to photographing device 3 using the UVC protocol.

[0020] The communication terminal 7 is realized by an information processing terminal (computer system) for communication equipped with a general OS or the like, and is one of the components that configure the image display system 1 (image management system 2). The communication terminal 7 also has a communication application or the like for communicating with the image management device 5 installed and stored in a storage means.

[0021] The communication terminal 7 may be a communication terminal having a communication function, such as a commonly used PC (Personal Computer), a portable notebook PC, a mobile phone, a smartphone, a tablet terminal, a wearable terminal (sunglasses type, wristwatch type, etc.), an electronic whiteboard device (IWB), etc. The communication terminal 7 may further be a communication device or a communication terminal on which various application software such as browser software, remote conference application, etc. can be installed and operated. In this embodiment, unless otherwise specified, the communication terminals 7A and 7B are simply referred to as the communication terminals 7.

[0022] ●About terminology● In this embodiment, an "event" refers to an online conference (Web conference), online meeting, etc., held via a communication network. In this embodiment, an "online conference (Web conference)" is sometimes referred to as an example of an "event."

[0023] In this embodiment, the term "user" refers to a user who uses the image display system 1 (image management system 2) or an administrator who manages the users. The term "user" also refers to a participant who takes part in a Web conference or the like. In this embodiment, the term "user" is used unless otherwise specified.

[0024] <Shooting range of the imaging device> Next, the imaging range of the imaging device will be described. Fig. 2 is a diagram showing an example of the imaging range of the imaging device according to the embodiment, where (a) is an example of imaging 360° around the imaging device (device), and (b) is an example of imaging a predetermined angle above and below the horizontal 0° of the imaging device (device). As shown in Fig. 2(a), the imaging device 3 captures an image of a 360° range in the horizontal direction. As shown in Fig. 2(b), the imaging device 3 sets the horizontal direction to the height of the imaging device 3 as 0°, and sets a predetermined angle above and below as the imaging range.

[0025] <Diagram explaining panoramic image and speaker image clipping> FIG. 3 is a diagram showing an example of cutting out a panoramic image and a speaker image according to the embodiment. As shown in FIG. 3, the image captured by the image capture device 3 forms a part 350 of a sphere, and therefore has a three-dimensional shape. As shown in FIG. 2(b), the image capture device 3 performs perspective projection transformation by dividing the angle of view into predetermined angles in the vertical direction and predetermined angles in the horizontal direction. By performing perspective projection transformation over the entire horizontal direction of 360° without gaps, a predetermined number of planar images are obtained, and a panoramic image 351 is obtained by connecting the predetermined number of planar images to the left and right. In addition, the image capture device 3 performs face detection from the panoramic image within a predetermined range centered on the voice direction, and generates a speaker image 352 by cutting out, for example, 15° left and right (30° in total) from the center of the face of the user at each location.

[0026] <Flow of processing performed on captured images> FIG. 4 is a conceptual diagram showing an example of a flow of a predetermined processing process executed on a photographed image according to the embodiment. In FIG. 4, a conceptual diagram is shown in which a photographing device 3 and a company A conference room (home base side) where a communication terminal 7A connecting the photographing device 3 is placed, a company B conference room (other base side) where a communication terminal 7B is placed, and a management device 5 are connected to each other via a communication network. Specifically, the photographed image related information including the event destination information (conference destination information) acquired from the communication terminal 7A is sent to the image management device 5 in addition to the image of the surroundings of the company A conference room photographed by the photographing device 3. After that, the image management device 5 generates a confidential information candidate list, a confidential information applicability determination result, and image data, and sends screen data constituting a screen to be displayed in the company B conference room (other base side). In the panoramic image shown in FIG. 4, "confidential information candidate 1" does not correspond to confidential information for the event destination (conference destination). For this reason, unprocessed output is permitted, and a predetermined process is not performed. On the other hand, "confidential information candidate 2" corresponds to confidential information for the event destination. Therefore, after a predetermined process (for example, blurring) is performed on the image representing the "confidential information candidate 2", the processed image is displayed on the communication terminal 7B in the company B's conference room.

[0027] Here, in the management device 5, an image processing setting file is generated based on captured image related information related to captured image data related to a captured image (panoramic image) captured by the imaging device 3 in the conference room of Company A. This image processing setting file is a setting file for performing a predetermined processing to conceal confidential information contained in the captured image. Specifically, the image processing setting file is a setting information program for performing a predetermined processing based on confidential information related to a non-disclosure agreement concluded between Company A and Company B, for example. In addition, this image processing setting file includes confidential information relevant / incompatible determination data based on a confidential information search list generated by searching for keywords contained in characters, etc. in the panoramic image.

[0028] After that, the image management device 5 generates an image processing setting file including image data to be displayed on the communication terminal 7B arranged in the company B's conference room, and causes the communication terminal 7B arranged in the company B's conference room to display a screen including an image that has been subjected to a predetermined processing process. In this embodiment, for a keyword to be kept secret that is included in the screen displayed on the communication terminal 7B arranged in the company B's conference room, a process is performed to blur the keyword or a predetermined area including the keyword.

[0029] <Relationship between resolution and object distance> FIG. 5 is a conceptual diagram showing an example of the relationship between the resolution and the object distance according to the embodiment. As shown in FIG. 5, in the relationship between the resolution and the object distance, the target range of the processing and the asymmetric range are separated by a predetermined boundary (threshold). Here, the resolution is an example of an index representing the performance of the photographing device 3. The object distance represents the distance between the photographing device 3 and the subject. At this time, for example, when the resolution is high and the object distance is relatively farther than the predetermined threshold, the confidential information candidate is more likely to be clearly displayed, so it is within the target range of the predetermined processing. On the other hand, for example, when the resolution is low and the object distance is relatively farther than the predetermined threshold, the confidential information candidate is less likely to be clearly displayed, so it is within the non-target range of the predetermined processing. Note that the conceptual diagram shown in FIG. 5 is an example, and the boundary (threshold) separating the target range and the non-target range is not limited to this.

[0030] <Screen display example 1 in display mode> Next, a screen display example in the display mode will be described. FIG. 6A is a diagram showing an example of a display screen for each display mode according to the embodiment. As shown in FIG. 6A, a display mode screen 7011 is displayed as an image of a pattern 1 of the display mode on the communication terminal 7B at the other base. On the display mode screen 7011, a panoramic image showing an image of the own base captured by the image capture device 3 and close-up images 1, 2, and 3 in which a specific speaker or a fixed image in the panoramic image is close-up are displayed side by side. The close-up images are examples of speaker images. Specifically, close-up image 1 is an image in which the speaker at the left end of the panoramic image (speaker 1) is close-up, and a "confidential information candidate 1" behind speaker 1 is displayed. Close-up image 2 is an image in which a fixed image (for example, a whiteboard image) in the panoramic image is close-up, and a "confidential information candidate 2" behind speaker 2 is displayed. Close-up image 3 is an image in which the speaker at the right end of the panoramic image (speaker 2) is close-up, and a "confidential information candidate 2" behind speaker 2 is displayed. In addition, since the contents of "classified information candidate 2" correspond to "blurring processing", which is an example of a predetermined processing in this embodiment, the entire contents of "classified information candidate 2" are displayed in a blurred state as shown in the figure. In this case, the entire contents of "classified information candidate 2" in the panoramic image are also displayed in a blurred state. In other words, in the display mode specific screen 7011, the predetermined processing is applied to both the panoramic image and the close-up image for portions that require the predetermined processing.

[0031] In this embodiment, "blurring" is described as an example of a predetermined processing for a captured image (captured image data), but the present invention is not limited to this as long as the information to be kept secret is processed to a state where it cannot be distinguished (read) by participants in an online event such as an online conference. Therefore, examples of other predetermined processing for a captured image (captured image data) include, for example, "erasing," "overpainting," "vibration," and "substitution (replacement) processing."

[0032] In the display mode specific screen 7011, the panoramic image and the close-up image are arranged as shown in the figure, but the arrangement is not limited to this. Furthermore, the number of images in the speaker image is not limited to three. In addition, in the example shown in Fig. 6A, each speaker is displayed facing forward, but the direction of each speaker in the panoramic image may be changed to an arbitrary angle, and the image of the speaker with the changed angle may be displayed as the corresponding close-up image.

[0033] <Screen display example 2 in display mode> 6B is a diagram showing another example of a display screen for each display mode according to the embodiment. As shown in FIG. 6B, a display mode screen 7012 is displayed on a communication terminal 7B at another base as an image of a display mode pattern 2. The display mode screen 7012 shows a state in which close-up image 2 on the display mode screen 7011 has been changed from a fixed image to an image of the upper body of the central participant in a panoramic image.

[0034] <Screen display example 3 in display mode> 6C is a diagram showing another example of a display screen for each display mode according to the embodiment. As shown in FIG. 6C, a display mode screen 7013 is displayed as an image of display mode pattern 3 on the communication terminal 7B at the other base. On the display mode screen 7013, a 180° panoramic image 1 and another 180° panoramic image 2 are displayed side by side among 360° panoramic images representing images captured at the own base by the image capturing device 3. At this time, for the "confidential information candidate 2", an image that has been subjected to "blurring processing" is displayed, as in the display mode screen 7011.

[0035] <Screen display example 4 in display mode> Fig. 6D is a diagram showing another example of a display screen for each display mode according to the embodiment. As shown in Fig. 6D, a display mode screen 7014 is displayed as an image of display mode pattern 4 on the communication terminal 7B at the other base. On the display mode screen 7014, a panoramic image showing an image captured at the own base by the image capture device 3 and a close-up image (fixed to one screen) in which a specific speaker or a fixed image in the panoramic image is close-up are displayed. As shown in Fig. 6D, on the display mode screen 7014, the close-up image is displayed larger.

[0036] <Screen display example 5 in display mode> Fig. 6E is a diagram showing another example of a display mode-specific display screen according to an embodiment. As shown in Fig. 6E, a display mode-specific screen 7015 as an image of display mode pattern 5 is displayed on the communication terminal 7B at the other base. On the display mode-specific screen 7015, a panoramic image showing an image captured at the own base by the photographing device 3 and a file at the time of screen sharing of the communication terminal at the own base or an icon image showing the file are displayed. As shown in Fig. 6E, on the display mode-specific screen 7015, a screen-shared object (for example, a file, a photo, text information, etc.) is displayed.

[0037] It should be noted that the screen display examples in the display mode are not limited to the five examples described above.

[0038] [Hardware configuration] Next, the hardware configuration of the device or apparatus constituting the image display system according to the embodiment will be described with reference to Figures 7 to 9. Note that components may be added or deleted from the hardware configuration of the device or apparatus shown in Figures 7 to 9 as necessary.

[0039] <Hardware configuration of the imaging device> 7 is a diagram showing an example of the hardware configuration of an image capturing device according to an embodiment. In the following, the image capturing device 3 is assumed to be a device that uses an image capturing element to capture a 360° image (still image or video) around the image capturing device 3 at a predetermined height, but the image capturing element may be one or more than two. In addition, it does not necessarily have to be a dedicated device, and a PC, digital camera, smartphone, etc. may be provided with substantially the same function by attaching a 360° image capturing unit to be attached later.

[0040] As shown in FIG. 7, the photographing device 3 is constructed, for example, by a computer, and is composed of an imaging unit 301, an image processing unit 304, an imaging control unit 305, a microphone 308, a sound processing unit 309, a CPU (Central Processing Unit) 311, a ROM (Read Only Memory) 312, an SRAM (Static Random Access Memory) 313, a DRAM (Dynamic Random Access Memory) 314, an operation unit 315, an external device connection I / F 316, a communication unit 317, an antenna 317a, a sound sensor 318, a distance measurement sensor 319, and a concave terminal 321 such as USB Type-C (registered trademark; omitted below).

[0041] Among these, the imaging unit 301 includes a wide-angle lens (so-called fisheye lens) 302 having a 360° angle of view for forming a hemispherical image, and an imaging element 603 (image sensor) provided corresponding to each wide-angle lens. The imaging element 303 includes an image sensor such as a CMOS (Complementary Metal Oxide Semiconductor) sensor that converts an optical image by the fisheye lens 602 into image data of an electric signal and outputs it, a timing generation circuit that generates a horizontal or vertical synchronization signal and a pixel clock for this image sensor, and a group of registers in which various commands and parameters required for the operation of this imaging element are set. The imaging unit 301 may be a 360° camera, which is an example of an imaging means capable of capturing an image of 360° around the photographing device 3. A plurality of pieces of information acquired by a plurality of imaging elements (for example, 180°+180°) may be synthesized to obtain a 360° angle of view.

[0042] The imaging elements 303 (image sensors) of the imaging unit 301 are connected to the image processing unit 304 via a parallel I / F bus. Meanwhile, the imaging elements 303 of the imaging unit 301 are connected to the imaging control unit 305 via a serial I / F bus (such as an I2C bus). The image processing unit 304, the imaging control unit 305, and the sound processing unit 309 are connected to a CPU 311 via a bus 310. Furthermore, the bus 310 is also connected to a ROM 312, an SRAM 313, a DRAM 314, an operation unit 315, an external device connection I / F 316, a communication unit 317, a sound sensor 318, a distance measurement sensor 319, and the like.

[0043] The image processing unit 304 takes in the image data output from the image sensor 303 via a parallel I / F bus, performs predetermined processing on each image data, and generates data on a panoramic image and a speaker image from the fisheye video. Furthermore, the image processing unit 304 synthesizes the panoramic image and the speaker image, etc., and outputs one video.

[0044] The imaging control unit 305 generally sets commands and the like in the registers of the imaging element 303 using an I2C bus or the like, with the imaging control unit 305 as a master device and the imaging element 303 as a slave device. Necessary commands and the like are received from the CPU 311. The imaging control unit 305 also uses the I2C bus to retrieve status data and the like from the registers of the imaging element 303 and send it to the CPU 311.

[0045] Furthermore, the imaging control unit 305 instructs the imaging elements 303a and 303b to output image data when the imaging start button of the operation unit 315 is pressed or when an imaging start instruction is received from the communication terminal 7. Some photographing devices 3 have a preview display function or a function corresponding to video display by a display device (for example, a display of a PC or smartphone). In this case, the image data is output from the imaging element 303 continuously at a predetermined frame rate (frames / minute).

[0046] As described later, the imaging control unit 305 also functions as a synchronization control means for cooperating with the CPU 311 to synchronize the output timing of image data from the imaging element 303. In this embodiment, the photographing device 3 is not provided with a display device, but a display unit may be provided.

[0047] The microphone 308 converts sound into sound (signal) data. The sound processing unit 309 takes in the sound data output from the microphone 308 via the I / F bus, and performs predetermined processing on the sound data.

[0048] The CPU 311 controls the overall operation of the image capturing device 3 and executes necessary processes. The ROM 312 stores various programs for the CPU 311.

[0049] The SRAM 313 and DRAM 314 are work memories that store programs executed by the CPU 311, data in the middle of processing, etc. In particular, the DRAM 314 stores image data in the middle of processing by the image processing unit 304 and data of a processed equirectangular projection image.

[0050] The operation unit 315 is a general term for operation buttons such as an imaging start button 315a. By operating the operation unit 315, a user can start imaging or recording, as well as turn the power on / off, establish a communication connection, and input settings such as various imaging modes and imaging conditions.

[0051] The external device connection I / F 316 is an interface for connecting various external devices. In this case, the external device is, for example, a PC (Personal Computer), a display, a projector, an electronic whiteboard device, etc. The external device connection I / F 316 may include, for example, a USB terminal, an HDMI (registered trademark) terminal, etc. The video data and image data stored in the DRAM 314 are transmitted to an external terminal via the external device connection I / F 316, or recorded in an external medium. In addition, by using a plurality of external device connection I / Fs 316, for example, image information captured and acquired by the image capture device 3 may be transmitted to the communication terminal 7 via a UVC protocol associated with a USB connection and recorded, while an image (for example, screen information displayed in a remote conference application, etc.) may be acquired from the communication terminal 7 to the image capture device 3, and further transmitted from the image capture device 3 via HDMI to another external device (such as a display, a projector, an electronic whiteboard device, etc.) for display.

[0052] The communication unit 317 may communicate with a cloud server via the Internet using a wireless communication technology such as Wi-Fi via an antenna 317a provided in the image capturing device 3, and transmit the stored video data and image data to the cloud server. The communication unit 317 may also communicate with nearby devices using a short-range wireless communication technology such as BLE (Bluetooth Low Energy, registered trademark) or NFC.

[0053] The audio sensor 318 is a sensor that acquires 360° audio information in order to identify from which direction a loud sound is input within 360° around (horizontal plane) the image capture device 3. The sound processing unit 609 identifies the strongest direction based on the input 360° audio parameters and outputs the audio input direction within 360°.

[0054] Other sensors (azimuth / acceleration sensors, GPS, etc.) may be used to calculate azimuth, position, angle, acceleration, etc. and to correct images or add location information.

[0055] In addition, the image processing unit 304 performs the following processes.

[0056] The CPU 311 generates a panoramic image in the following manner. The CPU 311 performs predetermined camera image processing such as Bayer conversion (RGB complementation processing) on ​​the RAW data input from the image sensor that inputs the spherical image to generate a fisheye image (image made up of curved images). The CPU 311 further performs flattening processing such as DeWarp processing (distortion correction processing) on ​​the generated fisheye image (curved image) to generate a panoramic image (image made up of flat images) that captures 360° around the image capture device 3.

[0057] The CPU 311 generates a speaker image in the following manner. The CPU 311 generates a speaker image (an example of a close-up image) by cutting out the speaker from a panoramic image (flat image) showing 360° of the surroundings. The CPU 311 cuts out the speaker image from the panoramic image, taking the voice input direction identified from the 360° output using the sound sensor 318 and the sound processing unit 309 as the speaker's direction. The method of cutting out a person's image from the voice input direction at this time is to cut out a 30° area centered on the voice direction identified from the 360°, and perform face detection within that area to cut out the image. The CPU 311 further identifies speaker images for a specific number of people (e.g., three people) who have spoken most recently from among the cut-out speaker images.

[0058] The panoramic image and one or more speaker images may be transmitted separately to an information recording application, etc., or the photographing device 3 may generate one image from them and transmit it to the information recording application, etc. In this embodiment, the panoramic image and one or more speaker images are transmitted separately from the photographing device 3 to the information recording application, etc.

[0059] <Hardware configuration of image management device> Fig. 8 is a diagram showing an example of the hardware configuration of an image management device according to an embodiment. As shown in Fig. 8, the image management device 5 is constructed by, for example, a computer, and includes a CPU 501, a ROM 502, a RAM 503, an EEPROM 504, a HD (Hard Disk) 505, a HDD (Hard Disk Drive) controller 506, and a short-range communication I / F 508. The image management device 5 further includes a network I / F 511, a keyboard 512, a pointing device 513, a media I / F 515, an external device connection I / F 516, and a bus line 520.

[0060] Among these, the CPU 501 controls the operation of the entire image management device 5. The ROM 502 stores programs and the like used to drive the CPU 501. The RAM 503 is used as a work area for the CPU 501. The EEPROM 504 reads or writes various data such as applications under the control of the CPU 501. The HD 505 stores various data such as programs. The HDD controller 506 controls the reading or writing of various data from the HD 505 under the control of the CPU 501. Here, the image management device 5 may have a hardware configuration equipped with an SSD (Solid State Drive) instead of the HD 505 and the HDD controller 206. The short-range communication I / F 508 is a communication circuit for performing data communication with a communication device or a communication terminal having a wireless communication interface such as NFC (Near Field Communication), Bluetooth (registered trademark, omitted below), or Wi-Fi (registered trademark, omitted below).

[0061] The network I / F 511 is an interface for data communication using the communication network 100. The keyboard 512 is a type of input means having a plurality of keys for inputting characters, numbers, various instructions, etc. Note that, instead of or in addition to the keyboard 512, an input means such as a touch panel for operating a predetermined button, icon, etc. may be used. The pointing device 513 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The media I / F 515 controls reading or writing (storing) data from and to a recording medium 514 such as a flash memory. The external device connection I / F 516 is an interface for connecting various external devices. In this case, the external device may be, for example, a USB (Universal Serial Bus) memory, etc. The bus line 520 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 501.

[0062] <Hardware configuration of communication terminal> 9 is a diagram showing an example of a hardware configuration of a communication terminal according to an embodiment. As shown in FIG. 9, the communication terminal 7 is constructed by, for example, a computer, and includes a CPU 701, a ROM 702, a RAM 703, an EEPROM 704, a HD (Hard Disk) 705, a HDD (Hard Disk Drive) controller 706, a display 707, a short-range communication I / F 708, a CMOS sensor 709, and an image sensor I / F 710. In this embodiment, the display 707 functions as an example of a display unit. The communication terminal 7 further includes a network I / F 711, a keyboard 712, a pointing device 713, a media I / F 715, an external device connection I / F 716, a sound input / output I / F 717, a microphone 718, a speaker 719, and a bus line 720.

[0063] Among these, the CPU 701 controls the operation of the entire communication terminal 7. The ROM 702 stores programs and the like used to drive the CPU 701. The RAM 703 is used as a work area for the CPU 701. The EEPROM 704 reads or writes various data such as applications under the control of the CPU 701. The HD 705 stores various data such as programs. The HDD controller 706 controls the reading or writing of various data from the HD 705 under the control of the CPU 701. Here, the communication terminal 7 may have a hardware configuration equipped with an SSD (Solid State Drive) instead of the HD 705 and the HDD controller 706. The display 707 displays various information such as a cursor, a menu, a window, a character, or an image. In this embodiment, the display 707 functions as an example of a display means. The near field communication I / F 708 is a communication circuit for performing data communication with a communication device or a communication terminal having a wireless communication interface such as NFC (Near Field Communication), Bluetooth (registered trademark; omitted below), or Wi-Fi (registered trademark; omitted below). The CMOS sensor 709 is a type of built-in imaging means that captures an image of a subject under the control of the CPU 701 to obtain image data or video data. The imaging means may be an imaging means constituted by a CCD (Charge Coupled Device) sensor or the like instead of a CMOS sensor. The imaging element I / F 710 is a circuit that controls the driving of the CMOS sensor 709.

[0064] The network I / F 711 is an interface for data communication using the communication network 100. The keyboard 712 is a type of input means having a plurality of keys for inputting characters, numbers, various instructions, etc. Note that, instead of or in addition to the keyboard 712, an input means such as a touch panel for operating a predetermined button, icon, etc. may be used. The pointing device 713 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The media I / F 715 controls reading and writing (storing) data from and to a recording medium 714 such as a flash memory. The external device connection I / F 716 is an interface for connecting various external devices. In this case, the external device may be, for example, a USB (Universal Serial Bus) memory, etc. The sound input / output I / F 717 is a circuit for processing input and output of sound signals between a microphone 718 and a speaker 719 according to the control of the CPU 701. The microphone 718 is a built-in circuit that converts sound into an electrical signal, and obtains information using the electrical signal by acquiring voice or sound waves emitted from an external speaker or the like. The speaker 719 is a built-in circuit that converts an electrical signal into physical vibrations to generate sounds such as music or voice. The bus line 720 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 701.

[0065] The above program may be recorded in a computer-readable recording medium as an installable or executable file, or may be downloaded and distributed via a network. Examples of the recording medium include a CD-R (Compact Disc Recordable), a DVD (Digital Versatile Disk), a Blu-ray Disc (Blu-ray is a registered trademark; omitted below), an SD card, and a USB memory. The recording medium may be provided domestically or internationally as a program product. For example, the imaging device 3 realizes the imaging processing method according to the present invention by executing the program according to the present invention.

[0066] [Functional configuration of the image management system] Next, the functional configuration of this embodiment will be described with reference to Fig. 10 to Fig. 17. Fig. 10 is a diagram showing an example of the functional configuration of an image display system according to an embodiment. Fig. 10 shows communication terminals or devices shown in Fig. 1 that are related to processing or operation described later.

[0067] <Functional configuration of the imaging device> Next, the functional configuration of the photographing device 3 will be described with reference to FIG. 10. As shown in FIG. 10A, the photographing device 3 has a transmitting / receiving unit 31, an operation receiving unit 32, an image capturing unit 33, a display control unit 34, a determining unit 35, a calculation processing unit 36, a generating unit 37, a setting registration unit 38, and a storage reading unit 39. Each of these functional units is a function or means realized by operating any of the hardware resources shown in FIG. 7 by an instruction from the CPU 311 according to a program for the photographing device 3 that is expanded from the ROM 312 to the DRAM 314. The photographing device 3 also has a storage unit 3000 (an example of a storage means) constructed by the ROM 312 shown in FIG. 7. A communication program (communication application) for communicating with the communication terminal 7 via the UVC protocol and the like are installed and stored in the storage unit 3000. The storage unit 3000 further stores photographed image related information related to a photographed image including the surroundings of the photographing device 3 photographed at the own base.

[0068] <<Functional configuration of the imaging device>> Next, each functional configuration of the photographing device 3 will be described in detail. The transmission / reception unit 31 of the photographing device 3 shown in FIG. 10A is mainly realized by the processing of the CPU 311 for the external device connection I / F 316 and the communication unit 317. In the case of communication using the external device connection I / F 316, the transmission / reception unit 31 transmits and receives various data (or information) to and from the communication terminal 7 via the UVC protocol. In addition, the transmission / reception unit 31 transmits the generated preview screen data to the communication terminal 7A (own base terminal) using the UVC protocol so that the preview screen is displayed on the communication terminal 7B (other base terminal). In addition, the transmission / reception unit 31 adds distance information representing a distance to the preview screen data and transmits it to the communication terminal 7A (own base terminal) using the UVC protocol. In addition, the transmission / reception unit 31 adds resolution information representing the performance of the photographing device 3 to the preview screen data and transmits it to the communication terminal 7A (own base terminal) using the UVC protocol. In this embodiment, the transmission / reception unit 31 functions as an example of at least one of the means of preview screen data transmission means and reception means.

[0069] The operation reception unit 32 is mainly realized by the CPU 311 processing signals generated by various operations received by the operation unit 315, and detects various operations on the imaging device 3. In this embodiment, the operation reception unit 32 functions as an example of a reception means.

[0070] The image capturing and acquiring unit 33 is mainly realized by the processing of the CPU 311 on the imaging unit 301, the image processing unit 304, and the imaging control unit 305, and captures an image of the own base by capturing an image of the surroundings of the imaging device 3. The image capturing and acquiring unit 33 also monitors changes in the surroundings of the own base, and if there is a change between the captured image of the own base and the image of the own base after a predetermined time has elapsed, captures an image of the surroundings of the imaging device again to acquire a re-captured image of the own base. In this embodiment, the image capturing and acquiring unit 33 functions as an example of an image capturing means.

[0071] The display control unit 34 is mainly realized by the processing of the CPU 311, and controls the display on the display or the like of the communication terminal 7 at the other base (communication terminal 7B). In this embodiment, the display control unit 34 functions as an example of a display control means.

[0072] The determination unit 35 is mainly realized by the processing of the CPU 311, and performs various determinations in the image capturing device 3. In this embodiment, the determination unit 35 functions as an example of a determination means.

[0073] The calculation processing unit 36 ​​is mainly realized by the processing of the CPU 311, and calculates coordinate information related to the photographed image data photographed by the photographing device 3. The calculation processing unit 36 ​​also performs a predetermined image processing for generating an image corresponding to the display mode. Specifically, the calculation processing unit 36 ​​performs a predetermined processing on a photographed image that can be displayed on a communication terminal 7B (other base terminal) used at another base that shares the photographed image at the own base, based on a predetermined display mode and an image processing setting file for performing a predetermined processing on the photographed image. The calculation processing unit 36 ​​further measures the distance between the photographing device 3 and a subject photographed by the photographing device 3. Specifically, the calculation processing unit 36 ​​calculates distance information of a photographed object existing at the own base or another base. In this embodiment, the calculation processing unit 36 ​​functions as an example of a calculation means and an example of a distance measurement means.

[0074] The generating unit 37 is mainly realized by the processing of the CPU 311, and generates an image corresponding to the display mode. The generating unit 37 also generates preview screen data representing a preview screen of the image after processing performed by the calculation processing unit 36. The generating unit 37 also generates preview screen data in which the entire captured image or a specific part thereof has been subjected to a process of blurring as a predetermined processing process. In this embodiment, the generating unit 37 functions as an example of a preview screen data generating means.

[0075] The setting registration unit 38 is mainly realized by the processing of the CPU 311, and specifies and registers destination information for an event such as a Web conference, etc. In this embodiment, the setting registration unit 38 functions as an example of a setting means and an example of a registration means.

[0076] The memory / reading unit 39 is mainly realized by processing of the CPU 311 on the ROM 312, and stores various data (or information) in the storage unit 3000 and reads various data (or information) from the storage unit 3000. In this embodiment, the memory / reading unit 39 functions as an example of a memory / reading means.

[0077] ●Photographic equipment information management table● Fig. 11 is a conceptual diagram showing an example of a photographing device information management table according to an embodiment. Note that the structure and contents of the data table described below are merely examples and are not limited thereto. A photographing device information management DB 3001 configured by a photographing device information management table as shown in Fig. 11 is constructed in the storage unit 3000. In the photographing device information management table, a photographing device name and photographing device information (performance information) are associated with each photographing device ID, stored, and managed.

[0078] Among these, the camera device ID is identification information for identifying the camera device 3, and the camera device name is a name representing the camera device. The camera device information includes performance information representing the performance of the camera device, and the performance information is, for example, a resolution [dpi].

[0079] In this embodiment, the photographing apparatus information management table (the photographing apparatus information management DB 3001) functions as an example of a photographing apparatus information management means.

[0080] ●Photographic image information management table● Fig. 12 is a conceptual diagram showing an example of a photographed image information management table according to an embodiment. Note that the structure and contents of the data table described below are merely examples and are not limited thereto. A photographed image information management DB 3002 configured by a photographed image information management table as shown in Fig. 12 is constructed in the storage unit 3000. In the photographed image information management table, a photographed image ID, a photographing date and time, photographed image data, and distance information are stored and managed in association with each other for each tab with a photographing device ID as a tab.

[0081] Among these, the captured image ID is identification information for identifying the captured image captured by the camera device 3. The capture date and time is date and time information indicating the date and time when the camera device 3 captured the captured image. The captured image data is a name indicating an image existing at a specific position or in a specific area included in the captured image (panoramic image) captured by the camera device 3, such as a whiteboard image or wall images 1 and 2. The distance information indicates the distance (capturing distance) between the camera device 3 and a participant or captured image data participating at the camera device's own base or another base.

[0082] In this embodiment, the photographed image information management table (photographed image information management DB 3002) functions as an example of a photographed image information management means.

[0083] ●Destination information management table● Fig. 13 is a conceptual diagram showing an example of a destination information management table according to an embodiment. Note that the structure and contents of the data table described below are merely examples and are not limited thereto. A destination information management DB 3003 configured by a destination information management table as shown in Fig. 13 is constructed in the storage unit 3000. In the destination information management table, a destination base ID, an event date and time, and event destination information are stored and managed in association with each other for each tab with the photographing device ID as a tab.

[0084] Of these, the destination location ID is identification information that identifies the destination location participating in a Web conference, etc. The event date and time is date and time information indicating the date and time when a specified event, such as a Web conference, will be held with the destination location. The event destination information indicates information about the destination participating in an event, such as a Web conference, and includes the destination name, the destination email address, the event link (event name), etc. In the destination information management table, the event destination information described above is managed, making it possible to grasp information about destinations at other locations.

[0085] In this embodiment, the destination information management table (destination information management DB 3003) functions as an example of a destination information management means.

[0086] <Functional configuration of communication terminal> Returning to FIG. 10A, the functional configuration of the communication terminal 7 will be described. As shown in FIG. 10A, the communication terminal 7 has a transmitting / receiving unit 71A(B), an operation receiving unit 72A(B), an acquiring unit 73A(B), a display control unit 74A(B), a generating unit 77A(B), and a storage / reading unit 79A(B). Each of these functional units is a function or means realized by operating any of the hardware resources shown in FIG. 7 by an instruction from the CPU 701 according to a program for the communication terminal 7 expanded from at least one of the ROM 702, the EEPROM 704, the HD 705, and the recording medium 714 to the RAM 703. In addition, the communication terminal 7 has a storage unit 7000 (an example of a storage means) constructed by at least one of the ROM 702, the EEPROM 704, the HD 705, and the recording medium 714 shown in FIG. 9. The memory unit 7000 also has installed and stored a communication program (communication application) for communicating with the photographing device 3 via the UVC protocol, a remote conference application for conducting web conferences with the communication terminal 7 at the other site, and the like.

[0087] <<Functional configuration of communication terminal>> Next, each functional configuration of the communication terminal 7 will be described in detail. In this embodiment, unless otherwise specified, the suffixes A and B of each functional unit of the communication terminal 7 will be omitted. The transmission / reception unit 71A of the communication terminal 7A shown in FIG. 10A is mainly realized by the processing of the CPU 701 on the short-range communication I / F 708 and the network I / F 711. The transmission / reception unit 71A also transmits the image processing setting file transmitted by the image management device 5 to the photographing device 3. In this embodiment, the transmission / reception unit 71A functions as an example of at least one of a transmitting means and a receiving means.

[0088] The operation acceptance unit 72 is mainly realized by processing of the CPU 701 on any one of the display 707, the keyboard 712, and the pointing device 713, and accepts various operations on the communication terminal 7. In this embodiment, the operation acceptance unit 72 functions as an example of an operation acceptance means.

[0089] The acquisition unit 73 is mainly realized by the processing of the CPU 701, and acquires captured image related information representing the captured image captured by the photographing device 3. In this embodiment, the acquisition unit 73 functions as an example of a calculation means and an example of a processing means. In this embodiment, the acquisition unit 73 functions as an example of an acquisition means.

[0090] The display control unit 74 is mainly realized by the processing of the CPU 701 for the display 707, and causes the communication terminal 7 to display captured images (panoramic images, close-up images) of the other party's location on the display 707 (display means). In this embodiment, the display control unit 74 functions as an example of a display control means.

[0091] The generating unit 77 is mainly realized by the processing of the CPU 701, and generates preview screen data (screen data related to a captured image that has been subjected to a predetermined process) for each location to be displayed on the communication terminal 7. In this embodiment, the generating unit 77 functions as an example of a generating means.

[0092] The storage / reading unit 79 is mainly realized by processing of the CPU 701 on at least one of the ROM 702, the EEPROM 704, the HD 705, and the recording medium 714, and stores various data (or information) in the storage unit 7000 and reads various data (or information) from the storage unit 7000. In this embodiment, the storage / reading unit 79 functions as an example of a storage / reading means.

[0093] <Functional configuration of the image management device> Next, the functional configuration of the image management device 5 will be described. FIG. 10B is a diagram showing an example of the functional configuration of the image display system according to the embodiment. As shown in FIG. 10B, the image management device 5 has a transmission / reception unit 51, an extraction acquisition unit 53, a display control unit 54, a judgment unit 55, an area specification unit 56, a generation unit 57, a setting registration unit 58, and a storage readout unit 59. Each of these functional units is a function or means realized by operating any of the hardware resources shown in FIG. 8 by an instruction from the CPU 501 according to a program for the image management device 5 that is expanded from at least one of the ROM 502, the EEPROM 504, the HD 505, and the recording medium 514 to the RAM 503. The image management device 5 also has a storage unit 5000 constructed by at least one of the ROM 502, the EEPROM 504, the HD 505, and the recording medium 514 shown in FIG. 8. Furthermore, in the storage unit 5000, a communication program (communication application) for communicating with each device and each terminal via the communication network 100, a browser application, and the like are installed and stored.

[0094] <<Functional configuration of information processing device>> Next, each functional configuration of the image management device 5 will be described in detail. The transmission / reception unit 51 of the image management device 5 shown in FIG. 10A is mainly realized by the processing of the CPU 501 for the network I / F 511 and the short-distance communication I / F 508, and transmits and receives various data (or information) between each terminal and each device via the communication network 100. The transmission / reception unit 51 also receives captured image related information related to the captured image at the own base captured by the imaging device 3 and transmitted by the communication terminal 7 (communication terminal 7A). The transmission / reception unit 51 also transmits the generated image processing setting file via the communication terminal 7 (communication terminal 7A) to set it in the imaging device 3. In this embodiment, the transmission / reception unit 51 functions as an example of at least one of the image processing setting file transmitting means and the captured image related information receiving means.

[0095] The extraction acquisition unit 53 is mainly realized by the processing of the CPU 501, and extracts confidential information candidates included in the captured image related information transmitted by the photographing device 3. The extraction acquisition unit 53 also checks distance information included in the captured image related information transmitted and acquired by the communication terminal 7A. The extraction acquisition unit 53 also checks character information included in the target range in the captured image based on the performance information and distance information of the photographing device 3. The extraction acquisition unit 53 further searches for synonyms of the extracted information (keywords). In this embodiment, the extraction acquisition unit 53 functions as an example of an extraction acquisition means.

[0096] The display control unit 54 is mainly realized by the processing of the CPU 501 for the display 507 or the communication terminal 7 connected via the communication network 100. The display control unit 54 controls the display of various screens and information (data) for the image management device 5. The display control unit 54 also uses, for example, a Web browser to display a display screen generated by HTML or the like on the display 707 of the communication terminal 7. The display control unit 54 further starts and executes a Launcher that runs on an OS (Operating System) and an external application, thereby displaying various screens generated by an API (Application Programming Interface) provided by the OS on the display 507 or the display of the communication terminal 7. In this embodiment, the display control unit 54 functions as an example of a display control means.

[0097] The determination unit 55 is realized mainly by the processing of the CPU 501, and performs the determination processing of the entire image manager 5. In this embodiment, the determination unit 55 functions as an example of a determination means.

[0098] The area identifying unit 56 is mainly realized by the processing of the CPU 501, and identifies an area of ​​potential confidential information included in the captured image of the own base transmitted by the image capturing device 3. In this embodiment, the area identifying unit 56 functions as an example of an identifying means.

[0099] Generation unit 57 is mainly realized by the processing of CPU 501, and generates various information in image management device 5. Generation unit 57 also generates confidential information determination data and an image processing setting file (setting information program) based on the search results of image management device 5. In this embodiment, generation unit 57 functions as an example of a generating means.

[0100] The setting registration unit 58 is mainly realized by the processing of the CPU 501, and sets various information in the image management device 5. The setting registration unit 58 also registers captured image information in the captured image information management DB 5001 managed by the image management device 5. In this embodiment, the setting registration unit 58 functions as an example of a setting means and an example of a registration means.

[0101] The storage / reading unit 59 is mainly realized by processing of the CPU 501 on at least one of the ROM 502, the EEPROM 504, the HD 505, and the recording medium 514, and stores various data (or information) in the storage unit 5000 and reads various data (or information) from the storage unit 5000. In this embodiment, the storage / reading unit 59 functions as an example of a storage / reading means.

[0102] ●Photographic image information management table● Fig. 14 is a conceptual diagram showing an example of a photographed image information management table according to an embodiment. Note that the structure and contents of the data table described below are merely examples and are not limited thereto. A photographed image information management DB 5001 configured by a photographed image information management table as shown in Fig. 14 is constructed in the storage unit 5000. In the photographed image information management table, a photographed image ID, a photographing date and time, a confidential information candidate ID, photographed image data, and distance information (photographing distance) are stored and managed in association with each other for each tab with a photographing device ID as a tab.

[0103] Among these, the confidential information candidate ID is identification information for identifying a confidential information candidate and is associated with the photographed image data, and the distance information is the distance from the photographing device 3 to the photographed image data.

[0104] In this embodiment, the photographed image information management table (photographed image information management DB 5001) functions as an example of a photographed image information management means and an example of a metadata management means.

[0105] ●Confidential information management table● FIG. 15 is a conceptual diagram showing an example of a confidential information management table according to an embodiment. Note that the structure and contents of the data table described below are merely examples and are not limited thereto. A confidential information management DB 5002 configured by a confidential information management table as shown in FIG. 15 is constructed in the storage unit 3000. In the confidential information management table, for each tab with a confidential information candidate ID as a tab, a data ID, a keyword (character information), a synonym 1, a synonym 2, a synonym 3, a synonym 4, and a confidential information candidate are stored and managed in association with each other. Note that the number of synonyms is not limited thereto.

[0106] Of these, the data ID is identification information that identifies data such as text information and image information contained in the captured image data associated with the confidential information candidate ID. The keyword (character information) represents a keyword contained in the captured image data, and includes, for example, "budget amount," "development team company history," and other content that may need to be treated as confidential information in some cases. The synonym is a word that represents a keyword, or a partial word contained in a keyword. The confidential information candidate represents the result determined in the process performed by the image management device 5, and is represented as a "candidate" or a "non-candidate."

[0107] In this embodiment, the confidential information management table (confidential information management DB 5002) functions as an example of confidential information management means.

[0108] ●Disclosure Management Table● Fig. 16 is a conceptual diagram showing an example of a confidential information management table according to an embodiment. Note that the structure and contents of the data table described below are merely examples and are not limited thereto. A disclosure permission management DB 5003 configured by a disclosure permission management table as shown in Fig. 15 is constructed in the storage unit 3000. In the disclosure permission management table, for each tab with the name of the other party as a tab, the items of data ID, keyword (text information), search result, non-disclosure agreement, and disclosure permission determination are associated and stored and managed. Note that the number of synonyms is not limited thereto.

[0109] Among these, the name of the other party represents the name of the other party participating at another base who shares images, etc. with the own base in a Web conference, etc. The search result represents the result of searching for synonyms by referring to the confidential information management table described above. If there is a synonym, "synonym exists" is registered, and if there is no synonym, "synonym does not exist" is registered. The confidentiality agreement represents the presence or absence of a confidentiality agreement between the participant at the own base (company, etc.) and the participant at the other base (company, etc.), and if there is a confidentiality agreement, "exists" is registered, and if there is no confidentiality agreement, "does not exist" is registered. The disclosure possibility determination is a flag that registers whether or not to disclose the corresponding keyword according to the search result and the confidentiality agreement. Therefore, the image management device 5 gives "possible" when disclosure is possible, and "no" when disclosure is impossible, as the disclosure possibility determination flag. When the disclosure possibility determination is performed, keywords that are usually used and have little relation to confidential information (for example, "next meeting date") are excluded in the selection of keywords.

[0110] In this embodiment, the publication permission management table (publication permission management DB 5003) functions as an example of a publication permission management means.

[0111] ● Processing method management table ● Fig. 17 is a conceptual diagram showing an example of a processing method management table according to an embodiment. Note that the structure and contents of the data table described below are merely examples and are not limited thereto. A processing method management DB 5004 configured by a processing method management table as shown in Fig. 17 is constructed in the storage unit 3000. In the processing method management table, for each tab in which a confidential information candidate ID is a tab, a data ID, a keyword (character information), a disclosure possibility determination, and processing information are associated with each other and stored and managed.

[0112] Among these, the processing information is information that indicates a predetermined processing for keywords that are judged as "no" in the disclosure judgment among the confidential information candidates. The processing information manages the presence or absence of "blurring" and the "blur range", and the "blur range" is changed depending on the display mode, such as the entire captured image data or only a specific keyword in the captured image data.

[0113] In this embodiment, the processing method management table (processing method management DB 5004) functions as an example of a processing method management means.

[0114] [Processing or Operation of the Embodiment] Next, each process or operation in the image display system 1 (image management system 2) according to the embodiment will be described with reference to Fig. 18 to Fig. 24. Fig. 18 is an example of a sequence diagram showing the overall process including the image capture process according to the embodiment. Note that in this embodiment, it is assumed that the communication terminal 7A at the own base and the communication terminal 7B at the other base are in a state in which they can communicate with each other, with login processes of apps etc. started up on the respective communication terminals 7 and establishment of a communication session between the two communication terminals 7.

[0115] 18, a usage scene will be described in which a participation notification for an online event (e.g., an online conference) is transmitted from the communication terminal 7B at the other base to the communication terminal 7A at the own base, and the online conference is started. First, the communication terminal 7B at the other base displays an online conference start screen and receives a start operation (step S11). Specifically, the display control unit 74B causes the display 707 to display a preview screen, which will be described later. Next, the operation receiving unit 72B receives a start operation by a participant at the other base.

[0116] Next, a participant who uses the communication terminal 7B at the other location to participate in a specific online conference operates the participation start screen displayed on the communication terminal 7B and operates a participation button for participating in the online conference. In response to this operation, the transmitting / receiving unit 71B of the communication terminal 7B transmits a participation notification for the online conference to the communication terminal 7A at the own location (step S12). As a result, the transmitting / receiving unit 71A of the communication terminal 7A receives the participation notification transmitted by the communication terminal 7B. At this time, the participation notification includes the participant ID of the participant who first joined the online conference at the other location and the event ID for identifying the online conference.

[0117] Next, the display control unit 74A and the operation reception unit 72A of the communication terminal 7A at the own base display a participation request screen and receive operations by the participant at the own base (step S13). Specifically, the display control unit 74A displays the participation request screen on the display 707 of the communication terminal 7A at the own base. Thereafter, the operation reception unit 72A receives the selection of an image capture device by the participant and various button operations related to the start of the conference. An example of the participation request screen will be described below.

[0118] ●Screen display example● Fig. 19 is a diagram showing an example of a participation request screen according to the embodiment. In Fig. 19, a participation request screen 7111 is displayed by the display control unit 74A. The participation request screen 7111 is, for example, a screen generated by an application or the like started on the communication terminal 7A. The participation request screen 7111 also displays the application name, the online event name, the holding date and time (current date and time), an imaging device selection area, a conference start button 7151, and a cancel button 7152. In addition to these, the participation request screen 7111 may also include a notification indicating that a participant at another location has joined the online conference.

[0119] The imaging device selection area includes a list of imaging devices including the imaging device 3 to be used in the online conference. This allows a participant (user) participating in the online conference to select the imaging device 3 used at his / her own location with a check box. The participation request screen 7111 shows a state in which the 360° camera 1 in the imaging device selection area has been selected by the participant. At this time, the display control unit 74A may set the check boxes of other imaging devices to a state in which they cannot be selected. This is to prevent multiple selection of imaging devices, but this does not apply when multiple imaging devices are used in the online conference. In that case, the display control unit 74A may display an imaging device selection area in which multiple imaging devices can be selected.

[0120] After a participant (user) selects an arbitrary imaging device, the selected imaging device is confirmed by operating the conference start button 7151, and the next process is executed.

[0121] <Shooting process> 18, the transmitter / receiver 71A of the communications terminal 7A transmits connection information with the photographing device 3 to the photographing device 3 using the UVC protocol (step S14). As a result, the transmitter / receiver 31 of the photographing device 3 receives the connection information with the photographing device 3 transmitted from the communications terminal 7A.

[0122] Next, the operation reception unit 32 and the image acquisition unit 33 of the photographing device 3 start up the photographing device and capture an image (step S15). Specifically, the operation reception unit 32 accepts the start of the photographing device 3 by the participant (user) at the own base, and the image acquisition unit 33 captures the surroundings of the own device and acquires a photographed image of the own base. At this time, the calculation processing unit 36 ​​measures the distance between the photographing device 3 and the subject photographed by the photographing device 3. After that, the setting registration unit 38 registers photographed image related information related to the acquired photographed image in each item corresponding to the photographing device ID managed in the photographed image information management DB 3002 (see FIG. 12).

[0123] Next, the transceiver 31 transmits the response of step S14 to the communication terminal 7A using the UVC protocol (step S16). As a result, the transceiver 71A of the communication terminal 7A receives the response transmitted by the photographing device 3.

[0124] Next, the transmitting / receiving unit 71A of the communication terminal 7A transmits a connection confirmation request to the communication terminal 7B (step S17). As a result, the transmitting / receiving unit 71B of the communication terminal 7B receives the connection confirmation request transmitted by the communication terminal 7A. The process of this step S17 is executed, for example, when the domain information of the participant cannot be ascertained even by searching the participant ID (email address, etc.) received in step S12.

[0125] Before the process of step S17 is executed, the communication terminal 7B acquires in advance conference information related to the scheduled online event from a conference management server (e.g., a cloud service of a conference application) connected via the communication network 100. As a result, the communication terminal 7B becomes a communication terminal that has acquired conference information related to the scheduled online event.

[0126] Next, the transmitting / receiving unit 71B of the communication terminal 7B transmits a connection confirmation response to the communication terminal 7A in response to the connection confirmation request received in step S17 (step S18). As a result, the transmitting / receiving unit 71A of the communication terminal 7A receives the connection confirmation response transmitted by the communication terminal 7B. The connection confirmation response includes event destination information (destination base ID, event date and time information, destination name, email address (domain information), and event link information).

[0127] In addition, various information such as online conference information transmitted and received between communication terminal 7B and communication terminal 7A by the connection confirmation request-response performed in steps S17 and S18 may be transmitted in bulk from communication terminal 7B to communication terminal 7A in the processing of step S12 described above.

[0128] Next, the transceiver 71A of the communication terminal 7A transmits a connection confirmation response to the photographing device 3 using the UVC protocol (step S19). As a result, the transceiver 31 of the photographing device 3 receives the connection confirmation response transmitted by the communication terminal 7A.

[0129] Next, the setting registration unit 38 of the photographing device 3 specifies (registers) the event destination information (step S20). Specifically, the setting registration unit 38 registers the event destination information (destination base ID, event date and time information, destination name, email address (domain information), and event link information) included in the connection confirmation response received in step S19 in the corresponding item of the destination information management DB 3003 (see FIG. 13).

[0130] Next, the calculation processing unit 36 ​​combines the captured image data and the event destination information (step S21). Specifically, the calculation processing unit 36 ​​combines the captured image data included in the captured image related information related to the captured image registered in step S15 with the event destination information registered in step S20.

[0131] Next, the calculation processing unit 36 ​​calculates coordinate information related to the captured image data (step S22). Specifically, the calculation processing unit 36 ​​calculates coordinate information (coordinate information of each of the start point and end point) of a rectangle indicating the frame of the captured image data (for example, a whiteboard image) captured and acquired in step S15. The coordinate calculation method at this time may be, for example, a method of giving XY coordinates with any vertex of the rectangular frame of the entire panoramic image shown in Fig. 3 as the origin, and giving each vertex of the rectangular frame indicating the area of ​​the captured image data within the XY coordinates as each coordinate value.

[0132] In the image display system according to the present embodiment, for example, when the processes of steps S12 and S16 described above are executed, other devices such as an online conference server (remote conference server) may exist between the communication terminal 7A at the own base and the communication terminal 7B at the other base. In other words, each piece of information (data) transmitted and received between the communication terminal 7A at the own base and the communication terminal 7B at the other base may be configured to be transmitted and received once via other devices. This allows the communication terminal 7A and the communication terminal 7B to receive, for example, predetermined online conference information from the online conference server and participate in an online conference held between the own base and the other base. The above-mentioned configuration can also be applied even when other processing steps exist between the communication terminal 7A at the own base and the communication terminal 7B at the other base.

[0133] 18, a usage scene has been described in which a participation notification for an online event (e.g., an online conference) is transmitted from the communication terminal 7B at the other base to the communication terminal 7A at the own base, and an online conference is started, but there may also be a case in which a participation notification for an online event (e.g., an online conference) is transmitted from the communication terminal 7A at the own base to the communication terminal 7B at the other base, and an online conference is started. In that case, the above-mentioned step S11 is executed by the communication terminal 7A at the own base. However, in this case, the process of the above-mentioned step S12 does not need to be executed.

[0134] Then, the processes of steps S13 to S16 are similarly executed.

[0135] Next, in step S17, the communication terminal 7A transmits information including the participant ID and the event ID of the own base included in the participation notification executed in the above-mentioned step S12 to the communication terminal 7B instead of a connection confirmation request. After that, the processing of steps S18 to S22 is the same as the contents shown in FIG.

[0136] <Various processing for captured images> 20 is an example of a sequence diagram showing the overall process including the process for the captured image according to the embodiment. Following the coordinate calculation process in step S21 described above, the memory readout unit 39 of the camera device 3 reads the performance information of the camera device 3 (step S31). Specifically, the memory readout unit 39 reads the resolution information as the performance information of the camera device 3 currently being used, which is managed in the camera device information management DB 3001 (see FIG. 11).

[0137] Next, the transmitting / receiving unit 31 transmits the captured image related information to the communication terminal 7A (step S32). Specifically, the transmitting / receiving unit 31 transmits the captured image related information to the communication terminal 7A using the UVC protocol.

[0138] Next, the transmitting / receiving unit 71A of the communication terminal 7A transmits the captured image related information to the image management device 5 (step S33). As a result, the transmitting / receiving unit 51 of the image management device 5 receives the captured image related information transmitted by the communication terminal 7A. At this time, the captured image related information includes the photographing device ID, the captured image data (surrounding environment image data), event destination information, performance information, distance information, and coordinate information.

[0139] Next, the setting registration unit 58 of the image management device 5 registers the captured image information (step S34). Specifically, the setting registration unit 58 registers the captured image information in a corresponding item managed in the captured image information management DB 5001 (see FIG. 14) based on the photographing device ID included in the captured image related information received in step S34.

[0140] <Classified information candidate extraction (registration) process> Next, the extraction and acquisition unit 53 extracts (registers) the confidential information copy (step S35).

[0141] <<Details of the process of extracting (registering) confidential information candidates>> Here, the details of the extraction (registration) of confidential information candidates will be described with reference to Fig. 21. Fig. 21 is a flowchart showing an example of the extraction process of confidential information candidates according to the embodiment. First, the storage and reading unit 59 of the image management device 5 checks the performance information of the photographing device (step S35-1). Specifically, the storage and reading unit 59 reads out the resolution information as the performance information associated with the photographing device ID managed in the photographed image information management DB 5001 (see Fig. 14) from the corresponding tab.

[0142] Next, the extracting and acquiring unit 53 checks the acquired distance information (step S35-2). Specifically, the extracting and acquiring unit 53 searches the photographed image information management DB 5001 using the photographing device ID as a search key via the storage and reading unit 59, thereby reading all distance information related to the corresponding photographed image data.

[0143] Next, the extraction and acquisition unit 53 checks the character information included in the target range in the captured image based on the performance information and distance information of the image capturing device (step S35-3). Specifically, the extraction and acquisition unit 53 extracts the confidential information part included in the captured image data by using, for example, OCR technology. Thereafter, the extraction and acquisition unit 53 checks the text information contained in the target range of the captured image for the extracted confidential information portion using the performance information (resolution information) of the imaging device read in steps S35-1 and S35-2 and the distance information related to each captured image data.

[0144] Next, the generating unit 57 generates a confidential information candidate list and exits this flow (step S35-4). Specifically, the generating unit 57 generates confidential information candidates based on the character information included in the target range in the captured image confirmed in step S35-3, and registers the candidates in the corresponding items in the confidential information management DB 5002 (see FIG. 15).

[0145] <Synonym search and image processing file generation> Returning to Fig. 20, the extraction and acquisition unit 53 of the image management device 5 searches for synonyms of the extracted information (keyword) (step S36). Specifically, the extraction and acquisition unit 53 performs a synonym search for keywords represented by character information managed in the confidential information management DB 5002 (see Fig. 15), and registers for each keyword whether it is a confidential information candidate. In this synonym search, the extraction and acquisition unit 53 searches whether the searched keyword corresponds to confidential information. Note that when searching whether the keyword corresponds to confidential information, the extraction and acquisition unit 53 may refer to information (partner information) of other bases that share information with its own base.

[0146] Next, the generating unit 57 generates confidential information pertinence determination data and an image processing setting file (setting information program) based on the search results (step S37). Specifically, the generating unit 57 generates confidential information pertinence determination data, which is an example of a pertinence flag for determining whether or not the disclosure is possible for each keyword for which the thesaurus search was performed. More specifically, the generating unit 57 generates confidential information pertinence determination data based on the search results using conceptual diagram data showing an example of the relationship between the resolution and the target distance as shown in FIG. 5 and the disclosure pertinence determination result associated with the thesaurus search. At this time, the generating unit 37 may cooperate with the area specifying unit 56 to generate new data specifying an area of ​​the image data including the confidential information pertinence determination data. The generating unit 37 also generates an image processing setting file based on specific keyword information and the presence or absence of a non-disclosure agreement between the own location and another location. The generation unit 57 further generates an image processing setting file as a setting information program related to the display mode executed by the photographing device 3, and registers processing information such as "blurring" to be performed for each keyword in the processing information item managed in the processing method management DB 5004 (see Figure 17).

[0147] In addition, when the image capturing device 3 supports multiple display modes, a setting information program is generated that allows different processing methods to be set according to the display mode set by the display control unit 34 of the image capturing device 3. For example, when using an image capturing device 3 that supports a display mode including a close-up screen, a setting information program is generated that targets only the confidential information portion included in the panoramic image, not the entire close-up screen, and minimizes the processing area as much as possible. This makes it possible to ensure security at each location while maintaining the sense of realism of the Web conference.

[0148] Next, the transmitting / receiving unit 51 transmits the image processing setting file generated in step S37 to the communication terminal 7A (step S38). As a result, the transmitting / receiving unit 71A of the communication terminal 7A receives the image processing setting file transmitted by the image manager 5. The image processing setting file at this time includes unprocessed screen data to be displayed on the communication terminal 7B at the other base.

[0149] Next, the transmitting / receiving unit 71A of the communication terminal 7 transmits the image processing setting file to the photographing device 3 (step S39). As a result, the transmitting / receiving unit 31 of the photographing device 3 receives the image processing setting file transmitted by the communication terminal 7A.

[0150] Next, the operation acceptance unit 32 of the photographing device 3 accepts a display mode switching operation, and the display control unit 34 executes a display mode switching process (step S40).

[0151] Next, the generating unit 37 generates an image according to the display mode (step S41). Specifically, the generating unit 37 cooperates with the calculation processing unit 36 ​​to perform image processing on the unprocessed screen data based on the setting information program received in step S39 and the selected display mode, and generates preview screen data. More specifically, the generating unit 37 generates preview screen data representing a preview screen of a processed image obtained by performing a predetermined processing on the captured image, which can be displayed on a different base terminal (communication terminal 7B) used at a different base that shares the captured image of the own base, based on a predetermined display mode and an image processing setting file for performing a predetermined processing on the captured image. That is, the generating unit 37 performs a process of blurring the entire captured image or a specific part of the captured image as the predetermined processing. The generation unit 37 performs a blurring process on the 360° panoramic image around the shooting device, and generates close-up images of specific areas (such as the speaker or the display area of ​​the electronic whiteboard device) of the blurred panoramic image, thereby generating preview screen data that includes the blurred panoramic image and the blurred close-up image.

[0152] Next, the transmitting / receiving unit 31 transmits the preview screen data to the communication terminal 7A using the UVC protocol (step S42). As a result, the transmitting / receiving unit 71A of the communication terminal 7A receives the preview screen data transmitted by the photographing device 3. In other words, the transmitting / receiving unit 31 transmits the preview screen data to the base terminal (communication terminal 7A) so that the preview screen is displayed on the other base terminal (communication terminal 7B).

[0153] Next, the transmitting / receiving unit 71A of the communication terminal 7A transmits the preview screen data to the communication terminal 7B (step S43). As a result, the transmitting / receiving unit 71B of the communication terminal 7B receives the preview screen data transmitted by the communication terminal 7A. In detail, the communication terminal 7A transmits the preview screen data resulting from a predetermined processing process performed by the image capturing device 3 connected to the communication terminal 7A via the remote conference app of the communication terminal 7A so that it is displayed on the communication terminal 7B (an example of a terminal at another base).

[0154] In the image display system according to the present embodiment, for example, when the processes of steps S33 and S38 described above are executed, other devices or the like may exist between the communication terminal 7A at the home base and the image management device 5. That is, each piece of information (data) transmitted and received between the communication terminal 7A at the home base and the image management device 5 may be configured to be transmitted and received via other devices or the like once. The above-described configuration can be applied even when other processing steps exist between the communication terminal 7A at the home base and the image management device 5.

[0155] FIG. 22 is an example of a sequence diagram including the processing after the preview screen is displayed according to the embodiment. In response to the processing of step S43 described above, the communication terminal 7B at the remote base displays the preview screen data (step S51). Specifically, the display control unit 34B of the communication terminal 7B causes the preview screen data related to the preview screen data received in step S43 to be displayed on the display 707.

[0156] Subsequently, the processing after the start of the meeting will be described. The operation reception unit 72B of the communication terminal 7B executes a process for determining whether or not a meeting start has been selected (step S52).

[0157] <Processing after the start of the Web conference: When a meeting start is selected> When a meeting start is selected, the image display system 1 executes the following as processing after the start of the meeting. First, the imaging device 3 executes processing after preparation for the Web conference (step S53). Details of this processing will be described below.

[0158] <<Details of the processing after preparation for the Web conference>> 23 is a flow chart showing an example of processing after preparation for a Web conference according to an embodiment. First, the image capturing unit 33 of the image capturing device 3 monitors changes in the captured image of the image capturing device 3 (step S53-1). Specifically, the image capturing unit 33 drives the imaging unit 301, image processing unit 304, and imaging control unit 305 of the image capturing device 3 to monitor whether there is a change in the surrounding situation that will become the captured image of the own location. The timing and length of monitoring may be arbitrary, and may be performed every 5 seconds, for example.

[0159] Next, the determination unit 35 determines whether there is a change in the captured image (step S53-2). Specifically, the determination unit 35 compares the surroundings of the own base (image to be captured) monitored, for example, 5 seconds ago with the current surroundings of the own base (image to be captured), and determines whether there is a difference.

[0160] When it is determined that there is no change in the captured image (step S53-2: NO), the determination unit 35 executes the process of step S53-1 again.

[0161] When it is determined that there is a change in the photographed image (step S53-2: YES), the photographing and acquiring unit 33 photographs the surroundings of the photographing device 3 again and acquires a photographed image of the own base (step S53-3).

[0162] Next, the determination unit 35 determines whether the captured image is partially or entirely hidden (step S53-4).

[0163] When it is determined that a part or the whole of the photographed image is hidden (step S53-4: YES), the calculation processing unit 36 ​​processes the confidential information candidate excluding the keyword included in the hidden photographed image (step S53-5). Specifically, the determination unit 35 determines that a part or the whole of the photographed image is hidden when, for example, the photographed image data including the confidential information candidate is in the shadow of a participant participating in an online event and the confidential information candidate is not visible or is unreadable. When it is determined as such, the calculation processing unit 36 ​​processes the confidential information candidate excluding the keyword included in the hidden photographed image.

[0164] Thereafter, the photographing device 3 re-executes the processes of steps S13-S21 and S31-S41 described above (step S53-6).

[0165] Next, the judgment unit 35 judges whether the online event has ended (step S53-7), and if it is judged that the online event has ended (step S53-7: YES), the judgment unit 35 exits this flow. On the other hand, if it is judged that the online event has not ended (step S53-7: NO), the judgment unit 35 returns to the process of step S53-1.

[0166] On the other hand, if it is determined in the judgment process of step S53-4 that the captured image is not hidden (step S53-4: NO), the calculation processing unit 36 ​​processes all of the confidential information candidates (step S53-8), and the process proceeds to the judgment unit 35's decision on the end of the online event in step S53-7.

[0167] In this way, the photographing device 3 monitors changes in the background image. In this case, if the background image changes during the online event, the photographing device 3 photographs the surroundings of the photographing device again to obtain the photographed image and transmits it to the image management device 5 via the communication terminal 7A. After that, a confidential information search is requested from the image management device 5, and the image management device 5 again performs a confidential information classification. As a result, even if confidential information is added to a whiteboard used during the online event, for example, the confidential information classification can be performed again and the display content can be changed according to the result.

[0168] 22, the transmitting / receiving unit 31 of the photographing device 3 transmits the preview screen data to the communication terminal 7B via the UVC protocol (step S54). As a result, the transmitting / receiving unit 71A of the communication terminal 7A receives the preview screen data transmitted by the photographing device 3.

[0169] Next, the transmitting / receiving unit 71A of the communication terminal 7 transmits preview screen data to the communication terminal 7B (step S55). As a result, the transmitting / receiving unit 71B of the communication terminal 7B receives the preview screen data transmitted by the communication terminal 7A. Thereby, the communication terminal 7B can display the preview screen data edited (processed) by the imaging device 3 on the display 707.

[0170] <Processing after Web conference starts: When conference start is not selected> On the other hand, if the communication terminal 7A has not selected to start the conference in the process of determining whether to select to start the conference in step S52, the communication terminal 7A executes a re-shooting request selection process (step S56). Specifically, the display control unit 74A of the communication terminal 7A displays a re-shooting request selection screen for selecting whether to request re-shooting on the display 707.

[0171] If a re-shooting request is selected, the image display system 1 executes the processes of steps S14 to S52 described above again (step S57).

[0172] On the other hand, if no re-shooting request is selected, the communication terminal 7B displays the preview screen again in step S51 (step S58). Note that the selection of whether there is a re-shooting request may be executed after a predetermined time has elapsed after the process of determining whether there is a conference start selection is executed.

[0173] <Example of processing for the captured image to be displayed on the communication terminal at another site> Here, an example of processing for the captured image to be displayed on the communication terminal 7 at another site will be described. Here, the change in the screen displayed on the communication terminal 7 at another site when the process of step S53-3 described above is included will be described.

[0174] ●Example of screen display● Fig. 24A is a diagram showing an example of a receiving screen at the other base side according to the embodiment. In Fig. 24A, a receiving screen 7211 is displayed on the display 707 by the display control unit 74B of the communication terminal 7B. The receiving screen 7211 shows a state in which "confidential information candidate 2" in the close-up image 3 has been subjected to "blurring processing." At this time, "confidential information candidate 2" in the close-up image 3 in the panoramic image has also been subjected to "blurring processing."

[0175] ●Screen display example● Fig. 24B is a diagram showing another example of a receiving screen at the other base side according to the embodiment. In Fig. 24B, a receiving screen 7212 is displayed on the display 707 by the display control unit 74B of the communication terminal 7B. The receiving screen 7212 shows a state in which a new "blurring process" has been applied since the state of the receiving screen 7211 has been changed to "classified information candidate'" in the close-up image 1. At this time, the "blurring process" is also applied to the "classified information candidate'" in the close-up image 1 in the panoramic image.

[0176] ●Screen display example● Fig. 24C is a diagram showing another example of the receiving screen at the other base side according to the embodiment. In Fig. 24C, a receiving screen 7213 is displayed on the display 707 by the display control unit 74B of the communication terminal 7B. The receiving screen 7213 shows a state in which, since the state of the receiving screen 7212, the result of the confidential information applicability determination for the "confidential information candidate'" in the close-up image 1 has turned out to be not confidential information, the "blurring process" has been cancelled.

[0177] ●Screen display example● FIG. 24D is a diagram showing another example of a receiving screen at the other base side according to the embodiment. In FIG. 24C, a receiving screen 7214 is displayed on the display 707 by the display control unit 74B of the communication terminal 7B. In the receiving screen 7214, the "confidential information candidate 2" in the close-up image 3 is hidden by the head of the participant. This shows a case where the "confidential information candidate 2" cannot be judged as confidential information or the result of the confidential information judgement is judged not to be confidential information. This causes the "confidential information candidate 2" in the close-up image 3 to be displayed as it is without performing "blurring processing". In other words, even if the contents of the confidential information candidate correspond to confidential information, if a state occurs in which the information is covered or hidden, processing such as "blurring processing" may not be performed. This makes it possible to continue an online event with a more realistic feel.

[0178] [Main Effects of the Embodiments] As described above, according to this embodiment, the photographing device 3 photographs the surroundings of the photographing device 3 to obtain a photographed image of the own base (S14), accepts a setting of a predetermined display mode for displaying the photographed image (S40), generates preview screen data representing a preview screen of a processed image obtained by performing a predetermined processing on the photographed image, which can be displayed on a different base terminal (communication terminal 7B) used at a different base that shares the photographed image of the own base based on the predetermined display mode and an image processing setting file for performing a predetermined processing on the photographed image (S41), and transmits the preview screen data to the own base terminal (communication terminal 7A) so that the preview screen is displayed on the different base terminal (communication terminal 7B) (S42). This provides the effect that, when a photographed image of the own base is shared at a different base, a processed screen obtained by performing a predetermined processing as necessary can be provided to the different base.

[0179] Furthermore, according to this embodiment, the image capture device 3 uses a setting information program to perform processing to minimize the processing area by targeting only the confidential information portion included in the panoramic image, rather than the entire close-up screen. This makes it possible to ensure security at each location while maintaining the sense of realism of online events such as web conferences.

[0180] Furthermore, according to this embodiment, even if confidential information is added to a whiteboard being used during an online event, the confidential information can be determined again and the display content can be changed according to the result.

[0181] [Supplementary description of the embodiment] Each function of the above-mentioned embodiment can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a device programmed to execute each function by software, such as a processor implemented by an electronic circuit. This device refers to, for example, a processor, an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), a SOC (System on a chip), a GPU (Graphics Processing Unit), and a conventional circuit module designed to execute each function described above.

[0182] In the above-described embodiment, the image display system 1 may be configured to execute various judgments in the above-described various processes by machine learning (learning using artificial intelligence: AI), thereby eliminating the need to use corresponding data tables.

[0183] Here, machine learning is a technology for making a computer acquire a human-like learning ability, and refers to a technology in which a computer autonomously generates an algorithm required for judgment such as data identification from learning data that is previously loaded, and applies this to new data to make predictions. The learning method for machine learning may be any of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning. The learning method for machine learning may further be a combination of these learning methods, and any learning method for machine learning is acceptable.

[0184] So far, an image management system, an image capturing device, an image processing method, and a program according to one embodiment of the present invention have been described. However, the present invention is not limited to the above-described embodiment, and other embodiments can be added, modified, or deleted, etc., within the scope that can be conceived by a person skilled in the art. In any aspect, as long as the functions and effects of the present invention are achieved, it is included in the scope of the present invention.

[0185] ■ Summary ■ For example, aspects of the present invention are as follows.

[0186] <First aspect> The photographing device 3 (an example of a photographing device; omitted below) in the image management system 2 (an example of an image management system; omitted below) as a first embodiment has an photographing acquisition unit 33 (an example of a photographing acquisition means; omitted below) that photographs the surroundings of the photographing device 3 to acquire photographed images of its own base, an operation receiving unit 32 (an example of a receiving means; omitted below) that receives a setting of a predetermined display mode for displaying the photographed image, a generation unit 37 (an example of a preview screen data generation means; omitted below) that generates preview screen data representing a preview screen of a processed image obtained by subjecting the photographed image to a predetermined processing process, which can be displayed on a communication terminal 7B (an example of a terminal at another base; omitted below) used at another base that shares the photographed image of its own base, based on the predetermined display mode and an image processing setting file for subjecting the photographed image to a predetermined processing process, and a transmission / reception unit 31 (an example of a preview screen data transmission means; omitted below) that transmits the generated preview screen data so that the preview screen is displayed on the communication terminal 7B.

[0187] According to the first aspect, when an image taken at one's own base is shared with another base, it is possible to provide the other base with a processed screen that has been subjected to a predetermined processing process as necessary.

[0188] <Second aspect> The image management system 2 in the second embodiment, in the first embodiment, further has an image management device 5 (an example of an image management device; omitted below), and the image management device 5 has a transmission / reception unit 51 (an example of a captured image related information receiving means; omitted below) that receives captured image related information relating to images captured at the camera's own base by the camera 3, a generation unit 57 (an example of an image processing setting file generation means; omitted below) that generates an image processing setting file based on specific keyword information contained in the captured image related information, and the transmission / reception unit 51 (an example of an image processing setting file transmission means; omitted below) that transmits the generated image processing setting file to be set in the camera 3.

[0189] According to the second aspect, it is possible to obtain the same effect as the first aspect.

[0190] <Third aspect> The image management system 2 in the third embodiment further has a communication terminal 7A (an example of a local base terminal; omitted below) used at the local base in the second embodiment, and the communication terminal 7A has a transmission / reception unit 71 (an example of a transmission means; omitted below) that transmits the image processing setting file sent by the image management device 5 to the photographing device 3.

[0191] According to the third aspect, it is possible to obtain the same effect as the first aspect.

[0192] <Fourth aspect> The image processing setting file in the image management system 2 as the fourth aspect is a setting file for carrying out a predetermined processing for concealing confidential information included in a captured image in any of the first to third aspects.

[0193] According to the fourth aspect, it is possible to obtain the same effects as those of the first aspect.

[0194] <Fifth aspect> The generation unit 57 of the image management device 5 in the image management system 2 as the fifth aspect generates an image processing setting file in any of the second to fourth aspects based on specific keyword information and the presence or absence of a confidentiality agreement between the own location and another location.

[0195] According to the fifth aspect, in addition to the effect of the first aspect, it becomes possible to provide a processed screen, in which information related to a non-disclosure agreement has been subjected to a predetermined processing, to another base.

[0196] <Sixth aspect> In the image management system 2 as the sixth aspect, in any of the first to fifth aspects, if there is a change between the captured image of the own location and the image of the own location after a predetermined time has elapsed, the image capturing unit 33 of the photographing device 3 re-captures the surroundings of the photographing device 3 and captures a re-captured image of the own location.

[0197] According to the sixth aspect, in addition to the effect of the first aspect, even if confidential information is added to the captured image data, it is possible to re-determine whether the confidential information is included and change the display content according to the result.

[0198] <Seventh aspect> The photographing device 3 in the image management system 2 as the seventh aspect, in any of the third to sixth aspects, further has a calculation processing unit 36 ​​(an example of a distance measuring means; omitted below) that measures the distance between the photographing device 3 and the subject photographed by the photographing device 3, and the transmission / reception unit 31 adds distance information indicating the distance to the preview screen data and transmits it to the communication terminal 7A.

[0199] According to the seventh aspect, by determining whether or not to apply a specified processing step depending on the distance between the photographic image data, which is part of the subject, and the photographing device 3, it is possible to provide a more realistic image without applying the specified processing step unnecessarily.

[0200] <Eighth aspect> In the seventh mode, the transmitting / receiving unit 31 of the photographing device 3 in the image management system 2 as the eighth mode adds resolution information indicating the performance of the photographing device 3 to the preview screen data and transmits the data to the communication terminal 7A.

[0201] According to the eighth aspect, in addition to the effect of the seventh aspect, by applying processing in accordance with the performance of the image capturing device 3, it is possible to provide a more realistic image while fully utilizing the performance of the image capturing device 3.

[0202] <Ninth aspect> The generation unit 37 of the photographing device 3 in the image management system 2 as a ninth aspect generates preview screen data in which, as a specified processing process, a process of blurring the entire photographed image or a specific part thereof has been performed in any of the first to eighth aspects.

[0203] According to the ninth aspect, in addition to the effect of the first aspect, it becomes possible to perform appropriate processing on information (captured image data) that should be kept secret.

[0204] <Tenth aspect> In the ninth mode, the photographing device 3 as a tenth mode generates a 360° panoramic image around the photographing device 3, and the preview screen data generation means performs a blurring process on the panoramic image and generates a close-up image that zooms in on a specified area of ​​the blurred panoramic image, thereby generating preview screen data that includes the blurred panoramic image and the blurred close-up image.

[0205] According to the tenth aspect, like the ninth aspect, it becomes possible to perform appropriate processing on information (captured image data) that should be kept secret.

[0206] <Eleventh aspect> The communication terminal 7 of the 11th aspect is a communication terminal having a remote conference app in any of the third to tenth aspects, and the communication terminal 7 includes a PC or an electronic whiteboard device, and the communication terminal 7A transmits preview screen data resulting from a predetermined processing process from the photographing device 3 connected to the communication terminal 7A via the remote conference app so that the data is displayed on the communication terminal 7B (an example of a terminal at another base station).

[0207] According to the eleventh aspect, even when a PC, an electronic whiteboard device, or the like is used, when an image captured at one's own location is shared with another location, it is possible to provide the other location with an edited screen that has been subjected to a specified processing step as necessary.

[0208] <Twelfth aspect> The photographing device 3 (an example of a photographing device; omitted below) as a twelfth aspect has an photographing acquisition unit 33 (an example of a photographing acquisition means; omitted below) that photographs the surroundings of the photographing device 3 and acquires a photographed image of its own base, an operation receiving unit 32 (an example of a receiving means; omitted below) that accepts the setting of a predetermined display mode for displaying the photographed image, a generation unit 37 (an example of a preview screen data generation means; omitted below) that generates preview screen data representing a preview screen of a processed image obtained by subjecting the photographed image to a predetermined processing process, which can be displayed on a communication terminal 7B (an example of a terminal at another base; omitted below) used at another base that shares the photographed image of its own base, based on the predetermined display mode and an image processing setting file for subjecting the photographed image to a predetermined processing process, and a transmission / reception unit 31 (an example of a preview screen data transmission means; omitted below) that transmits the generated preview screen data so that the preview screen is displayed on the communication terminal 7B.

[0209] According to the twelfth aspect, as in the first aspect, when an image taken at one's own base is shared with another base, it becomes possible to provide the other base with an edited screen that has been subjected to a specified processing as necessary.

[0210] <Thirteenth aspect> The photographing processing method performed by the photographing device 3 (an example of a photographing device; omitted below) as the 13th aspect performs processing including: a photographing acquisition step of photographing the surroundings of the photographing device 3 to acquire a photographed image of its own base; a reception step of accepting the setting of a predetermined display mode for displaying the photographed image; a preview screen data generation step of generating preview screen data representing a preview screen of a processed image obtained by subjecting the photographed image to a predetermined processing process, which can be displayed on a communication terminal 7B (an example of a terminal at another base; omitted below) used at another base that shares the photographed image of its own base, based on the predetermined display mode and an image processing setting file for subjecting the photographed image to a predetermined processing process; and a transmission step of transmitting the generated preview screen data so that the preview screen is displayed on the communication terminal 7B.

[0211] According to the thirteenth aspect, as in the first aspect, when an image taken at one's own base is shared with another base, it becomes possible to provide the other base with an edited screen that has been subjected to a specified processing as necessary.

[0212] <14th aspect> The program as the 14th aspect causes the photographing device 3 (an example of a photographing device; omitted below) to execute processing including an imaging acquisition step of photographing the surroundings of the photographing device 3 to acquire a photographed image of its own base, a reception step of accepting a setting of a predetermined display mode for displaying the photographed image, a preview screen data generation step of generating preview screen data representing a preview screen of a processed image obtained by subjecting the photographed image to a predetermined processing process, which can be displayed on a communication terminal 7B (an example of a terminal at another base; omitted below) used at another base that shares the photographed image of its own base, based on the predetermined display mode and an image processing setting file for subjecting the photographed image to a predetermined processing process, and a transmission step of transmitting the generated preview screen data so that the preview screen is displayed on the communication terminal 7B.

[0213] According to the fourteenth aspect, as in the first aspect, when an image taken at one's own base is shared with another base, it becomes possible to provide the other base with an edited screen that has been subjected to a specified processing as necessary. [Explanation of symbols]

[0214] 1 Image display system 2. Image Management System 3. Imaging Equipment 5. Image management device 7A Communication terminal 7B Communication terminal 31 Transmission / reception unit (an example of a preview screen data transmission means, an example of a reception means) 32 Operation reception unit (an example of a reception means) 33 Image capture unit (an example of an image capture means) 34 Display control unit (an example of a display control means) 35 Judgment unit (an example of a judgment means) 36 Calculation processing unit (an example of a calculation means, an example of a distance measuring means) 37 Generation unit (an example of a preview screen data generation means) 38 Setting registration unit (an example of a setting means, an example of a registration means) 39 Memory readout unit (an example of a memory readout means) 3000 Memory unit (an example of a memory means) 51 Transmission / reception unit (an example of an image processing setting file transmission means, an example of a captured image related information reception means) 53 Extraction acquisition unit (an example of an extraction acquisition means) 54 Display control unit (an example of a display control means) 56 Area specification unit (an example of a specification means) 57 Generation unit (an example of an image processing setting file generation means) 58 Setting registration unit (an example of a setting means, an example of a registration means) 71A Transmitting / receiving unit (an example of a transmitting means, an example of a receiving means) 72A, 72B Operation reception unit (an example of a reception means) 73A, 73B Acquisition unit (an example of an acquisition means) 74A, 74B Display control unit (an example of a display control means) 77A, 77B Generation unit (an example of a generation means) 707A, 707B Display (an example of a display means) [Prior art documents] [Patent documents]

[0215] [Patent Document 1] JP 2019-219736 A

Claims

1. A program for transmitting captured images from one location to a terminal used at another location in order to share the captured images from one location, On the computer, An extraction step, which extracts confidential information contained in the captured image based on confidential information management data stored in the memory unit, Image generation step: Based on settings for applying predetermined processing, image data is generated in which confidential information contained in the captured image has undergone the predetermined processing; A program to execute.

2. A setting data generation step, which generates image processing setting data for performing the predetermined processing based on the captured image-related information contained in the captured image, The program according to claim 1, for causing a computer to execute the following.

3. The image generation step includes a predetermined processing process that conceals the confidential information contained in the captured image based on the image processing setting data. The program according to claim 2, which causes a computer to execute the following.

4. The setting data generation step is a process of generating the image processing setting data based on whether or not there is a confidentiality agreement between the local site and the other site. The program according to claim 2, which causes a computer to execute the following.

5. If there is a change between the image of the site and the image of the site after a predetermined time has elapsed, the surrounding area of ​​the shooting device is photographed again to obtain a new image of the site, shooting step The program according to claim 1, for causing a computer to execute the following.

6. A distance measuring step of measuring the distance between a shooting device that acquires the captured image and a subject that is photographed by the shooting device, An information addition step of adding distance information representing the distance to the image data, Furthermore, the program according to claim 1 for causing a computer to execute it.

7. The information addition step involves adding resolution information representing the performance of the imaging device to the image data. The program according to claim 6, which causes a computer to execute the following.

8. The image generation step involves generating image data in which the entire or a specific part of the captured image is blurred as a predetermined processing step. The program according to claim 1, which causes a computer to execute the following.

9. The aforementioned imaging device generates a panoramic image of the 360° surrounding the imaging device. The image generation step involves blurring the panoramic image and generating a close-up image by zooming in on a predetermined area of ​​the blurred panoramic image, thereby generating image data that includes the blurred panoramic image and the blurred close-up image. The program according to claim 6, which causes a computer to execute the following.

10. A shooting device for acquiring a captured image, A means for extracting confidential information contained in the captured image based on confidential information management data stored in the memory unit, A means for generating image data in which confidential information contained in the captured image has been subjected to the predetermined processing based on settings for performing the predetermined processing, Having, A photographic device characterized by the following features.

11. A method for performing imaging processing on an imaging device, The process includes a shooting and acquisition step where the shooting device is used to photograph the surrounding area and acquire images of the site, An extraction step, which extracts confidential information contained in the captured image based on confidential information management data stored in the memory unit, Image generation step: Based on settings for applying predetermined processing, image data is generated in which confidential information contained in the captured image has undergone the predetermined processing; Execute the process that includes A method for processing images characterized by the following features.

12. An image display system comprising: a shooting device at one location, an image management device for processing images acquired by the shooting device, and a terminal at another location, all connected to each other via a network, The aforementioned imaging device transmits the acquired images of its own location to the image management device. The image management device extracts confidential information contained in the captured image based on confidential information management data stored in the memory unit. Based on the settings for applying a predetermined processing treatment, image data is generated in which the confidential information contained in the captured image has been subjected to the predetermined processing treatment. The aforementioned terminal at the other location displays the image data generated by the image management device. Characterized by, Image display system.

Images