Image transmission system and image transmission method

Abstract

To provide technology with which, while taking into account the privacy of a person whose image is acquired, it is possible to cause the connection with a person existing in a different area to be felt by a person existing in some area, giving a peace of mind and a sense of solidarity.SOLUTION: An image transmission system executes: a process of imaging a first person existing in a first area with a first camera and acquiring a first image; a process of generating a first person silhouette image that represents the silhouette of the first person; a process of projecting or showing the first person silhouette image to a second object installed in a second area different from the first area; a process of determining on the basis of the first image, whether or not the first person performed a first trigger action; and a process of projecting or showing a first person detail image closer to the image of the first person itself than the first person silhouette image to the second object when the first person has performed the first trigger action.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present disclosure relates to an image transmission system and an image transmission method. [Background technology]

[0002] Patent Document 1 discloses an information gathering method for responding to crimes or accidents occurring in a specific area by using a network camera to collect information on illegal or unlawful activities, etc. In the information gathering method disclosed in Patent Document 1, two or more network cameras are installed on the streets of a shopping district, and each camera captures an image within a designated viewing area, and the images are saved. Then, a recording screen capturing an image of a person committing a crime or the entry and exit routes of the person committing the crime to the scene of the crime is selected, and the selected screen is stored as image information in a storage medium. The storage medium can be submitted to the police or other authorities and used to solve the case.

[0003] Patent Document 2 discloses a display control device that aims to enable smooth communication between users who are simultaneously viewing content. Information about one user acquired by a camera, microphone, etc. is output to another user, giving the user the feeling that they are in a lively atmosphere with other users.

[0004] In addition to Patent Documents 1 and 2 above, Patent Documents 3 to 6 below can be cited as examples of documents that show the technical level at the time of filing in the technical field of the present disclosure. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2003-317168 [Patent Document 2] International Publication No. 2014 / 192552 [Patent Document 3] International Publication No. 2019 / 097802 [Patent Document 4] Patent Publication No. 2021-027384 [Patent Document 5] International Publication No. 2011 / 043100 [Patent Document 6] Japanese Patent Application Laid-Open No. 2004-348078 Summary of the Invention [Problem to be solved by the invention]

[0006] We consider using network-connected cameras and display devices to improve the sense of connection and communication between people. By using network-connected cameras and display devices to project images of people in one place onto another, it is possible to create a sense of connection. However, in this case, it is also important to consider the privacy of the people whose images are captured.

[0007] The purpose of the present disclosure is to provide technology that can make a person in one area feel connected to a person in a different area, giving them a sense of security and solidarity, while respecting the privacy of the person whose image is being captured. [Means for solving the problem]

[0008] The first aspect relates to an image transmission system. The image transfer system includes one or more processors. The one or more processors A process of capturing an image of a first person present in a first area with a first camera to obtain a first image; generating a first person silhouette image representing a silhouette of the first person; a process of projecting or displaying the first person silhouette image on a second object installed in a second area different from the first area; determining whether a first person has performed a first trigger action based on the first image; a process of projecting or displaying, on a second object, a detailed image of the first person that is closer to an image of the first person than the silhouette image of the first person when the first person performs a first trigger action; is configured to execute

[0009] The second aspect relates to an image transmission method. The image transmission method is A process of acquiring a first image by a first camera present in a first area; generating a first person silhouette image representing a silhouette of a first person included in the first image; a process of projecting or displaying the first person silhouette image on a second object installed in a second area different from the first area; determining whether a first person has performed a first trigger action based on the first image; a process of projecting or displaying, on a second object, a detailed image of the first person that is closer to an image of the first person than the silhouette image of the first person when the first person performs a first trigger action; Includes. [Effects of the Invention]

[0010] According to the technology of the present disclosure, it is possible to give a person in one area a sense of connection with a person in a different area, while respecting the privacy of the person whose image is being captured, thereby giving them a sense of security and solidarity. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram showing an example of the configuration of an image transmission system according to an embodiment of the present invention; [Figure 2] FIG. 10 is a block diagram showing another example of the configuration of the image transmission system according to the present embodiment. [Figure 3] 1 is a schematic diagram showing an example of a scene in which an image transmission system according to an embodiment of the present invention is applied; [Figure 4] FIG. 10 is a schematic diagram showing an example of a first detailed person image. [Figure 5]10 is a flowchart illustrating an example of an image transmission mode setting process executed by the image transmission system according to the present embodiment. [Figure 6] 10 is a flowchart illustrating an example of an image transmission process executed by the image transmission system according to the present embodiment. [Figure 7] 10 is a flowchart illustrating another example of the image transmission mode setting process. [Figure 8] 10 is a flowchart illustrating yet another example of the image transmission mode setting process. [Figure 9] 1 is a block diagram showing an example of the configuration of an image transmission system according to an embodiment of the present invention; [Figure 10] 1 is a schematic diagram showing an example of a scene in which an image transmission system according to an embodiment of the present invention is applied; [Figure 11] 10 is a flowchart illustrating an example of an image transmission mode setting process. [Figure 12] 1 is a schematic diagram showing an example of a scene in which an image transmission system according to an embodiment of the present invention is applied; [Figure 13] FIG. 10 is a schematic diagram showing another example of a scene in which the image transmission system according to the present embodiment is applied. DETAILED DESCRIPTION OF THE INVENTION

[0012] Embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0013] 1. Overview 1 shows the minimum configuration of an image transmission system 100 according to this embodiment. The image transmission system 100 includes an image processing device 110 and a projection / display device 120.

[0014] The image processing device 110 is a device that processes images acquired from a camera or the like, or processes data acquired in advance, to generate various images to be projected or displayed by the projection / display device 120. The various images that are generated include images of real people captured by a camera or the like, images of real people represented in silhouette, images of real people represented as avatars, images of non-existent characters represented in silhouette, etc. When generating an image of a real person, the image processing device 110 can generate the image in real time from images captured by the camera so that the person's current movements can be seen.

[0015] Note that an image of a real person includes an image obtained by removing the background from an image captured by a camera or other device, and extracting only the person. A silhouette image of a person or character is an image in which the outline of the person or character is filled in with black so that only the outline is visible. An avatar image of a person is an image of a character that has been processed to look like an illustration or 3D model, and which expresses the person's characteristics while abstracting unnecessary parts.

[0016] The projection / display device 120 is a device that projects or displays the image generated by the image processing device 110. The projection / display device 120 may be a projector that projects an image onto a projection surface such as a wall or a screen, or may be a device that displays an image on a display.

[0017] The image processing device 110 includes one or more processors 111 (hereinafter simply referred to as processors 111) and one or more storage devices 112 (hereinafter simply referred to as storage devices 112). The storage devices 112 store a plurality of programs executable by the processor 111 and various data related thereto. The processor 111 executes the programs stored in the storage devices 112, thereby enabling the image processing device 110 to perform various processes including image generation.

[0018] Such an image transmission system 100 can be used in a variety of situations.

[0019] 2. First example of use 2-1.Configuration example 2 shows an example configuration of an image transmission system 100 in a first application example. The image transmission system 100 includes an image capturing device 130 in addition to an image processing device 110 and a projection / display device 120. The image capturing device 130 is a device for capturing images and includes a first camera 131. The image processing device 110 can acquire the image captured by the image capturing device 130 and generate an image to be projected or displayed by the projection / display device 120 based on the acquired image. The projection / display device 120 includes a second device 122 for projecting or displaying an image on a second object O2.

[0020] In a first application example, the image transmission system 100 is used as a system that enables a person in one location to feel a connection with a person in a different location, thereby improving the sense of connection and communication between people. Specifically, by projecting or displaying an image using the image transmission system 100, the presence of a first person in a first area is communicated to a second person in a second area different from the first area, allowing the second person to feel a connection with the first person. Examples of areas that can be the first area and the second area include a hallway (corridor), a hallway, or a room inside or outside a building. The first area and the second area may be located in two different buildings, two different areas within the same building, or either or both may be outdoor areas.

[0021] 2-2. Form of use (basic form) FIG. 3 shows a first person 1 in a first area A1 and a second person 2 in a second area A2. In the example of FIG. 3, the first area A1 and the second area A2 are corridors located in different places within the same building 1000. A first camera 131 is installed in the first area A1, and is capable of capturing an image of the first person 1 in the first area A1. A second device 122 configured as a projection device is installed in the second area A2, and is capable of projecting an image onto a second object O2 installed in the second area A2. The second object O2 may be a structure such as a wall, pillar, or door of the second area A2, or may be a screen or the like installed specifically for projecting images.

[0022] In the following description, the second device 122 is a projection device, but in the first application example, the second device 122 may be a display device that displays an image on the second object O2. In this case, the second object O2 may be a display screen such as a display, or may be an object that constitutes a part of the second device 122.

[0023] 3, first camera 131 is installed within first area A1, but the installation location does not necessarily have to be limited to within first area A1 and may be outside first area A1 as long as it is located in a position where it can capture an image of first person 1 in first area A1. Similarly, second device 122 may be installed in a position where it can project or display an image on second object O2, and the installation location does not have to be within second area A2.

[0024] The image transmission system 100 executes image transmission processing while the first person 1 is present in the first area A1. The image transmission processing is processing for communicating the presence of the first person 1 to the second person 2 by means of an image. The image transmission processing is performed as follows: First, the first camera 131 captures an image of the first person 1. The first image obtained by capturing the image of the first person 1 is acquired by the processor 111. The processor 111 generates an image P1 representing the first person 1 in real time based on the first image. The generated image P1 is projected in real time onto the second object O2 by the second device 122.

[0025] By performing the image transmission process, the second person 2 can feel the presence of the first person 1 as if they were in the same area, even though they are in a different area from the first person 1. This gives the second person 2 a sense of security and solidarity.

[0026] However, one thing to be aware of here is the issue of the privacy of the first person 1. The first person 1 does not necessarily want the second person 2 to see an image revealing his or her detailed condition. To address the issue of the privacy of the first person 1, multiple image transmission modes are provided for the image transmission process. When the first person 1 first appears in the first area A1, the silhouette mode among the image transmission modes is initiated. The silhouette mode is a mode in which the image P1 is a first person silhouette image that represents the silhouette of the first person 1. In FIG. 3, the first person silhouette image is projected as the image P1.

[0027] By starting the image transmission mode in silhouette mode first, a detailed image of first person 1 is not suddenly revealed to second person 2, allowing first person 1 to enter and pass through first area A1 with peace of mind. On the other hand, for second person 2, regardless of whether image P1 is a silhouette image, the presence of first person 1 is felt and a sense of security and solidarity is gained. In fact, by making image P1 a silhouette image, a sense of depth is created, as if the shadow of someone is being reflected there, and the presence of image P1 is emphasized. In this way, a sense of security and solidarity can be more effectively imparted to second person 2 while respecting the privacy of first person 1.

[0028] As described above, silhouette mode is appropriate as the initial image transmission mode when the image transmission process starts. However, it is possible that the first person 1 wishes to disclose a more detailed image of himself / herself to the second person 2. For example, the first person 1 may want to convey not only the movements conveyed by the silhouette but also his / her clothing and facial expressions to the second person 2. From the perspective of improving communication and a sense of connection between the first person 1 and the second person 2, it is desirable for a more detailed image of the first person 1 to be disclosed to the second person 2 in situations where the first person 1 wishes to do so. Therefore, when it is determined that the first person 1 has expressed his / her intention, the image transmission system 100 changes the image transmission mode from silhouette mode to live mode. The live mode is a mode in which the image P1 is a first person detailed image that is closer to the first person 1's actual image than the first person silhouette image.

[0029] 4 shows an example of a first person detailed image in which an image of the first person 1 itself is projected onto a second object O2 as image P1. The first person detailed image may be an image of the first person 1 itself, or an image of an avatar representing the first person 1. By using image P1 as the first person detailed image, the second person 2 can learn details about the first person 1, including clothing, facial expressions, etc., which cannot be conveyed from the first person silhouette image.

[0030] The presence or absence of the first person 1's intention, which is required to change the image transmission mode to live mode, is determined by a first trigger action performed by the first person 1. When it is determined that the first person 1 has performed the first trigger action, the image transmission system 100 switches the image transmission mode from silhouette mode to live mode. The action that serves as the first trigger action is set in advance by a system administrator or the like. Any action may be set as the first trigger action, but it is desirable to set an action that is not normally performed when simply passing through the first area A1 so that it is clear that the first person 1 has performed the first trigger action based on their own will. For example, actions such as waving both hands widely, jumping three times on the spot, or touching both hands simultaneously twice to the wall may be set as the first trigger action.

[0031] The first trigger action is preferably taught in advance to the first person 1. Examples of methods for informing the first person 1 of the first trigger action include posting an explanation of the first trigger action on a wall or at the entrance of the first area A1, or distributing an explanation to people who are expected to pass through the first area A1 via an app on their smartphones, etc. In this way, by specifying the first trigger action for the first person 1 in advance, it becomes clearer that the first trigger action by the first person 1 was performed based on his or her own will.

[0032] As another example, the first trigger action may be an action that can be set by the first person 1 himself / herself. For example, a desired action may be registered in advance through a smartphone app or the like. By setting the first trigger action to an action designated by the first person 1 himself / herself, the possibility that the first person 1 will perform the first trigger action without his / her knowledge is reduced, and it is possible to make it clearer that the first trigger action by the first person 1 has been performed based on his / her own will.

[0033] 2-3. Flowchart In the first application example, the processing executed by the processor 111 includes an image transfer mode setting processing and an image transfer processing. The image transfer mode setting processing and the image transfer processing are executed by different programs. Fig. 5 is a flowchart showing an example of the image transfer mode setting processing. The image transfer mode setting processing is repeatedly executed at a predetermined control period.

[0034] In step S101, the processor 111 determines whether a new first person 1 has appeared in the first area A1. The appearance of a new first person 1 in the first area A1 may be determined based on the first image, or may be determined by a sensor such as a human sensor installed in the first area A1. If it is determined that a new first person 1 has appeared in the first area A1 (step S101; Yes), the processing proceeds to step S102. If a new first person 1 has not appeared in the first area A1 (step S101; No), the processing for this cycle ends.

[0035] In step S102, processor 111 turns on the image transfer process flag. The image transfer process flag is a flag for executing the image transfer process. At this time, the image transfer mode is set to the silhouette mode. When the silhouette mode starts, the process proceeds to step S103.

[0036] In step S103, the processor 111 determines whether or not a first trigger action has been performed by the first person 1. Whether or not the first trigger action has been performed by the first person 1 is determined based on the first image. If it is determined that the first trigger action has been performed by the first person 1 (step S103; Yes), the process proceeds to step S104. If it is determined that the first trigger action has not been performed by the first person 1 (step S103; No), the process proceeds to step S105. In step S103, the processor 111 may determine that the first trigger action has been performed only when it is determined that the first trigger action has been performed voluntarily by the first person 1, and may determine that the first trigger action has not been performed when it is determined that the first trigger action performed by the first person 1 is not a voluntarily action.

[0037] In step S104, the processor 111 starts the live mode. That is, the image transmission mode is changed from the silhouette mode to the live mode. When the live mode is started, the process proceeds to step S105. Note that if the current image transmission mode is already set to the live mode, the image transmission mode is not changed and the live mode continues.

[0038] In step S105, the processor 111 determines whether or not the first person 1 is present in the first area A1. Whether or not the first person 1 is present in the first area A1 may be determined based on the first image, or may be determined based on information acquired from a sensor installed in the first area A1. If the first person 1 continues to be present in the first area A1 (step S105; Yes), the process returns to step S103. If the first person 1 is no longer present in the first area A1 (step S105; No), the process proceeds to step S106.

[0039] In step S106, the processor 111 turns off the image transfer process flag. When the image transfer process flag is turned off, the image transfer process described below is no longer performed, and the projection of the image P1 onto the second object O2 ends. When the image transfer process flag is turned off, the processing of the current cycle ends.

[0040] 6 is a flowchart showing an example of image transmission processing executed by the processor 111. The image transmission processing is repeatedly executed at a predetermined control cycle while the image transmission processing flag is ON. The execution cycles of the image transmission mode setting processing and the image transmission processing may be the same or different.

[0041] In step S201, processor 111 acquires a first image. The first image is acquired by capturing an image of first person 1 present in first area A1 with first camera 131. Once the first image is acquired, the process proceeds to step S202.

[0042] In step S202, the processor 111 generates an image P1 based on the first image. When the image transmission mode is silhouette mode, the outline of the first person 1 is extracted from the first image, and a first person silhouette image is generated as image P1. When the image transmission mode is live mode, an image of the first person 1 itself is extracted from the first image and used as image P1, or an avatar image is generated from the image of the first person 1 included in the first image. Image P1 is generated in real time relative to the acquisition of the first image. Once image P1 is generated, the process proceeds to step S203.

[0043] In step S203, the processor 111 causes the second device 122 to project the image P1 onto the second object O2. The projection of the image P1 is performed in real time relative to the generation of the image P1. Once the image P1 is projected, the processing for this cycle ends.

[0044] Note that if there are multiple first persons 1 in the first area A1, the processes for each first person 1 are not linked. In other words, the image transmission mode setting process and the image transmission process are executed independently for each first person 1. Therefore, there may be cases where first person silhouette images and first person detailed images are mixed among the multiple images P1 projected onto the second object O2.

[0045] For example, suppose a first person 1-X enters the first area A1 and the image transmission process flag is turned ON, and then the image transmission mode is set to live mode by the first trigger action of the first person 1-X. If a second person 1-Y then enters the first area A1, the image transmission process flag for the first person 1-Y is again turned ON. The image transmission mode for the first person 1-Y is initially set to silhouette mode regardless of whether the first person 1-X performs the first trigger action, and is only set to live mode when the first person 1-Y performs the first trigger action. In this way, setting the image transmission mode independently for each first person 1 enables each first person 1 to control the amount of information disclosed to the second area A2 based on their own will, even when there are multiple first persons 1.

[0046] As described above, in the first application example, the image transmission system 100 displays an image P1 of a first person 1 in a first area A1 in a different second area A2, giving a sense of security and unity to the second person 2. Furthermore, the image of the first person 1 displayed first is always a silhouette image, which takes into consideration the privacy of the first person 1 and at the same time emphasizes the presence of the image P1, effectively creating a sense of security and unity for the second person 2. Furthermore, by displaying the image P1 as a detailed first person image when the first person 1 expresses his or her intention, communication between the first person 1 and the second person 2 can be made smoother.

[0047] 2-4. Application Form Two applications of the first application example will be described below. The first and second application examples can also be combined.

[0048] As a first application form, the image transmission system 100 may make the image P1 closer to the image of the first person 1 itself each time the first person 1 performs the first trigger action. Specifically, a plurality of stages of live mode are set, such as a first stage live mode, a second stage live mode, ..., and each time the first person 1 performs the first trigger action, the live mode transitions from the current stage to the next higher stage live mode. Each stage differs in how close the projected detailed image of the first person is to the first person 1. FIG. 7 is a flowchart showing an example of image transmission mode setting processing executed by the processor 111 in the first application example.

[0049] The processes from step S301 to step S303 are the same as the processes from step S101 to step S103 in Fig. 5. If it is determined in step S303 that the first trigger action has been performed by the first person 1 (step S303; Yes), the process proceeds to step S304. If it is determined that the first person 1 has not performed the trigger action (step S303; No), the process proceeds to step S305.

[0050] In step S305, it is determined whether the first person 1 is present in the first area A1. If the first person 1 continues to be present in the first area A1 (step S305; Yes), the process returns to step S303. If the first person 1 has left the first area A1 (step S305; No), the process proceeds to step S309.

[0051] In step S304, the image transmission mode is changed from the silhouette mode to the first stage live mode. When the first stage live mode starts, the process proceeds to step S306.

[0052] The processing from step S306 to step S308 is basically the same as the processing from step S303 to step S305. However, the live mode set in step S307 is determined by the current live mode. When the current live mode is the (N-1)th stage live mode, the Nth stage live mode starts in step S307. However, when the current live mode is the highest stage live mode, the image transmission mode continues to be set to the live mode of the same stage.

[0053] Thereafter, the processes of steps S306 to S308 are repeated until the first person 1 is no longer present in the first area A1. If it is determined in step S308 that the first person 1 is not present in the first area A1 (step S308; No), the process proceeds to step S309. In step S309, the image transmission process flag is turned OFF by the processor 111, similar to step S106 in FIG. 5. When the image transmission process flag is turned OFF, the process of this cycle ends.

[0054] The multiple stages of the live mode may be two stages, and in a first stage of the live mode, the first person detailed image may be an image of an avatar representing the first person 1, and in a second stage of the live mode, the first person detailed image may be an image of the first person 1 itself. Alternatively, the multiple stages of the live mode may be more than two stages. For example, in a first stage of the live mode, the first person detailed image may be an image of an avatar that largely abstracts the first person 1 without showing the finer details of the first person 1's features, and as the live mode stage increases, the first person detailed image may transition to an image that shows the first person 1 in more detail. Then, in the highest stage of the live mode, the first person detailed image may be an image of the first person 1 itself.

[0055] In the first application form, by providing multiple stages in the live mode, the first person 1 can control the amount of information disclosed by the image P1, resulting in a system that is more considerate of privacy. For example, this is effective in cases where the first person 1 wants to convey to the second person 2 facial expressions, such as whether the first person 1 is smiling or angry, which cannot be conveyed from the first person silhouette image, but does not want the second person 2 to know detailed characteristics that would clearly identify the first person 1. Even in such cases, the level of abstraction of the image P1 can be controlled by the number of times the first person 1 performs the trigger action, making it possible to disclose only the information that the second person 2 is willing to know.

[0056] Next, a second application example will be described. In the second application example, a reset operation is set to return image P1 to the first human silhouette image. FIG. 8 is a flowchart showing an example of image transmission mode setting processing in the second application example. The processing in steps S401 to S404 in the flowchart of FIG. 8 is the same as the processing in steps S101 to S104 in FIG. 5. Note that if the image transmission processing flag is already ON in step S402, the ON state is maintained. After the live mode is started in step S404, the processing proceeds to step S405.

[0057] In step S405, the processor 111 determines whether or not a reset operation has been performed by the first person 1. The reset operation is an operation for returning the image transmission mode, which has been set to live mode, to silhouette mode again. Whether or not a reset operation has been performed by the first person 1 is determined based on the first image. If it is determined that a reset operation has been performed by the first person 1 (step S405; Yes), the process returns to step S402. If it is determined that a reset operation has not been performed by the first person 1 (step S405; No), the process proceeds to step S406. In step S405, the processor 111 determines that a reset operation has been performed only if the reset operation has been performed voluntarily by the first person 1, and may determine that a reset operation has not been performed if it is determined that the reset operation was not a voluntarily performed by the first person 1.

[0058] The reset action is set in advance by a system administrator or the like. Any action may be set as the reset action as long as it is different from the first trigger action. However, it is desirable to set an action that is not normally performed when simply passing through the first area A1, so that it is clear that the first person 1 has performed the reset action of their own volition. Also, it is desirable to set an action that is significantly different from the first trigger action, so as to clearly distinguish it from the first trigger action. For example, the first trigger action may be set to be an action of vigorously waving both hands, and the reset action may be set to be an action of vigorously jumping on the spot.

[0059] The processes in steps S406 and S407 are the same as the processes in steps S105 and S106 in Fig. 5. When the image transmission process flag is turned OFF in step S407, the process of the current cycle ends.

[0060] In the second application form, by setting the reset operation, the first person 1 can restore the image P1 to the first person silhouette image at his / her own will when he / she accidentally performs the first trigger operation even though he / she does not want to disclose the first person detailed image to the second person 2. This makes it possible to make the system more considerate of the privacy of the first person 1, and to increase the sense of security for the first person 1.

[0061] 2-5. Two-way information transmission The above description has been given of image-based information transmission in one direction, from the first area A1 to the second area A2. Such image-based information transmission may also be bidirectional. That is, image-based information may be transmitted from the first area A1 to the second area A2, and at the same time, image-based information may be transmitted from the second area A2 to the first area A1.

[0062] 9 shows an example of the configuration of an image transmission system 100 for two-way information transmission. The image capturing device 130 includes two cameras: a first camera 131 and a second camera 132. The projection / display device 120 includes two devices: a first device 121 that projects or displays an image on a first object O1, and a second device 122 that projects or displays an image on a second object O2.

[0063] FIG. 10 shows an example of a scene in which two-way information transmission is taking place. A first person 1 is present in a first area A1, and a second person 2 is present in a second area A2. Similar to FIG. 4, the first person 1 is photographed by a first camera 131, and an image P1 representing the first person 1, created based on the photographed image, is projected onto a second object O2. In addition, in FIG. 10, a first device 121 is installed in the first area A1, and a second camera 132 is installed in the second area A2. A second person 2 is photographed by the second camera 132, and an image P2 representing the second person 2, created based on the photographed image, is projected onto a first object O1 installed in the first area A1.

[0064] When projection begins, image P1 and image P2 are set to a first person silhouette image and a second person silhouette image representing the silhouette of a second person, respectively. When a first trigger action is performed by first person 1, live mode is initiated for first person 1, and image P1 is set to the first person detailed image. When a second trigger action is performed by second person 2, live mode is initiated for second person 2, and image P2 is set to the second person detailed image that is closer to the image of the second person himself than the second person silhouette image. The second trigger action is set in advance by a system administrator, etc. The first trigger action and the second trigger action may be the same action or different actions. Since first person 1 and second person 2 can see each other's images, this leads to two-way communication.

[0065] Like the first object O1, the second object O2 may be a structure in the second area A2, such as a wall, pillar, or door, or may be a screen or the like installed specifically for projecting an image. The second device 122 has also been described as a projection device, but it may also be a display device that displays an image on the second object O2. In this case, the second object O2 may be a display screen such as a display, or may be part of the second device 122.

[0066] Furthermore, the installation location of the second camera 132 does not necessarily have to be limited to within the second area A2, and may be outside the second area A2 as long as it is in a position where it can capture an image of the second person 2 in the second area A2. Similarly, the installation location of the second device 122 does not have to be within the second area A2 as long as it is installed in a position where it can project or display an image on the second object O2.

[0067] When two-way information transmission is performed, the image transmission mode for the first person 1 and the image transmission mode for the second person 2 may be set independently or in conjunction with each other. When they are set independently, the processing related to the projection of image P2 can be realized by replacing the first area A1, the second area A2, the first person 1, the second person 2, the first camera 131, the second device 122, the second object O2, and the first trigger operation in the above description with the second area A2, the first area A1, the second person 2, the first person 1, the second camera 132, the first device 121, the first object O1, and the second trigger operation, respectively.

[0068] 2-6. Trigger operation coordination When two-way information transmission is performed, the image transmission mode for the first person 1 and the image transmission mode for the second person 2 may be set in conjunction with each other. In this case, the trigger action for transitioning from silhouette mode to live mode can be an action realized by cooperation between the first person 1 and the second person 2. The trigger action made possible by cooperation between the first person 1 and the second person 2 is assumed to be simultaneous or identical trigger actions. "Simultaneous trigger actions" means that a first trigger action by the first person 1 and a second trigger action by the second person 2 are performed simultaneously. In this case, too, the first trigger action and the second trigger action may be the same action or different actions. "Identical trigger actions" means that the first trigger action and the second trigger action are the same action, and both the first trigger action by the first person 1 and the second trigger action by the second person 2 are performed.

[0069] For example, simultaneous or identical trigger actions may be when a first person touches image P2 projected onto first object O1 and a second person 2 touches image P1 projected onto second object O2 at the same time. When linking trigger actions, not only can simultaneous actions be defined as trigger actions, but actions that require cooperation between first person 1 and second person 2 can also be defined as trigger actions, making it possible to create a stronger sense of connection or solidarity between first person 1 and second person 2.

[0070] FIG. 11 is a flowchart showing an example of an image transmission mode setting process when trigger operations are linked.

[0071] In step S501, processor 111 determines whether a first person 1 is present in first area A1 and a second person is present in second area A2, and whether first person 1 and second person 2 are present together. Whether first person 1 and second person 2 are present in each area may be determined based on images captured by first camera 131 and second camera 132, or may be determined by a motion sensor or the like installed in each area. If it is determined that first person 1 and second person 2 are present together (step S501; Yes), the process proceeds to step S502. If first person 1 and second person 2 are not present together (step S501; No), the process for this cycle ends.

[0072] In step S502, the processor 111 turns on the image transmission processing flag. At this time, the image transmission mode is set to the silhouette mode. When the silhouette mode starts, the processing proceeds to step S503. Note that in the processing shown in the flowchart in Fig. 11, the ON / OFF of the image transmission processing flag for the first person 1 and the ON / OFF of the image transmission processing flag for the second person 2, and the setting of the image transmission mode for the first person 1 and the setting of the image transmission mode for the second person 2 are performed in conjunction with each other.

[0073] In step S503, the processor 111 determines whether simultaneous or identical trigger actions have occurred. Whether simultaneous or identical trigger actions have occurred is determined based on the first image and the second image. Here, the second image is an image obtained by photographing the second person 2 with the second camera 132. If it is determined that simultaneous or identical trigger actions have occurred (step S503; Yes), the process proceeds to step S504. If it is determined that simultaneous or identical trigger actions have not occurred (step S503; No), the process proceeds to step S505. When determining whether simultaneous trigger actions have occurred in step S503, the processor 111 may determine that simultaneous trigger actions have occurred if the difference between the timing at which the first trigger action was performed by the first person 1 and the timing at which the second trigger action was performed by the second person 2 is within a certain time period.

[0074] In step S504, the processor 111 starts the live mode. If the current image transmission mode is the live mode, the current mode is continued. Once the live mode is started, the process proceeds to step S505.

[0075] In step S505, processor 111 determines whether first person 1 is present in first area A1 and second person 2 is present in second area A2. If both first person 1 and second person 2 are still present (step S505; Yes), the process returns to step S503. If either or both of them are no longer present (step S505; No), the process proceeds to step S506.

[0076] In step S506, the processor 111 turns off the image transfer process flag. When the image transfer process flag turns off, the projection of the image P1 onto the first object O1 and the projection of the image P2 onto the second object O2 are completed. When the image transfer process flag is turned off, the processing for the current cycle is completed.

[0077] 2-7.Other examples When the first area A1 and the second area A2 are far apart, a time difference may occur between the first area A1 and the second area A2. The image transmission system 100 may be a system that accommodates such a time difference that occurs between the two areas. In other words, the image transmission system 100 may not immediately project the image P1 acquired by photographing the first area A1 onto the second object O2, but may project it onto the second object O2 at a timing that takes the time difference into consideration.

[0078] Furthermore, the image transmission system 100 may project image P2 onto the second object O2 together with image P1, and project image P1 onto the first object O1 together with image P2. This allows for interaction between the images. For example, the first person 1 and the second person 2 can feel a stronger connection with each other by having fun by overlaying their images on each other.

[0079] Furthermore, the density of image P1 may be varied so that the density of image P1 increases as the distance between first person 1 and first object O1 decreases. Similarly, the density of image P2 may be varied so that the density of image P2 increases as the distance between second person 2 and second object O2 decreases. This further enhances the sense of realism of projected images P1 and P2.

[0080] 3. Second example of use In the first application example, the image transmission system 100 is used as a system for making users feel a connection with real people. In the second application example, the image transmission system 100 is used as a system for making users feel a connection with fictitious characters.

[0081] 12 shows an example of a scene in which the image transmission system 100 is applied in the second application example. In the second application example, the projection / display device 120 includes a projection device 123 that projects an image onto a projection surface S1 within a specific area A3. The projection surface S1 may be a structure within the specific area A3, such as a wall, floor, pillar, or door, or may be a screen installed for projecting an image. Furthermore, the projection surface S1 may be flat or curved.

[0082] The image transmission system 100 (processor 111) generates an image P3 of a silhouette representing a character C3 and projects it onto the projection surface S1 using the projection device 123. The projection of the image P3 is performed on the condition that a person 3 is present in the specific area A3. The person 3 may be an unspecified person, and may be a single person or multiple people. The presence of the person 3 in the specific area A3 may be determined, for example, by a sensor such as a human presence sensor. When the sensor detects a person, it is determined that the person 3 is present in the specific area A3. Furthermore, such a sensor may be a device included in the image transmission system 100. As another example, as shown in FIG. 13 , a third camera 133 may be installed in the specific area A3, and the presence of the person 3 may be determined based on information acquired by the third camera 133. In this case, the third camera 133 may be included in the imaging device 130.

[0083] The image transmission system 100 (processor 111) projects an image P3 representing a character C3 onto a projection surface S1 within the specific area A3, provided that at least a person 3 is present within the specific area A3. Here, the image P3 is a silhouette image. Making the image P3 a silhouette image is effective in emphasizing the presence of the character C3. Furthermore, making the image P3 a silhouette image makes it easier to project an image using an existing structure within the specific area A3 as the projection surface S1, rather than a dedicated screen. When an existing structure is used, it becomes more difficult for the person 3 to predict where the image P3 will appear, which enhances the sense of realism when they encounter the character C3.

[0084] In the second application example, the following several embodiments are assumed. The following embodiments may be implemented either individually or in combination.

[0085] In the first embodiment, the image transmission system 100 (processor 111) does not project two or more images P3 representing one character C3 into the specific area A3. By expressing in the image P3 that there is only one character C3, the presence of the character C3 can be emphasized. Furthermore, the rarity of the character C3 is enhanced.

[0086] In the second embodiment, the image transmission system 100 (processor 111) identifies the relationship between the person 3 and the character C3 and changes the behavior of the character C3 represented in the image P3 in accordance with the identified relationship. The image transmission system 100 can, for example, recognize the person 3 using the third camera 133 and identify the relationship between each person 3 and the character C3. The identified relationship may be determined based on predefined parameters, for example, corresponding to the number of times and duration a person has met a character, or may be determined by random factors. Relationship information indicating the relationship between the person 3 and the character C3 is stored in the storage device 112. The processor 111 changes the behavior of the character C3 represented in the image P3 in accordance with the relationship information. Changing the behavior of the character C3 in accordance with the relationship makes it seem as if the character has emotions, enhancing the sense of realism.

[0087] In the third embodiment, the image transmission system 100 (processor 111) sets the condition for starting the projection of image P3 as person 3 performing a trigger action. The trigger action is set in advance by a system administrator or the like. Whether person 3 has performed a trigger action can be determined, for example, based on an image obtained by photographing person 3 with third camera 133. Unlike the first and second use examples, the trigger action in this case is not taught to person 3 in advance. In other words, the projection of image P3 starts only when person 3 happens to perform the trigger action. By setting such a trigger action as a condition for the projection of image P3, the character's rarity is enhanced, providing a sense of surprise and excitement. [Explanation of symbols]

[0088] A1 Area 1 A2 Area 2 A3 Specific Area C3 character O1 1st object O2 2nd object P1 Image P2 image P3 Images S1 projection surface 1 1st person 2 2nd person 3 people 100 Image Transmission System 110 Image processing device 111 processors 112 Storage device 120 Display device 121 1st device 122 2nd device 123 Projection device 130 Imaging Device 131 Camera 1 132 Second Camera 133 Third Camera 1000 buildings

Claims

1. one or more processors; the one or more processors: a process of capturing an image of a first person present in a first area with a first camera to obtain a first image; generating a first person silhouette image representing a silhouette of the first person in the first image; a process of projecting the first person silhouette image onto a structure in a second area different from the first area, regardless of whether or not a specific action is performed by the first person; determining whether the first person has performed a first trigger action based on the first image; a process of projecting a first person detailed image, which is closer to an image of the first person than the first person silhouette image, onto the structure when the first person performs the first trigger action; configured to run Image transmission system.

2. 2. The image transmission system according to claim 1, The first trigger action is an action that is taught to the first person in advance or an action that is set in advance by the first person. Image transmission system.

3. 3. The image transmission system according to claim 1 or 2, The first person detailed image is an image of the first person itself or an image of an avatar of the first person. Image transmission system.

4. 3. The image transmission system according to claim 1 or 2, the one or more processors: A process of making the first person detailed image closer to the image of the first person each time the first person performs the first trigger action. configured to further perform Image transmission system.

5. one or more processors; the one or more processors: a process of capturing an image of a first person present in a first area with a first camera to obtain a first image; generating a first person silhouette image representing a silhouette of the first person in the first image; a process of projecting the first person silhouette image onto a second object, which is a structure in a second area different from the first area, regardless of whether or not a specific action is performed by the first person; a process of capturing an image of a second person present in the second area with a second camera to obtain a second image; generating a second person silhouette image representing a silhouette of the second person included in the second image; a process of projecting the second person silhouette image onto a first object, which is a structure in the first area, regardless of whether or not a specific action is performed by the second person; a process of determining whether the first person and the second person performed simultaneous or identical trigger actions based on the first image and the second image; a process of projecting a first person detail image that is closer to an image of the first person than the first person silhouette image onto the second object when the first person and the second person perform the simultaneous or identical trigger actions, and projecting a second person detail image that is closer to an image of the second person than the second person silhouette image onto the first object; configured to run Image transmission system.

6. 6. The image transmission system according to claim 5, The simultaneous or identical triggering action includes the first person touching the second person silhouette image projected onto the first object at the same time that the second person touches the first person silhouette image projected onto the second object. Image transmission system.

7. A process of acquiring a first image by a first camera present in a first area; generating a first person silhouette image representing a silhouette of a first person included in the first image; a process of projecting the first person silhouette image onto a structure in a second area different from the first area, regardless of whether or not a specific action is performed by the first person; determining whether the first person has performed a first trigger action based on the first image; a process of projecting a first person detailed image, which is closer to an image of the first person than the first person silhouette image, onto the structure when the first person performs the first trigger action; Contains Image transmission method.

Images