IMAGE PROCESSING APPARATUS, IMAGE DISPLAY SYSTEM AND IMAGE PROCESSING METHOD

By detecting and replacing the reflective areas of the machine equipment on the mirror or glass in the image processing device, the problem of affecting the sense of reality in the image is solved, and a higher sense of reality in the image and user immersion is achieved.

JP7674987B2Active Publication Date: 2025-05-12DENSO TEN LTD
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
JP2021169743
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-10-15
Publication Date
2025-05-12
Estimated Expiration
2041-10-15

AI Technical Summary

Technical Problem

In the prior art, when the machine is reflected on a mirror or glass, the real image of the user appears in the image, which may affect the sense of reality of the image.

Method used

By setting a control unit in the image processing device, detecting the reflective area of ​​the machine device in the image, and removing or replacing the machine device image in the area with the user's image, thereby enhancing the sense of reality of the image.

Benefits of technology

It effectively enhances the realistic sense of the image, reduces the user's presence of the machine and equipment in the image, and enhances the user's immersion.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007674987000001
    Figure 0007674987000001
  • Figure 0007674987000002
    Figure 0007674987000002
  • Figure 0007674987000003
    Figure 0007674987000003
Patent Text Reader

Abstract

To provide an image processing device, an image display system, and an image processing method capable of enhancing presence.SOLUTION: An image processing device includes a control unit. The control unit performs image processing of an image captured by an imaging device located on a site. The control unit detects a device region corresponding to the imaging device reflected on a reflector from among captured images to perform an erasing processing of an image of the imaging device in the device region.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an image processing device, an image display system, and an image processing method. [Background technology]

[0002] Conventionally, there is a technology that enables a user at a remote location to enjoy realistic images of a location by displaying real-time images captured by a robot that can be remotely moved and controlled by a user stationed at the location (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2002-176547 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, with conventional technology, for example, when a robot is reflected in an image on a reflective surface such as glass or a mirror, the user's image would actually be reflected (if the user were actually acting on-site), and this difference could lead to a loss of realism.

[0005] The present invention has been made in view of the above, and has an object to provide an image processing device, an image display system, and an image processing method that can enhance the sense of realism. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the object, the image processing device according to the present invention includes a control unit. The control unit performs image processing of an image captured by an imaging device installed on-site. The control unit detects a device area corresponding to the imaging device reflected by a reflecting object from the captured image, and erases the image of the imaging device in the device area. Effect of the Invention

[0007] According to the present invention, the sense of realism can be enhanced. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of the operation of the image display system according to the embodiment. [Diagram 2] FIG. 2 is a block diagram showing an example of the configuration of an image display system. [Diagram 3] FIG. 3 is a diagram illustrating an example of the on-site device information. [Figure 4] FIG. 4 is a diagram showing an example of image information. [Diagram 5] FIG. 5 is an explanatory diagram illustrating the processing of the display control unit. [Figure 6] FIG. 6 is a flowchart showing the procedure of image processing executed by the image processing device according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, embodiments of an image processing device, an image display system, and an image processing method disclosed in the present application will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to the embodiments described below.

[0010] First, an operation example of an image display system including an image processing device will be described with reference to Fig. 1. Fig. 1 is a diagram showing an operation example of an image display system according to an embodiment. An image processing method according to the embodiment is executed by the image processing device.

[0011] 1, an image display system S according to an embodiment includes a remote device 1 and a local device 100. The remote device 1 is configured as, for example, a satellite dome or a VR device, and includes a display unit 2 and an image processing device 4. The local device 100 is configured as, for example, a mobile robot, and includes an imaging device 110.

[0012] In the image display system S according to the embodiment, first, a user at a remote location establishes a communication connection with the local device 100 via the remote device 1. When there are multiple local devices 100, the user may specify a specific local device 100 to establish a communication connection. After the communication is established, the user becomes able to remotely operate the local device 100 via the operation unit 3 (see FIG. 2) of the remote device 1.

[0013] Furthermore, the local device 100, which is connected to the remote device 1 for communication, transmits an image captured by the imaging device 110 to the remote device 1 and displays it on the display unit 2 of the remote device 1. This allows a user at a remote location to feel as if he or she is actually at the local location by viewing the image of the local location displayed on the display unit 2. The imaging device 110 is configured to be able to capture an image of the entire surroundings (360°) of the local device 100, as will be described in detail later.

[0014] 1, an example is taken in which the local location is an aquarium. When the local device 100 captures an image at an aquarium, the local device 100 may be reflected on the glass of the aquarium (an example of a reflecting object) and may appear in the image. In such a case, the image in which the local device 100 is reflected is displayed on the display unit 2 of the remote device 1, which may impair the user's sense of presence.

[0015] Therefore, in the image processing method according to the embodiment, the on-site device 100 reflected in the image is replaced with a user image corresponding to a user at a remote location, thereby enhancing the sense of realism.

[0016] Specifically, the image processing device 4 according to the embodiment first detects a device region R, which is the imaging device 110 (or the local device 100) reflected by a reflecting object, from an image captured by the imaging device 110 located on-site and displayed on a display unit 2 located in a remote location where a user is present (step S1). The reflecting object is an object that reflects light, such as glass or a mirror. A detailed method for detecting the device region R will be described later.

[0017] Next, in the image processing method according to the embodiment, the detected device area R is replaced with a user image UP corresponding to the user (step S2). The user image UP may be an actual image of the user, or may be a pseudo image such as a polygon image. That is, in the image processing method according to the embodiment, the image of the imaging device 110 (or the on-site device 100) in the device area R is erased. Note that the erasing process may include, in addition to replacing with the user image UP, erasing the image of the imaging device 110 (or the on-site device 100) in the device area R or replacing with an image other than the user image UP.

[0018] In this way, by erasing the image of the imaging device 110 (or the local device 100) in the device area R, the presence of the imaging device 110 (local device 100) in the image displayed on the display unit 2 is reduced, thereby reducing the sense of incongruity in the image and enhancing the sense of realism.

[0019] Furthermore, by replacing the device area R with the user image UP, the user can be given the illusion that he or she is being reflected by a reflecting object, thereby enhancing the sense of realism felt by the user.

[0020] It should be noted that the image processing device 4 according to the embodiment can change the orientation of the user image UP in accordance with the orientation of the imaging device 110, and this point will be described in detail later.

[0021] Furthermore, when a user performs an operation to change the orientation of the imaging device 110, the image processing device 4 according to the embodiment can synthesize an image of the user's line of sight and display it on the display unit 2 until the orientation of the imaging device 110 matches the user's line of sight (changed orientation); details of this point will be described later.

[0022] Next, a configuration example of the image display system S will be described with reference to Fig. 2. Fig. 2 is a block diagram showing a configuration example of the image display system S. As shown in Fig. 2, the image display system S has a remote device 1 and a local device 100.

[0023] The remote device 1 includes a display unit 2, an operation unit 3, and an image processing device 4. Although Fig. 2 shows an example of a configuration in which the remote device 1 includes the image processing device 4, the local device 100 may include the image processing device 4, or a server device including the image processing device 4 may be disposed separately from the remote device 1 and the local device 100.

[0024] The display unit 2 is, for example, a screen onto which an image generated by the image processing device 4 is projected by a projector, a liquid crystal display, VR goggles, or the like.

[0025] The operation unit 3 is an operation member for operating the on-site device 100, and is, for example, a push button, a mouse, a joystick, or the like.

[0026] The image processing device 4 includes a communication unit 41, a control unit 42, and a storage unit 43. The control unit 42 includes an acquisition unit 421, a detection unit 422, a replacement unit 423, and a display control unit 424. The storage unit 43 stores local device information 431 and image information 432. The communication unit 41 is a communication interface, and transmits and receives various information to and from the local device 100.

[0027] Here, the image processing device 4 includes, for example, a computer having a central processing unit (CPU), a read only memory (ROM), a random access memory (RAM), a flash memory, an input / output port, and various other circuits.

[0028] The CPU of the computer functions as an acquisition unit 421, a detection unit 422, a replacement unit 423, and a display control unit 424 of the control unit 42, for example, by reading and executing a program stored in the ROM.

[0029] In addition, at least one or all of the acquisition unit 421, detection unit 422, replacement unit 423 and display control unit 424 of the control unit 42 can be configured with hardware such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0030] The storage unit 43 corresponds to a RAM or a flash memory. The RAM or the flash memory can store local device information 431, image information 432, information on various programs, etc. The image processing device 4 may acquire the above-mentioned programs and various information via other computers or portable recording media connected via a wired or wireless network.

[0031] The local device information 431 is information related to the local device 100. Fig. 3 is a diagram showing an example of the local device information 431. As shown in Fig. 3, the local device information 431 includes items such as "device ID", "current location", "movable area", and "image information".

[0032] The "device ID" is identification information for identifying the on-site device 100. The "current location" is information indicating the current location of the on-site device 100 on-site. The "movable area" is information indicating an area in the on-site where the on-site device 100 can move. The "image information" is information including an image of the on-site device 100. Specifically, the "image information" includes images of the exterior of the on-site device 100 captured from various angles.

[0033] Image information 432 is information related to a user image UP. Fig. 4 is a diagram showing an example of image information 432. As shown in Fig. 4, image information 432 is a diagram showing "user ID", "user image", "user information", and "permission flag".

[0034] "User ID" is identification information for identifying a user. "User image" is information indicating a user image UP. The user image UP is composed of multiple images of the user captured from different angles. The user image UP may also be a three-dimensional image obtained by combining the multiple images. The user image UP may also be an image obtained by converting the three-dimensional image into a polygon, or an avatar image. The user image UP may also be a template image selected by the user from multiple preset template images.

[0035] The "user information" includes information about the user. The "user information" includes, for example, attribute information of the user. The "permission flag" is information indicating whether or not to permit the use of the user's actual image as the user image UP when replacing the device area R with the user image UP. When the "permission flag" is "1", the use of the user's actual image as the user image UP is permitted, and when the "permission flag" is "0", the use of the user's actual image as the user image UP is not permitted. Note that when the "permission flag" is "0", an avatar image is used as the user image UP, or a background image is used instead of the user image UP.

[0036] Note that the image information 432 shown in FIG. 4 is an example, and may include, for example, list information of other users who are permitted to use the user's actual image as the user image UP (for example, list information of user IDs).

[0037] Next, each function of the control unit 42 (the acquisition unit 421, the detection unit 422, the replacement unit 423, and the display control unit 424) will be described in detail.

[0038] The acquisition unit 421 acquires various information. For example, the acquisition unit 421 acquires an image captured by the imaging device 110 of the on-site device 100 via the communication unit 41. The acquisition unit 421 also acquires a sound collected by the microphone 120 of the on-site device 100 via the communication unit 41.

[0039] Furthermore, the acquisition unit 421 acquires the position information of the local device 100 via the communication unit 41. The acquisition unit 421 stores the acquired position information as local device information 431 stored in the storage unit 43. Note that the acquisition unit 421 periodically acquires the position information of the local device 100 and updates the local device information 431.

[0040] The acquisition unit 421 also acquires information on the imaging direction of the imaging device 110 that captured the image. The information on the imaging direction includes information on the time stamp added by the on-site device 100.

[0041] The detection unit 422 detects a device region R, which is the on-site device 100 (imaging device 110) reflected by a reflecting object, from the image acquired by the acquisition unit 421. For example, the detection unit 422 detects the device region R by pattern matching using the acquired image and the image of the on-site device 100 (on-site device information 431).

[0042] The device region R is not limited to including the entire on-site device 100, and may include only a part of the on-site device 100.

[0043] Furthermore, the detection unit 422 detects the orientation of the on-site device 100 (the orientation of the imaging device 110) in the detected device region R, for example, from the result of pattern matching. Furthermore, the detection unit 422 detects the transparency of the on-site device 100 (imaging device 110) in the device region R. The transparency can be detected based on, for example, the luminance of the device region R, the overlap (luminance difference) between the device region R and another object, etc.

[0044] Furthermore, the detection unit 422 detects other on-site devices 100 (other imaging devices 110) as the device region R. The other on-site devices 100 can be detected, for example, by pattern matching with images of other on-site devices 100 registered in advance.

[0045] The replacement unit 423 replaces the device area R detected by the detection unit 422 with a user image UP corresponding to a user who exists in a remote location. Specifically, the replacement unit 423 replaces the device area R with the user image UP included in the image information 432. The replacement with the user image UP can be realized, for example, by cutting out the device area R and embedding the user image UP therein, or by superimposing the user image UP on the device area R.

[0046] Furthermore, the replacement unit 423 replaces the user image UP with a user image UP oriented in accordance with the orientation of the on-site device 100 detected by the detection unit 422. This allows the user image UP oriented in the direction of the on-site device 100 to be displayed, thereby reducing the sense of discomfort felt by the user and enhancing the sense of immersion in the on-site image. Furthermore, when a device region R including some of the on-site devices 100 is detected, the replacement unit 423 replaces the user image UP with some of the user images UP corresponding to some of the on-site devices 100.

[0047] Further, the replacement unit 423 replaces the user image UP with a user image UP that has been subjected to image processing according to the reflection characteristics of the device region R detected by the detection unit 422. That is, the replacement unit 423 replaces the user image UP with a user image UP that has been subjected to image processing according to the reflection characteristics of the device region R. An example of performing image processing according to the reflection characteristics is, for example, a case where image processing is performed on the device region R reflected by a spherical reflector. Specifically, the replacement unit 423 replaces the user image UP by deforming it to match the device region R that has been deformed by reflection on a spherical reflector. In this way, by replacing the user image UP with a user image UP that has been subjected to image processing according to the reflection characteristics of the device region R, the reality of the reflection by the reflector can be enhanced.

[0048] Furthermore, the replacement unit 423 may change the replacement process according to the size (area) of the device region R. For example, the replacement unit 423 may replace the device region R with the user image UP when the size of the device region R is equal to or larger than a threshold, and may erase the device region R when the size is less than the threshold. The process of erasing the device region R is, for example, a process of replacing with a background image. In other words, the process of erasing the device region R is an example of an obscuration process, and the obscuration process may be a blurring process of the device region R, a replacement with an image obtained by performing a blurring process on the user image UP, a replacement with a replacement image (stored in advance) used when the size of the device region R is less than a threshold, or the like.

[0049] Methods of replacing it with a background image include a process of replacing it with the color and pattern around the device area R, or a process of cutting out and pasting an area corresponding to the device area R (detected by image recognition processing) from an image taken a little earlier (which is stored briefly in the memory unit 43).

[0050] In this way, when the size of the device area R is small and the presence of the local device 100 is weak, it is possible to reduce the possibility that the user feels uncomfortable due to replacement with the user image UP by deleting the device area R. Also, it is possible to reduce the sense of discomfort caused by erasing the device area R by replacing it with a background image.

[0051] Furthermore, when the detection unit 422 detects a device region R that is another local device 100, the replacement unit 423 replaces it with a user image UP of another user viewing the display unit 2 of another remote device 1 that is communicatively connected to the other local device 100. This makes it possible to display an image as if the other user were on-site, thereby enhancing the user's sense of immersion.

[0052] Regarding uploading user images of other local devices 100, in the case of a person outside the group from the viewpoint of personal information (determined, for example, by prior group registration or personal image display permission registration), it is desirable to use an avatar image (by stratifying the pre-registered user information and applying an illustrated image corresponding to the stratification) rather than an actual image of the user.

[0053] Furthermore, when the replacement unit 423 replaces the device region R with the user image UP, the replacement unit 423 performs processing to blur the contour of the user image UP. This allows the user image UP to blend into the image, thereby reducing the sense of incongruity of the user image UP.

[0054] The display control unit 424 controls displaying an image captured by the imaging device 110 of the on-site device 100 on the display unit 2. Specifically, the display control unit 424 displays on the display unit 2 an image in which the device region R has been replaced by a user image UP or a background image by the replacement unit 423.

[0055] Here, the processing of the display control unit 424 when the imaging direction of the imaging device 110 is changed by a user's operation on the operation unit 3 will be described with reference to Fig. 5. Fig. 5 is an explanatory diagram for explaining the processing of the display control unit 424. Fig. 5 shows an example in which a user operation (rotation operation) is performed to rotate the imaging direction of the imaging device 110 in the horizontal direction.

[0056] 5, before a rotation operation is performed, the imaging direction of the imaging device 110 and the user's line of sight (the direction of the image displayed on the display unit 2) are aligned. When a rotation operation is performed in this state, a certain amount of time is required for the imaging direction of the imaging device 110 to face in a direction that matches the changed user's line of sight (mechanical movement time), and therefore a delay occurs before the image displayed on the display unit 2 matches the changed user's line of sight.

[0057] Therefore, when the imaging direction of the imaging device 110 is changed, the display control unit 424 first generates an image of the user's line of sight after the change and displays it on the display unit 2. Specifically, the display control unit 424 generates an image corresponding to the user's line of sight after the change from an all-around image (or a wide-angle image (wider than the angle corresponding to the display range of the display unit 2)) captured by the imaging device 110, and displays it on the display unit 2. In other words, the user has the illusion that he or she is viewing an image of the user's line of sight after the change in real time.

[0058] Then, the display control unit 424 continues to display the generated image on the display unit 2 until the imaging direction of the imaging device 110 coincides with the user's line of sight. Note that the synchronization between the image captured by the imaging device 110 and displayed on the display unit 2 and the imaging direction of the imaging device 110 is performed by comparing the timestamp of the image displayed on the display unit 2 with the timestamp included in the information on the imaging direction. In other words, the display control unit 424 synchronizes the image and the imaging direction by comparing the elapsed time from the start of changing the imaging direction of the imaging device 110.

[0059] Then, the display control unit 424 displays the image captured by the imaging device 110 on the display unit 2 after the imaging direction of the imaging device 110 and the user's line of sight are aligned.

[0060] This can reduce the sense of discomfort felt when an image is captured until the imaging direction of the imaging device 110 coincides with the user's line of sight.

[0061] Furthermore, the display control unit 424 displays the image on the display unit 2 and outputs the sound collected by the microphone 120 of the local device 100 from a speaker (not shown). This allows the user to enjoy the local video and audio from a remote location.

[0062] Next, a description will be given of an example configuration of the on-site device 100. As shown in FIG.

[0063] The imaging device 110 is a camera that captures images of the site. The imaging device 110 can change the imaging direction, for example, by a user operation using the operation unit 3 of the remote device 1. For example, the imaging device 110 can change the imaging direction or the imaging height by rotating in the horizontal or vertical direction.

[0064] The image captured by the imaging device 110 is transmitted to the remote device 1 via the control device 140.

[0065] The imaging device 110 is configured to be capable of capturing images of the entire surroundings (360°) of the on-site device 100. For example, one imaging device 110 captures an image by scanning the entire surroundings. Alternatively, multiple imaging devices 110 may capture images of the entire surroundings.

[0066] It should be noted that the imaging device 110 does not necessarily have to cover the entire periphery (360°) of the on-site device 100, and in that case, a device capable of wide-angle imaging such as 180° is preferable.

[0067] The microphone 120 collects sounds around the local device 100. The sounds collected by the microphone 120 are transmitted to the remote device 1 via the control device 140.

[0068] The moving unit 130 is a moving mechanism that moves the on-site device 100. The moving unit 130 has, for example, wheels or a walking mechanism. The moving unit 130 moves the on-site device 100 by being operated by a user using the operation unit 3 of the remote device 1.

[0069] The control device 140 includes a communication unit 141, a control unit 142, and a storage unit 143. The communication unit 141 is a communication interface, and transmits and receives various information to and from the remote device 1.

[0070] Here, the control device 140 includes, for example, a computer having a central processing unit (CPU), a read only memory (ROM), a random access memory (RAM), a flash memory, an input / output port, and various other circuits.

[0071] The CPU of the computer functions as the control unit 142, for example, by reading and executing a program stored in the ROM.

[0072] Furthermore, at least one or all of the functions of the control unit 142 can be configured using hardware such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0073] The storage unit 143 corresponds to a RAM or a flash memory. The RAM or the flash memory can store information on various programs, etc. The control device 140 may acquire the above-mentioned programs and various information via other computers or portable recording media connected via a wired or wireless network.

[0074] Next, a description will be given of the function of the control unit 142. The control unit 142 controls the imaging device 110, the microphone 120, and the moving unit 130. For example, the control device 140 acquires information on a user operation on the operation unit 3 of the remote device 1 via the communication unit 141, and controls the imaging device 110, the microphone 120, and the moving unit 130 in response to the user operation.

[0075] The control unit 142 also transmits information on the image captured by the imaging device 110 and the sound collected by the microphone 120 to the remote device 1 via the communication unit 141. The control unit 142 also transmits position information of the local device 100 measured by a positioning device (not shown) to the remote device 1 via the communication unit 141.

[0076] Next, a processing procedure of processing executed by the image processing device 4 according to the embodiment will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the processing procedure of image processing executed by the image processing device 4 according to the embodiment.

[0077] 6, first, the control unit 42 acquires an image captured by the imaging device 110 of the on-site device 100 (step S101). Next, the control unit 42 determines whether or not a device region R, which is an image of the on-site device 100 reflected by a reflecting object, has been detected from the acquired image (step S102).

[0078] When the control unit 42 detects the device region R (step S102: Yes), the control unit 42 determines whether the size of the device region R is equal to or larger than a threshold value (step S103).

[0079] If the size of the device area R is greater than or equal to the threshold (step S103: Yes), the control unit 42 acquires orientation information of the imaging device 110 in the image of the device area R (step S104) and generates a user image UP oriented according to the orientation of the imaging device 110 (step S105).

[0080] The orientation of the imaging device 110 can be determined by image matching between a reference image for comparison obtained in advance by photographing the imaging device 110 from each direction and an image of the device region R in the photographed image.

[0081] Next, the control unit 42 replaces the device area R with the user image UP (step S106). Next, the control unit 42 displays the image replaced with the user image UP on the display unit 2 (step S107), and ends the process.

[0082] On the other hand, in step S102, if the control unit 42 does not detect the device region R (step S102: No), it displays the image that has not been replaced with the user image UP on the display unit 2 (step S107), and ends the process.

[0083] Furthermore, in step S103, if the size of the device region R is less than the threshold value (step S103: No), the control unit 42 replaces the device region R with a background image (step S108), and proceeds to step S107.

[0084] In addition to replacing the device area R with a background image, it is also possible to apply obscuration processing such as blurring the device area R, replacing the user image UP with an image that has been subjected to blurring processing, and replacing with a replacement image (prestored) used when the size of the device area R is less than a threshold value. In other words, since the device area R is small and the impact on the user of the accuracy of the image is small, the device area R is subjected to obscuration processing that makes the image unclear, and processing that makes the reflected image of the imaging device 110 less noticeable can be performed to obtain the required effect with a light processing load.

[0085] As described above, the image processing device 4 according to the embodiment includes a detection unit 422 and a replacement unit 423. The detection unit 422 detects a device region R corresponding to the imaging device 110 reflected by a reflecting object from an image captured by the imaging device 110 located locally and displayed on a display unit 2 located in a remote location where the user is present. The replacement unit 423 replaces the detected device region R with a user image UP corresponding to the user. This can enhance the user's sense of presence.

[0086] Further advantages and modifications may readily occur to those skilled in the art. Thus, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and equivalents thereof. [Explanation of symbols]

[0087] 1 Remote equipment 2 Display section 3 Control section 4. Image Processing Device 41, 141 Communications Department 42, 142 Control section 43, 143 Storage section 100 On-site equipment 110 Imaging device 120 Mike 130 Mobile Unit 140 Control device 421 Acquisition Department 422 Detection unit 423 Substitution part 424 Display control section 431 Local equipment information 432 Image information R device area S Image Display System UP User Image

Claims

1. A control unit is provided for performing image processing of an image captured by an imaging device disposed on-site, The control unit is Detecting a device area corresponding to the imaging device reflected by a reflecting object from the captured image, and erasing the image of the imaging device in the device area. An image processing device comprising:

2. The control unit is As an erasure process, the device area is replaced with a user image corresponding to the user.

2. The image processing device according to claim 1,

3. The control unit is The erasure process involves replacing the device area with a background image.

3. The image processing device according to claim 1, wherein:

4. The control unit is replacing the image of the device area with the user image oriented in accordance with the orientation of the imaging device; 3. The image processing device according to claim 2,

5. The control unit is If the size of the device region is less than a predetermined threshold, an obscuration process is performed on the device region.

3. The image processing device according to claim 1, wherein:

6. The control unit is The obfuscation process includes erasing an image of the imaging device in the image of the device region.

6. The image processing device according to claim 5,

7. The control unit is replacing the user image with an image processed according to the reflection characteristics of the device area; 3. The image processing device according to claim 2,

8. The control unit is A device area corresponding to an imaging device associated with another user reflected on a reflecting object is detected as the other user's device area, and an image of the imaging device in the detected other user's device area is erased.

8. The image processing device according to claim 1, wherein:

9. An image processing device according to any one of claims 1 to 8, an imaging device that transmits an image captured by the imaging device to the image processing device; a display unit that displays an image generated by the image processing device; An image display system comprising:

10. An image processing method comprising the steps of: detecting a device area corresponding to the imaging device reflected by a reflective object from an image captured by an imaging device located on-site and displayed on a display unit located at a remote location where a user associated with the imaging device is present; and erasing the image of the imaging device in the device area.

Citation Information

Patent Citations

  • Composite image system

    JP2002176547A

  • Image output device, image output method, image output processing program, image distribution server, and image distribution processing program

    JP2004302939A

  • Imaging apparatus

    JP2011023814A

  • Image processing program and image processing apparatus

    JP2011249926A

  • Image processing system, imaging system, image processing apparatus, imaging apparatus and program

    JP2020204874A