COMMUNICATION SUPPORT SYSTEM, DISPLAY DEVICE ON REMOTE PARTY SIDE, AND COMMUNICATION SUPPORT PROGRAM

Abstract

To enable the upbringing of children in a system that supports communication between children located in different spaces and remote persons.SOLUTION: A communication support system 1 includes a child-side camera 22 that outputs child video data MC that captures a child, a remote camera 32 that outputs remote person video data MR that captures a remote person located outside a space where the child exists, a video control unit 41 that outputs superimposed video data MS in which an image specified by the remote person is superimposed on a position specified by the remote person in frame data of the remote person video data MR output by the camera 32 on the remote person side, a children's display device 21 that displays the superimposed video data MS, and a remote display device 31 that displays the child video data MC.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a communication support system, a remote user's display device, and a communication support program. [Background technology]

[0002] There is a system that supports communication between a child and a remote person who is located in a different space (see Patent Document 1). Patent Document 1 discloses a system in which a remote person watches over a child using a robot placed in the space where the child is located and a remote person terminal used by the remote person who is located in a space different from the child.

[0003] Also, a type of childcare teaching material called "Bepsart" has become popular. Bepsart is a type of two-dimensional doll that is created by drawing a character on each of two pieces of paper, pasting the two pieces of paper together with a stick, and cutting out the periphery of the character. The doll's movements are expressed by moving it in front of a background or by turning it over. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-048158 A Summary of the Invention [Problem to be solved by the invention]

[0005] The system described in Patent Document 1 focuses on monitoring, and does not specifically disclose any stimulation to the child's audiovisual senses, or education such as the development of language ability or the cultivation of imagination.

[0006] Therefore, an object of the present invention is to provide a technique that enables the upbringing of children in a system that supports communication between children and remote persons located in different spaces. [Means for solving the problem]

[0007] A communication support system according to one embodiment of the present invention includes a child's side camera that outputs child's video data filmed of a child, a remote person's side camera that outputs remote person's video data filmed of a remote person positioned outside the space in which the child is present, a video control unit that outputs superimposed video data in which an image specified by the remote person is superimposed at a position specified by the remote person on frame data of the remote person's video data output by the remote person's side camera, a child's side display device that displays the superimposed video data, and a remote person's side display device that displays the child's video data.

[0008] One aspect of the present invention relates to a remote person's display device used in a communication support system including a child side camera that outputs child video data of a child, a remote person's camera that outputs remote person's video data of a remote person located outside the space where the child is present, a child side display device that displays the remote person's video data, and a remote person's display device that displays the child video data. The remote person's display device displays a child video display section that displays the child video data and a remote person's video display section that displays left-right inverted remote person's video data, arranged vertically.

[0009] One aspect of the present invention is a processing program that causes a computer to function as a processing device connected to a child's side camera that outputs child video data that captures a child, a remote person's side camera that outputs remote person's video data that captures a remote person located in a space other than the child's presence, a child's side display device that displays the superimposed video data, and the remote person's side display device that displays the child video data, the processing program causing a computer to function as a video control unit that outputs superimposed video data in which an image specified by the remote person is superimposed at a position specified by the remote person in frame data of the remote person's video data output by the remote person's side camera. Effect of the Invention

[0010] According to the present invention, it is possible to provide a technique that enables the upbringing of children in a system that supports communication between children and remote parties located in different spaces. [Brief description of the drawings]

[0011] [Figure 1] 1 is a diagram illustrating a system configuration of a communication support system according to an embodiment of the present invention. [Diagram 2] 11A and 11B are diagrams illustrating examples of display screens of a child terminal and a remote terminal. [Diagram 3] FIG. 2 is a diagram illustrating functional blocks of the processing device. [Figure 4] 13 is a diagram for explaining a method in which a remote person specifies a position at which an image is to be superimposed on the remote person video display unit. FIG. [Diagram 5] 13A and 13B are diagrams illustrating a method by which a remote person specifies a position at which an image is to be superimposed on a remote person video display unit that displays superimposed video data in a horizontally inverted manner. [Figure 6] 13 is a sequence diagram illustrating the processing of the communication support system when a remote person specifies a position to superimpose an image on a remote person video display unit that displays superimposed video data in a horizontally inverted manner. FIG. [Figure 7] 11 is a sequence diagram illustrating the process of the communication support system when a remote person holds a printed matter of a code corresponding to a desired image and specifies a position to superimpose the image. FIG. [Figure 8] FIG. 13 is a diagram illustrating extended remote person video data. [Figure 9] FIG. 11 is a sequence diagram illustrating a process for generating extended remote person video data. [Figure 10] FIG. 13 is a diagram for explaining a process for adjusting the display positions of a child video display section and a remote person video display section (part 1). [Figure 11] FIG. 13 is a diagram for explaining the process of adjusting the display positions of the child video display section and the remote person video display section (part 2). [Figure 12] FIG. 13 is a diagram for explaining the process of adjusting the display positions of the child video display section and the remote person video display section (part 3). [Figure 13] FIG. 13 is a diagram for explaining the process of adjusting the display positions of the child video display section and the remote person video display section (part 4). [Figure 14]FIG. 2 is a diagram illustrating a hardware configuration of a computer used in the processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Next, an embodiment of the present invention will be described with reference to the drawings. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.

[0013] (Communication Support System) A communication support system 1 according to an embodiment of the present invention will be described with reference to Fig. 1. The communication support system 1 includes a child terminal 2, a remote person terminal 3, and a processing device 4. The processing device 4 is communicatively connected to each of the child terminal 2 and the remote person terminal 3. The remote person is a person located outside the space where the child is present.

[0014] A communication support system 1 according to an embodiment of the present invention uses a child terminal 2, a remote person terminal 3, and a processing device 4 to connect a space where a child is and a space where a remote person is with video data in real time, and supports remote communication between the child and the remote person. In the embodiment of the present invention, the processing device 4 particularly supports communication between the remote person and the child using BEPSART. The communication support system 1 superimposes image data that serves as a substitute for BEPSART on video data of a remote person and displays it on the child terminal 2, thereby stimulating the child's visual and auditory senses, and realizing education such as the development of language ability and the cultivation of imagination.

[0015] The child terminal 2 is a computer placed in a space where a child is present. As shown in FIG. 2(a), it includes a child side display device 21 and a child side camera 22. The child side display device 21 includes a child side remote person video display unit 211. The child side remote person video display unit 211 displays remote person video data MR captured by the remote person camera 32 of the remote person terminal 3, or superimposed video data MS generated from the remote person video data MR. The superimposed video data MS is video data in which image data substituting for a VAPSART is superimposed on the remote person video data MR.

[0016] The child's camera 22 outputs child video data MC capturing an image of the child. The child's camera 22 is controlled so that the tip of the child's hand is within the capturing area. For example, when the tip of the child's hand moves out of the capturing area, the child's camera 22 changes the viewpoint so that the tip of the child's hand is within the capturing area. The child's display device 21 may further have an area for displaying the video captured by the child's camera 22.

[0017] In the example shown in FIG. 1, the housing of the child terminal 2 is a robot type having a face with a child side display device 21 and a body supporting the face, but may be different depending on the child's growth stage. For example, when used by a toddler (0 to 3 years old) child, the child terminal 2 is preferably a robot type as shown in FIG. 1 in order to realize sufficient non-verbal communication. When used by a preschooler (3 to 6 years old), it is difficult for the child to stay still and continue receiving education, so the child side display device 21 is preferably a television. When used by a school-age child, the child terminal 2 is preferably a personal computer or a tablet. A terminal that can be used by children, such as a robot, may be used when they start school. The housings listed here are only examples, and any housing may be used as long as it is equipped with the child side display device 21 and the child side camera 22 and is capable of communication with the remote terminal 3 using video.

[0018] The remote person terminal 3 is a computer arranged in a space where the remote person is present. As shown in FIG. 2(b), the remote person terminal 3 includes a remote person's display device 31, a remote person's camera 32, a keyboard, a mouse, and the like. The remote person's display device 31 includes a child video display unit 311 and a remote person's video display unit 312. The child video display unit 311 displays child video data MC. The remote person's video display unit 312 displays remote person's video data MR, superimposed video data MS generated from the remote person's video data MR, and the like. The remote person's video display unit 312 may also display remote person's video data MRa generated by left-right inverting the remote person's video data MR, or superimposed video data MSa generated by left-right inverting the superimposed video data MS. The remote person's camera 32 outputs remote person's video data MR obtained by photographing the remote person.

[0019] The processing device 4 controls the process of transmitting child video data MC captured by the child terminal 2 to the remote terminal 3, and transmitting remote person video data MR captured by the remote terminal 3 to the child terminal 2 and the remote terminal 3. When the processing device 4 is instructed by the remote terminal 3 to display image data, the processing device 4 generates superimposed video data MS by superimposing image data on the remote person video data MR, and transmits the superimposed video data MS to the child terminal 2 and the remote terminal 3. The processing device 4 may also transmit remote person video data MRs obtained by left-right inverting the remote person video data MR, or superimposed video data MSa obtained by left-right inverting the superimposed video data MS, to the remote terminal 3. The processing device 4 may obtain the voice of the remote person from the remote terminal 3 and output the voice of the remote person to the child terminal 2, and obtain the voice of the child from the child terminal 2 and output the voice of the child to the remote terminal 3.

[0020] In the embodiment of the present invention, the processing device 4 generates superimposed video data MS by superimposing image data replacing VAPSART on each frame data of the remote person video data MR. The processing device 4 displays the superimposed video data MS with the image data superimposed on it on the child side display device of the child terminal 2. The processing device 4 uses the child terminal 2 and the remote person terminal 3 to connect the space where the child is and the space where the remote person is with video data in real time, and supports remote communication between the child and the remote person using image data replacing VAPSART.

[0021] In the embodiment of the present invention, a case will be described in which the processing device 4 is formed in a housing different from the housings of the child terminal 2 and the remote terminal 3, but this is not limited to the case. For example, each function of the processing device 4 may be implemented in either the child terminal 2 or the remote terminal 3, or may be divided and implemented in the child terminal 2 and the remote terminal 3, respectively. For example, in the embodiment of the present invention, a case will be described in which the processing device 4 generates the superimposed video data MSa by horizontally flipping the superimposed video data MS, but the remote terminal 3 may also generate the superimposed video data MSa by horizontally flipping the superimposed video data MS.

[0022] As shown in FIG. 3, the processing device 4 includes a video control unit 41 and a display control unit .

[0023] The video control unit 41 outputs superimposed video data MS in which an image specified by the remote person is superimposed at a position specified by the remote person in the frame data of the remote person video data MR, and displays it on the child terminal 2. The video control unit 41 identifies the position specified by the remote person on the remote person video display unit 312, converts the identified position to a position in the frame data of the remote person video data MR, and outputs superimposed video data MS in which an image is superimposed at the converted position.

[0024] The remote person specifies the identifier of the desired image and the position in the frame data of the remote person video data MR at which the desired image is to be displayed. For example, as shown in the lower right of the remote person's display device 31 in FIG. 2(b), the remote person scrolls to select an image to replace the VAPSART, and specifies a position on the remote person's video display unit 312 to specify the desired image and the position at which the image is to be displayed. If the remote person's display device 31 is a touch panel, the remote person may scroll the images by swiping and select the desired image by touching. If the remote person's display device 31 is not a touch panel, the remote person may select the desired image by operating a mouse or the like. In addition, a case will be described in which the image to replace the VAPSART is created in advance in the communication support system 1, but the remote person may create the image before or during communication with the child.

[0025] The remote terminal 3 transmits the identifier and the position of the image specified by the remote user to the processing device 4. The remote terminal 3 may transmit the specified image data itself to the processing device 4 instead of the type of image specified.

[0026] The video control unit 41 of the processing device 4 superimposes image data corresponding to the identifier of the image transmitted from the remote terminal 3 on the frame data MR1 of the remote person video data MR according to the position transmitted from the remote terminal 3. The video control unit 41 continues superimposing the specified image data for a predetermined time or until an instruction to stop displaying the desired image is input from the remote terminal 3. The video control unit 41 may shift the position at which the specified image data is displayed for each frame, and may perform an effect such as shaking the image data in the left and right direction. Furthermore, in response to an instruction from the remote person, the video control unit 41 may change the position at which the image is superimposed in the up, down, left and right directions, switch to a different image, change the size of the image, etc. The change in the position at which the image is superimposed in the up, down, left and right directions is an effect that imitates the movement of a vape-sart. The switch to a different image is an effect that imitates the rotation of a vape-sart. The change in the size of the image is an effect that imitates the movement of a vape-sart approaching in front of a child.

[0027] The display control unit 42 controls the display of video data on the remote party's display device 31. The display control unit 42 acquires child video data MC captured by the child terminal 2, and outputs it to the remote party's terminal 3 for display. The display control unit 42 also outputs remote party video data MR, etc. to the remote party's terminal 3 for display.

[0028] There are several possible methods for the remote person to specify the position on which to superimpose the desired image. For example, (1) the remote person specifies the position on the remote person video display unit 312, (2) the remote person specifies the position on the remote person video display unit 312 which displays the superimposed video data MS in a horizontally inverted manner, and (3) the remote person carries a printed matter of a code corresponding to the desired image. In an embodiment of the present invention, a case will be described where an image of an apple is superimposed instead of Vapesart.

[0029] First, (1) a method in which a remote person specifies a position on remote person video display unit 312 will be described.

[0030] 4(a), the remote person video display section 312 displays the remote person video data MR or the superimposed video data MS. In this remote person video display section 312, the remote person specifies the position at which to superimpose the image by using a mouse, touching, etc. The remote person terminal 3 acquires the position specified by the remote person in the remote person video display section 312.

[0031] The vertical and horizontal sizes of remote person video display section 312 and the vertical and horizontal sizes (number of pixels) of one frame data of remote person video data MR are compared, and the position specified by the remote person in remote person video display section 312 is converted to the position of the frame data of remote person video data MR. The converted position is specified as the position at which image data is to be superimposed. Note that the position at which image data is to be superimposed may be specified by remote person terminal 3, and the specified position may be transmitted to processing device 4. Alternatively, the position specified by the remote person in remote person video display section 312 may be transmitted to processing device 4, and processing device 4 may specify the position at which image data is to be superimposed.

[0032] In the example shown in Figure 4(a), when the right side facing the remote person video display unit 312 (the left side as seen by the remote person in the remote person video data MR) is selected, the video control unit 41 generates superimposed video data MS in which an image is superimposed on the right side facing the frame data (the left side as seen by the remote person in the remote person video data MR).

[0033] As shown in FIG. 4(b), the child-side display device 21 displays superimposed video data MS with an image of an apple superimposed on the right side as viewed from the direction facing the child-side remote person video display unit 211 (the left side as viewed from the remote person in the remote person video data MR). For example, when the remote person asks the child, "Which is the apple?", the child facing the child-side remote person video display unit 211 answers that it is the right by raising his right hand or touching the right display. In this case, the child-side camera 22 that has captured this state outputs child video data MC that selects the left side as viewed from the direction facing the child video display unit 311 (the right side as viewed from the child in the child video data MC), as shown in FIG. 4(c). At this time, the remote person video display unit 312 of the remote person terminal 3 displays the video data with the image superimposed on the right side as viewed from the remote person video display unit 312 (the left side as viewed from the remote person in the remote person video data MR), as shown in FIG. 4(d).

[0034] In the example shown in Figure 4, the remote person will understand that he or she is communicating appropriately with the child if the direction in which the child responds is opposite to the position indicated to display the image, or opposite to the image position in the remote person video display section 312.

[0035] Next, (2) a method in which a remote person specifies a position on the remote person video display section 312 that displays the superimposed video data MS in a horizontally inverted manner will be described.

[0036] 5(a), the remote user video display unit 312 of the remote user terminal 3 displays the remote user video data MR or the superimposed video data MS in a horizontally inverted manner. In this remote user video display unit 312, the remote user specifies the position at which to superimpose the image by using a mouse, touch, etc. The remote user terminal 3 acquires the position specified by the remote user in the remote user video display unit 312.

[0037] The vertical and horizontal sizes of remote person video display section 312 are compared with the vertical and horizontal sizes (number of pixels) of one frame data of remote person video data MR, and the position specified by the remote person in remote person video display section 312 is converted into the position of the frame data of remote person video data MR by left-right inverting the position. The converted position is specified as the position where image data is to be superimposed. As in (1) above, the position where image data is to be superimposed may be specified by remote person terminal 3 or by processing device 4.

[0038] In the example shown in Figure 5(a), when the left side facing the remote participant video display unit 312 of the remote participant terminal 3 (the left side as seen by the remote participant in the remote participant video data MR) is selected, the video control unit 41 generates superimposed video data MS in which an image is superimposed on the right side of the frame data (the left side as seen by the remote participant in the remote participant video data MR).

[0039] As shown in FIG. 5(b), the child-side display device 21 displays the superimposed video data MS with an image of an apple superimposed on the right side as viewed from the direction facing the child-side remote person video display unit 211 (the left side as viewed from the remote person in the remote person video data MR). For example, when the remote person asks the child, "Which is the apple?", the child facing the child-side remote person video display unit 211 answers that it is the right by raising his right hand or touching the right display. In this case, the child-side camera 22 that has captured this state outputs child video data MC that selects the left side as viewed from the direction facing the child video display unit 311 (the right side as viewed from the child in the child video data MC), as shown in FIG. 5(c). At this time, the superimposed video data MS is displayed in a horizontally inverted manner on the remote person video display unit 312 of the remote person terminal 3, as shown in FIG. 5(d). The video data with the image superimposed is displayed on the left side as viewed from the direction facing the remote person video display unit 312 (the left side as viewed from the remote person in the remote person video data MR).

[0040] An example of the process of the communication support system 1 in case (2) will be described with reference to FIG.

[0041] The display control unit 42 displays, vertically side by side, a child video display unit 311 that displays child video data MC and a remote person video display unit 312 that displays superimposed video data MS in a horizontally inverted manner on the remote person's display device 31. In step S11, the remote person's terminal 3 specifies a position on the remote person's video display unit 312 and inputs an instruction to the video control unit 41 to display an image.

[0042] In step S12, video control unit 41 horizontally flips the position specified on remote user terminal 3 and converts it to a position on remote user video data MR. In step S13, video control unit 41 superimposes image data of an apple on the position converted in step S12 to generate superimposed video data MS.

[0043] In step S14, the video control unit 41 transmits the superimposed video data MS generated in step S13 to the child terminal 2. The child terminal 2 reproduces the received superimposed video data MS in the child-side remote person video display unit 211.

[0044] In step S15, the video control unit 41 transmits the superimposed video data MS generated in step S13 to the display control unit 42. In step S16, the display control unit 42 plays the received superimposed video data MS in the remote user video display unit 312 of the remote user terminal 3 by flipping the data horizontally.

[0045] 5 and 6, the remote person understands that he is communicating appropriately with the child if the direction in which the child reacts is the same as the direction shown to display the image or the same as the direction of the image position in remote person video display section 312. In the example shown in Fig. 5, compared to the example shown in Fig. 4, the direction in which the child reacts matches the image position, so the remote person can intuitively understand that he is communicating appropriately with the child.

[0046] Here, it is preferable that the display control unit 42 display the child video display unit 311 and the remote person video display unit 312 side by side in the vertical direction, as shown in Fig. 2(b) etc. In this way, the direction in which the child reacted and the position shown to display the image, or the image position in the remote person video display unit 312, are aligned in the vertical direction, so that the remote person can more intuitively understand that he or she is communicating appropriately with the child.

[0047] Furthermore, (3) a method in which a remote party possesses a printout of a code corresponding to a desired image is described.

[0048] The remote person's camera 32 outputs remote person video data MR including a medium on which a symbol is written within the shooting area. The symbol is a notation that can be read by the processing device 4, such as a barcode, a two-dimensional barcode, letters such as the alphabet, an image, etc. Each symbol is associated with image data that replaces a Vapsert. The remote person may hold the medium on which the symbol is written, or may place it on a desk. When the video control unit 41 detects a specific symbol in the frame data of the remote person's video data MR, it outputs superimposed video data MS in which an image specified by the remote person is superimposed at the position where the symbol was detected. Image data corresponding to the symbol detected by the video control unit 41 is superimposed at the position where the symbol was detected, and the superimposed video data MS is generated.

[0049] An example of the process of the communication support system 1 in case (3) will be described with reference to FIG.

[0050] In step S31, the remote user terminal 3 inputs remote user video data MR including a medium on which a symbol is written within a photographing area to the video control unit 41. In the example shown in Fig. 7, the symbol is a code in the form of a two-dimensional barcode.

[0051] In step S32, video control unit 41 identifies the position of the symbol in the frame data of the remote person video data MR transmitted in step S31. In step S33, video control unit 41 superimposes image data of an apple at the position identified in step S32 to generate superimposed video data MS.

[0052] In step S34, the video control unit 41 transmits the superimposed video data MS generated in step S33 to the child terminal 2. The child terminal 2 reproduces the received superimposed video data MS in the child-side remote person video display unit 211.

[0053] In step S35, the video control unit 41 transmits the superimposed video data MS generated in step S33 to the display control unit 42. In step S36, the display control unit 42 plays the received superimposed video data MS in the remote user video display unit 312 of the remote user terminal 3 by flipping it horizontally.

[0054] In the example shown in Fig. 7, the superimposed video data is displayed with left-right inversion in remote person video display section 312. The remote person understands that he is communicating appropriately with the child if the direction in which the child reacts is the direction indicated to display the image or the direction of the image position in remote person video display section 312. In the example shown in Fig. 7, compared to the example shown in Fig. 4, the direction in which the child reacts matches the image position, so the remote person can intuitively understand that he is communicating appropriately with the child.

[0055] As explained in (2) above, it is preferable that the display control unit 42 displays the child video display area 311 and the remote person video display area 312 side by side in the vertical direction, as shown in Fig. 2(b) etc. The remote person can more intuitively grasp that he or she is communicating appropriately with the child.

[0056] In the method shown in (3), the remote person only has to hold a medium that displays a symbol that replaces the VAPSART symbol and behave in the same way as if they were in the same space as the child. The remote person can communicate using an image that replaces the VAPSART symbol without having to select an image to replace the VAPSART symbol or select a display position, reducing the operational burden on the remote person.

[0057] Although the above methods (1) to (3) have been described as methods for specifying the position at which a desired image is to be superimposed, the communication support system 1 only needs to support at least one of these methods, and may support multiple methods. The communication support system 1 may be equipped with methods (1) and (2), for example, and a method for specifying the position at which an image is to be drawn may be specified by an operation of a remote person.

[0058] Next, the extension mode used when displaying the remote person video data MR will be described. When the remote person's camera 32 is vertically long and the distance to the remote person is short, the remote person may be captured over a wide range of the frame data MR1 of the remote person video data MR, as shown in Fig. 8(a). Since the area in which image data is superimposed in such remote person video data MR is narrow, it may be difficult for the remote person to identify the direction in which a child who has difficulty in fine operations, such as an infant, reacts.

[0059] Therefore, when the extension mode is specified, the video control unit 41 outputs extended remote person video data MR' in which extension areas are provided on the edges of the frame data of the remote person video data MR. Frame data MR1' of the extended remote person video data MR' shown in Fig. 8(b) has extension areas ER provided on both the left and right ends of frame data MR1 of the remote person video data MR.

[0060] The video control unit 41 superimposes an image on the frame data of the extended remote user video data based on the position specified by the remote user on the remote user video display unit 312, which displays the horizontally inverted extended remote user video data. The extended mode expands the area in which the remote user specifies the image display. For example, by displaying an image on both the right and left ends, the range of the child's reaction can be expanded, making it easier for the remote user to check the child's reaction.

[0061] The process in the extended mode will be described with reference to FIG.

[0062] In step S51, the remote user terminal 3 transmits remote user video data MR to the video control unit 41. At this time, it is assumed that the extension mode is set to ON.

[0063] In step S52, the video control unit 41 adds an extension area ER to each end of each frame data of the remote person video data MR to generate each frame data of the extended remote person video data MR'. In step S53, the video control unit 41 transmits the extended remote person video data MR' generated in step S52 to the child terminal 2, and plays it on the child side remote person video display unit 211.

[0064] In step S54, the video control unit 41 inputs the extended remote viewer video data MR' generated in step S52 to the display control unit 42. In step S55, the display control unit 42 plays the received extended remote viewer video data MR' in the remote viewer video display unit 312 of the remote viewer terminal 3 with horizontal flipping.

[0065] The extended mode allows the remote person to specify the position where the image is to be superimposed, including the extended area ER, making it easier to communicate with infants and young children who have difficulty with fine movements.

[0066] In the above description, the remote person checks the child video data MC to identify the direction in which the child responded, but the present invention is not limited to this. For example, if the child-side display device 21 of the child terminal 2 is a touch panel, the coordinates at which the child touched the child-side display device 21 may be transmitted from the processing device 4 to the remote person's terminal 3 and displayed on the child video display unit 311. The remote person can verify whether the remote person's question has been answered appropriately based on the coordinates at which the child touched. In addition, when the child touches the image displayed on the child-side display device 21, the video control unit 41 may change the position at which the image is superimposed in the up, down, left, or right direction, rotate the image, change the size of the image, etc.

[0067] Next, when the position of child's side camera 22 is not at the center in the left-right direction of child's side display device 21 and the child does not face child's side camera 22, the remote person may not be able to accurately grasp the touch position of the child observed on child video display unit 311. Therefore, a case will be described where display control unit 42 adjusts the positions of child video display unit 311 and remote person video display unit 312 in remote person's side display device 31 so that the direction of the child's touch can be accurately grasped even when the child does not face child's side camera 22.

[0068] The display control unit 42 adjusts the display position of the child video display unit 311 or the remote person video display unit 312 so that in the child video data MC displayed by the child video display unit 311, the center of the position where the child reacted to the left and right sides of the child side display unit 21, respectively, and the left-right center of the remote person in the overlaid video data MS displayed by the remote person video display unit 312 are aligned vertically.

[0069] As shown in Fig. 10(a), the child is made to touch the right side of the screen. As shown in Fig. 10(b), in the child video display unit 311 of the remote person's display device 31, the display control unit 42 acquires the X coordinate value Xr of the position touched by the child. The display control unit 42 may estimate the position of the child's hand from an estimation of the child's posture when the child reacts to the right edge, and acquire the X coordinate value Xr of the position of the child's hand. The method of estimating the position of the child's hand from the estimation of the posture may be an existing method.

[0070] As shown in Fig. 11(a), the child is made to touch the left side of the screen. As shown in Fig. 11(b), in the child video display unit 311 of the remote person's display device 31, the display control unit 42 acquires the X coordinate value Xl of the position touched by the child. The display control unit 42 may estimate the position of the child's hand from an estimation of the posture when the child reacts to the left edge, and acquire the X coordinate value Xl of the position of the child's hand.

[0071] 12(a) shows the relationship between Xr acquired in FIG. 10, Xl acquired in FIG. 11, XC_C of the center between Xr and Xl, and the left-right center line XT_C of the remote person on remote person video display unit 312. As shown in FIG. 12(b), display control unit 42 shifts the display position of child video display unit 311 or remote person video display unit 312 so that XC_C of the center between Xr and Xl coincides with the left-right center line XT_C of the remote person on remote person video display unit 312. In FIG. 12(b), the display position of child video display unit 311 is shifted. This allows the remote person to know whether the child is touching the left side or the right side based on the left-right center line of the remote person on remote person video display unit 312.

[0072] The conversion before and after the coordinate shift on the remote person's display device 31 in the child video display section 311 is expressed by equation (1). In the example shown in Fig. 12(a), since XC_C is larger than XT_C, the child video display section 311 shifts to the left as shown in Fig. 12(b).

[0073] [Number 1] For all x, childImage(x,y) =childimage(x-(XC_C - XT_C), y) ···Formula (1) childimage(x,y): coordinates before shifting childImage(x,y): The coordinates after shifting

[0074] As shown in FIG. 12(a), in the image before the coordinates are shifted, the relationship between the X coordinate on the remote viewer's display device 31 where the child appears to be touching and the X coordinate on the remote viewer's display device 31 of the position where the remote viewer is superimposing the image is expressed by equation (2).

[0075] [Number 2] XT_S = (X_S)-(XC_C -XT_C) Formula (2) XT_S: X coordinate of the position where the remote person is overlaying the image X_S: The X coordinate of the position where the child appears to be touched.

[0076] 12, the shift amount is expressed as XC_C-XT_C. If XC_C-XT_C is a and child video display unit 311 is shifted left or right, when the value of a is positive, child video display unit 311 shifts leftward, and when the value of a is negative, child video display unit 311 shifts rightward.

[0077] 12(b), the range in which the remote person can specify the position where the image is to be superimposed ranges from Xl-a to Xr-a in the X-axis direction. The display control unit 42 may display the range in which the remote person can specify the position where the image is to be superimposed on the remote person video display unit 312.

[0078] In the state shown in Fig. 12(b), in child video display unit 311, when the child touches the left side, the position Xl-a to Xr-a where the remote person superimposes the image overlaps the remote person, and there is not enough area to specify the position to superimpose the image. Therefore, display control unit 42 reduces the display area of ​​remote person video data MR displayed on remote person video display unit 312 as shown in Fig. 13. Remote person video display unit 312 is reduced so that the remote person fits within the range from Xl-a to Xr-a in the X-axis direction and an area to superimpose the image is secured to the left and right of the remote person. This secures areas to the left and right of the remote person to specify the position to superimpose the image.

[0079] According to such a communication support system 1, it is possible to support communication between a child located in a different space and a remote person.

[0080] The processing device 4 of the present embodiment described above is, for example, a general-purpose computer system including a CPU (Central Processing Unit, processor) 901, a memory 902, a storage 903 (HDD: Hard Disk Drive, SSD: Solid State Drive), a communication device 904, an input device 905, and an output device 906. In this computer system, the CPU 901 executes a predetermined program loaded on the memory 902, thereby realizing each function of the processing device 4.

[0081] The processing device 4 may be implemented in one computer, or in multiple computers, or may be a virtual machine implemented in a computer.

[0082] The program of the processing device 4 can be stored in a computer-readable recording medium such as an HDD, SSD, USB (Universal Serial Bus) memory, CD (Compact Disc), or DVD (Digital Versatile Disc), or can be distributed via a network.

[0083] The present invention is not limited to the above-described embodiment, and various modifications are possible within the scope of the present invention. [Explanation of symbols]

[0084] 1. Communication Support System 2. Children's devices 3 Remote terminal 4 Processing Equipment 21 Child side display device 22 Child's camera 31 Remote display device 32 Remote camera 41 Video control section 42 Display control section 211 Child side remote person video display unit 311 Children's video display section 312 Remote Video Display Unit MC Children's Video Data MR remote person video data MR' Extended Remote Person Video Data MRa Remote subject video data (left-right flipped) MS Overlay Video Data MSa Overlay video data (left-right flip)

Claims

1. A child-side camera that outputs child video data capturing a child; a remote person side camera that outputs remote person video data capturing a remote person located outside the space where the child is present; a video control unit that outputs superimposed video data in which an image designated by the remote user is superimposed on a position designated by the remote user in frame data of the remote user video data output by the remote user side camera; a child-side display device that displays the superimposed video data; A remote-side display device for displaying the child video data Equipped with A display control unit that vertically displays a child video display unit that displays the child video data and a remote video display unit that displays the superimposed video data in a horizontally inverted manner on the remote person side display device. The communication support system further comprises:

2. The video control unit is A position designated by the remote person on the remote person video display unit is identified, the identified position is converted to a position in frame data of the remote person video data, and the superimposed video data in which the image is superimposed at the converted position is output.

2. The communication support system according to claim 1.

3. The video control unit is outputting extended remote person video data in which an extended area is provided at an edge of frame data of the remote person video data; The image is superimposed on frame data of the extended remote person video data based on a position designated by the remote person on a remote person video display unit that displays the left-right inverted extended remote person video data.

3. The communication support system according to claim 2.

4. The video control unit is When a predetermined symbol is detected in frame data of the remote person video data, the superimposed video data in which an image designated by the remote person is superimposed at the position where the symbol is detected is output.

2. The communication support system according to claim 1.

5. The display control unit is In the remote person side display device, the display position of the child video display unit or the remote person video display unit is adjusted so that the center of the position where the child reacted to the left and right sides of the child side display device in the child video data displayed by the child video display unit and the center of the remote person in the left-right direction in the superimposed video data displayed by the remote person video display unit are aligned in the vertical direction.

2. The communication support system according to claim 1, further comprising:

6. A remote person's side display device for use in a communication support system including a child side camera which outputs child video data obtained by capturing a child, a remote person's side camera which outputs remote person's video data obtained by capturing a remote person located in a space other than the space in which the child is present, a child side display device which displays the remote person's video data, and a remote person's side display device which displays the child video data, a child moving image display section for displaying the child moving image data and a remote person moving image display section for displaying the remote person moving image data in a left-right inverted manner, the child moving image display section and the remote person moving image display section are vertically arranged and displayed; The remote person video display unit displays superimposed video data in which an image designated by the remote person is superimposed at a position designated by the remote person in frame data of the remote person video data and the superimposed video data is inverted from left to right.

13. A remote user's display device comprising:

7. A child-side camera that outputs child video data capturing a child; a remote person side camera that outputs remote person video data capturing a remote person located outside the space where the child is present; A child-side display device that displays the superimposed video data described later; A processing program for causing a computer to function as a processing device connected to a remote user side display device that displays the child video data, a video control unit that outputs superimposed video data in which an image designated by the remote user is superimposed on a position designated by the remote user in frame data of the remote user video data output by the remote user side camera; A display control unit that vertically displays a child video display unit that displays the child video data and a remote video display unit that displays the superimposed video data in a horizontally inverted manner on the remote person side display device. A processing program that acts as a

Images