Meta live service provision system and method
The meta live service system integrates PTZ-controlled video streaming with virtual and augmented reality to create an immersive metaverse experience, addressing limitations in existing technologies by enabling real-time video merging and user-controlled environments, thus enhancing user engagement and interaction.
Patent Information
- Application Number
- PCT/KR2024/006153
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-08
- Filing Date
- 2024-05-08
- Publication Date
- 2024-11-14
AI Technical Summary
Current technologies lack the capability to provide immersive and interactive meta live services that combine real-time video streaming with virtual and augmented reality elements, limiting user engagement and control in virtual environments.
A meta live service system and method that utilizes a video providing device with PTZ (Pan Tilt Zoom) control, allowing for online and offline operation, merging real-time video with virtual and augmented reality layers to create a 360-degree immersive experience, and enabling users to control camera viewpoints and environment settings through a network-connected interface.
Enables the delivery of a realistic and interactive metaverse experience by merging real-time video with virtual and augmented reality, providing users with enhanced control and immersion, and allowing for dynamic background changes based on video content changes, thereby improving user engagement and interaction.
Smart Images

Figure KR2024006153_14112024_PF_FP_ABST
Abstract
Description
Meta Live Service Provision System and Method
[0001] The present invention relates to a system and method for providing a meta live service through linkage with a video providing device.
[0002]
[0003] The content described below merely provides background information related to the present embodiment and does not constitute prior art.
[0004] A field that has recently been receiving a lot of attention from around the world is virtual reality (hereinafter abbreviated as "VR").
[0005] VR refers to a specific environment or situation, or the technology itself, created using artificial technologies such as computers, that resembles reality but is not real. These virtual environments or situations stimulate the user's five senses, allowing them to experience spatial and temporal experiences similar to reality, freely blurring the boundaries between reality and imagination. Furthermore, users can interact with virtual objects not only by simply immersing themselves in the virtual world but also by using real devices to control and command them. VR is distinct from unilateral simulations in that it allows for user interaction and creates user experiences.
[0006] Recently, with the advancement of VR technology, interest in the metaverse has been on the rise. The metaverse, a portmanteau of "meta" (virtual), "transcendence" (meta), and "universe" (world), refers to a three-dimensional virtual world, a universe beyond the universe itself. More specifically, it refers to a virtual world, both real and unreal, that can coexist across all aspects of politics, economy, society, and culture, both for living and for gaming.
[0007] This is an alternative world created using digital data that is similar to or completely different from reality. In the virtual world, users engage in activities similar to economic and social activities in the real world through avatars.
[0008] The virtual world is the most familiar form of metaverse to us, and is a general term for communities implemented in a 3D computer graphics environment, ranging from online role-playing games like Lineage to living virtual worlds like Second Life developed by Linden Lab.
[0009] With recent advancements in communication technology and the increasing performance of smart devices like smartphones, the market for various content streaming services via smart devices has also grown significantly. Content streaming services essentially transmit digital content from a specific server to a subscriber's device, such as a smartphone. In other words, users receive and consume digital content in real time via their smartphones.
[0010] Additionally, with the advancement of high-performance smart devices and hardware, development of services utilizing realistic metaverse spaces is underway.
[0011] [Prior Art Literature]
[0012] [Patent Document]
[0013] (Patent Document 1) Domestic Patent Publication No. 10-1770648
[0014]
[0015] The present invention provides a meta live service providing system and method capable of providing a meta live service to a user terminal connected through a network based on an image received in conjunction with an image providing device.
[0016] In addition, the present invention provides a meta live service providing system and method capable of online or offline control of a video providing device.
[0017] The problems to be solved by the present invention are not limited to those mentioned above, and other problems to be solved that are not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention pertains from the description below.
[0018]
[0019] In order to achieve the above purpose, a meta live service providing system according to an embodiment of the present invention may include an image providing device that is installed at a predetermined location and displays a live image in a viewing direction and is capable of moving up, down, left, right, and zoom based on the installed location, and changes the live image according to the up, down, left, right, and zoom movements and displays the changed live image on a display, and transmits the live image through a network, and a service providing server that is connected to the image providing device through a network to receive the live image, and provides a meta live service to a user terminal connected through the network using merged data in which the live image and a virtual reality layer are merged.
[0020] According to an embodiment of the present invention, the image providing device can be a telescope capable of PTZ (Pan Tilt Zoom) control, and capable of up / down / left / right movement and zoom movement through the control of the PTZ.
[0021] According to an embodiment of the present invention, the image providing device can display on the display a combined layer for a background for the live image, a layer for augmented reality for servicing a plurality of contents within the live image, and a layer for virtual reality for a virtual reality service within the live image.
[0022] According to an embodiment of the present invention, the image providing device can operate in an online control state in which up, down, left, right, and zoom are controlled by the service providing server based on a control request from the user terminal, or in an offline control state in which the device is controlled by the user at the installed location.
[0023] According to an embodiment of the present invention, the service providing server generates a 360-degree background image using a live image received from the image providing device and stores the same in a database, and when a connection request is received from a user terminal through a wired or wireless network while the image providing device is in an offline control state, provides an interface for setting a virtual camera viewpoint to the user terminal, and then generates a virtual camera machine having a virtual camera viewpoint (a direction looking at a part of a 360-degree image) set through the interface, and a background layer can be generated using the generated virtual camera machine.
[0024] According to an embodiment of the present invention, the service providing server provides the user terminal with a control interface capable of controlling up / down / left / right and zoom movement of the virtual camera machine from the virtual camera viewpoint of the virtual camera machine created above, and when a control request is received through the control interface, the virtual camera machine can be controlled to generate a background layer corresponding to a live image.
[0025] According to an embodiment of the present invention, the service providing server provides the user terminal with an environment setting interface that can set an environment, creates an environment layer according to the environment set by the environment setting interface, and creates the merged data using the environment layer and the background layer, and then provides a meta live service based on the merged data.
[0026] According to an embodiment of the present invention, the server providing server generates a virtual reality layer including an avatar and a ground shape selected by a user, and provides a meta live service to the user terminal using merged data that merges a background layer corresponding to the live image, a virtual reality layer, and an augmented reality layer for providing a plurality of contents.
[0027] According to an embodiment of the present invention, the service providing server calculates the degree of change in a live image received from the image providing device, and based on the degree of change, determines whether to change a background layer in an image corresponding to a meta live service streamed to the user terminal.
[0028] In order to achieve the above object, a method for providing a meta live service according to an embodiment of the present invention is provided, which is installed at a predetermined location and displays a live image in a viewing direction and is capable of moving up, down, left, right, and zoom based on the installed location, and changes the live image according to the up, down, left, right, and zoom movements and displays the changed live image on a display, and transmits the live image through a network. The method may include a step of receiving a meta live service request from a user terminal connected through a network at a service providing server connected to the image providing device through a network, a step of receiving a live image from the image providing device according to the meta live service request, and a step of providing a meta live service to a user terminal connected through the network using merged data in which the live image and a virtual reality layer are merged.
[0029] According to an embodiment of the present invention, the method for providing the meta live service includes a step of providing the user terminal with an environment setting interface that can set an environment when the meta live service request is received, and the step of providing the meta live service includes creating an environment layer according to the environment set by the environment setting interface when the live video is received, and using the environment layer and the background layer to create the merged data, and then providing the meta live service based on the merged data.
[0030] According to an embodiment of the present invention, the step of providing the meta live service may include a step of creating a virtual reality layer including an avatar and a ground shape selected by a user, a step of merging a background layer corresponding to the live image and an augmented reality layer for providing a plurality of contents based on the live image into the virtual reality layer to create merged data, and a step of providing the meta live service using the merged data.
[0031] According to an embodiment of the present invention, the meta live service providing method further includes a step of receiving a live image in real time from the image providing device, and a step of calculating a degree of change between the received live images.
[0032] The step of providing the above meta live service can determine whether to change the background layer in the video corresponding to the meta live service streamed to the user terminal based on the above change rate.
[0033]
[0034] According to the embodiments of the present invention described above, it is possible to implement a realistic metaverse-based service by providing a meta live service to a user terminal connected through a network based on an image received in conjunction with an image providing device.
[0035]
[0036] FIG. 1 is a diagram illustrating the overall configuration of a meta live service providing system according to an embodiment of the present invention.
[0037] FIG. 2 is a block diagram illustrating a detailed configuration of a service providing server in a meta live service providing system according to an embodiment of the present invention.
[0038] FIG. 3 is a drawing showing the overall structure of an image providing device having a PTZ function according to an embodiment of the present invention.
[0039] FIG. 4 is a perspective view illustrating a coupling structure between a position conversion unit and a photographing unit of an image providing device having a PTZ function according to an embodiment of the present invention.
[0040] FIG. 5 is a side view of a module mounted inside a housing in an image providing device having a PTZ function according to an embodiment of the present invention.
[0041] Fig. 6 is a drawing showing the structure of a pedestal applied to an embodiment of the present invention.
[0042] FIG. 7 is a drawing showing the structure of a first support bracket in an image providing device having a PTZ function according to an embodiment of the present invention.
[0043] FIG. 8 is a drawing for explaining the structure of a second support bracket in an image providing device having a PTZ function according to an embodiment of the present invention.
[0044] FIG. 9 is a drawing for explaining the mounting state of a tilting module of an image providing device having a PTZ function according to an embodiment of the present invention.
[0045] FIG. 10 is a drawing for explaining the components of a photographing unit of an image providing device having a PTZ function according to an embodiment of the present invention.
[0046] FIG. 11 is a drawing for explaining the structure of a housing of an image providing device having a PTZ function according to an embodiment of the present invention.
[0047] FIG. 12 is a drawing for explaining a configuration for limiting the panning angle and tilting angle of an image providing device having a PTZ function according to an embodiment of the present invention.
[0048] Figure 13 is a flowchart illustrating a process in which an image providing device according to an embodiment of the present invention transmits an image.
[0049] Figure 14 is a flowchart illustrating a process in which a service providing server according to an embodiment of the present invention provides a meta live service.
[0050] FIG. 15 is a flowchart illustrating a process for providing a meta live service when an image providing device according to an embodiment of the present invention is unable to be controlled online.
[0051]
[0052] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings. Prior to this, terms or words used in this specification and claims should not be interpreted as limited to their typical or dictionary meanings, and should be interpreted with meanings and concepts that conform to the technical spirit of the present invention based on the principle that the inventor can appropriately define the concept of the term to best explain his or her own invention. Therefore, the embodiments described in this specification and the configurations illustrated in the drawings are merely the most preferred embodiments of the present invention and do not represent all of the technical spirit of the present invention. Therefore, it should be understood that various equivalents and modified examples may exist that can replace them at the time of this application.
[0053] Hereinafter, a meta live service providing system and method according to an embodiment of the present invention will be described with reference to the attached drawings.
[0054] FIG. 1 is a drawing showing the overall configuration of a meta live service providing system according to an embodiment of the present invention, and FIG. 2 is a block diagram showing the detailed configuration of a service providing server in a meta live service providing system according to an embodiment of the present invention.
[0055] As illustrated in FIGS. 1 and 2, the meta live service providing system may include a video providing device (510) installed in a predetermined location and displaying a live video in a direction of view, and a service providing server (520) connected to the video providing device (510) through a network, for example, a wired or wireless communication network, to receive the live video, and to provide a streaming video service using merged data in which a virtual reality layer and a live video are merged to a user terminal (530) connected through the network.
[0056] In particular, in the embodiment of the present invention, the service provided using merged data may refer to a metaverse service. In other words, the meta live service may refer to providing content that enables the provision of an extended reality service that combines various contents, such as virtual, augmented, and mixed reality, at a precise location based on actual images acquired through an image providing device (510) based on structured data.
[0057] Additionally, in the embodiment of the present invention, the live image is an image captured by the image providing device (510), and may be an image captured by the image providing device (510) when looking at an arbitrary point for a preset period of time.
[0058] The merged data streamed by the service providing server (520) according to the embodiment of the present invention expresses virtual reality, augmented reality, and mixed reality, and may include a background layer expressing an actual environment (i.e., a background area) using live video, a virtual reality layer expressing an avatar and ground shape, and an augmented background layer for providing various information and highlighting fun elements. That is, the merged data may be generated by merging a background layer, a virtual reality layer, and an augmented reality layer, and then provided as a streaming service to a user terminal (530).
[0059] In an embodiment of the present invention, an augmented reality layer may be generated based on a live image, i.e., a background layer corresponding to a background area, i.e., may be generated based on 2D or 3D data for providing guide content for major viewing points within the background area and various information for highlighting fun elements.
[0060] Additionally, the virtual reality layer can reflect changes in the state of the avatar and the ground based on the surrounding environment, such as weather and seasonal information.
[0061] To this end, the service providing server (520) according to the embodiment of the present invention can generate a streaming video for a meta live service by modifying a reference virtual reality layer stored in a database (525) based on weather and seasonal information about a location where a video providing device (510) transmitting a live video is installed, and then using the background layer corresponding to the received live video and the modified reference virtual reality layer.
[0062] In addition, the service providing server (520) can generate a streaming video by including an augmented reality layer generated based on 2D or 3D data to provide guide content for major viewing points within the background area and various information to highlight fun elements, and then service the streaming video to the user terminal (530).
[0063] The user terminal (530) is a device that can receive the meta live service, and may be, for example, a computing device having a wired or wireless communication function, a memory in which a program that can provide the meta live service is stored by being executed by at least one process that can receive the meta live service and that can provide the meta live service in conjunction with the service providing server (520), and an input interface (not shown) that allows for various user inputs.
[0064] In the embodiment of the present invention, the computing device corresponding to the user terminal (530) may include, but is not limited to, a wireless communication device such as a smartphone, tablet PC, personal computer, or laptop.
[0065] The image providing device (510) according to an embodiment of the present invention can control up, down, left, right, and zoom movement based on the installed location, that is, PTZ (Pan Tilt Zoom) control, and can change and display a live image according to up, down, left, right, and zoom movement, and transmit the live image to a service providing server (520).
[0066] In particular, the video providing device (510) can capture a live video as the up / down / left / right and zoom movements are completed, and transmit the captured live video to the service providing server (520).
[0067] In addition, the video providing device (510) can control up, down, left, right, and zoom movement based on offline control (i.e., control by any user at the installation location) and online control (i.e., control by a service providing server (520) connected via a network), and can provide various services provided by the video providing device (510). Here, the services can include live video display, various content services, etc.
[0068] In addition, the image providing device (510) is controlled up, down, left, right, and zoom (i.e., online controlled) by the service providing server (520) based on a control request from the user terminal (530), and according to the online control, a message indicating that online control is in progress is displayed on the display, and a real-time live image can be transmitted to the service providing server (520) according to the online control. Accordingly, the service providing server (520) can provide the user terminal (530) with a meta live service including a background area and a virtual background layer and an augmented background layer required for the background area based on the real-time live image.
[0069] In addition, the video providing device (510) transmits a live video shot according to offline control to the service providing server (520) as it is controlled at the installed location (i.e., offline control), and transmits a control-related message to the service providing server (520) to notify that the control is in an offline state in order to deactivate the control by the user terminal (530), and transmits a control-related message to the service providing server (520) to notify that the control is possible as it enters a standby state for a preset period of time. In this case, the service providing server (520) may include a control-related notification, such as a notification such as controllable or uncontrollable, in the meta live serviced video to notify the user terminal (530) of whether control is possible based on the control-related message.
[0070] The service providing server (520) according to an embodiment of the present invention collects live images from the image providing device (510), stores and manages them in a database (525), and provides a meta live service based on the live images stored in the database (525) when a meta live service request is received from a user terminal (530) in a state where online control is impossible. At this time, the service providing server (520) can provide a meta live service including a control-related notification (i.e., a notification that online control is impossible).
[0071] Meanwhile, in an embodiment of the present invention, when the video providing device (510) provides a meta live service in an offline control state, the service providing server (520) may check whether there is a change in the background area using the live image received from the video providing device (510). That is, the service providing server (520) calculates the change between the previous live image received from the video providing device (510) and the current live image, and may change the background area in the video being served as a meta live service only when the calculated change is greater than or equal to a preset threshold value.
[0072] Here, the change of the background area by the service providing server (520) can be done by creating a background layer using the live image received from the image providing device (510) and then changing the streaming image provided as a meta live service based on this.
[0073] In addition, the service providing server (520) provides information including whether each of the plurality of video providing devices (510) can be controlled to the user terminal (530), and when an uncontrollable video providing device (510) is selected, an interface can be provided to the user terminal (530) that allows selection of either the use of collected video or the use of live video. In this case, when the use of collected video is selected, the service providing server (520) provides the meta live service to the user terminal (520) using the video stored in the database (525), and when the use of live video is selected, the meta live service can be provided based on the live video received from the video providing device (510) that is controlled offline.
[0074] For this operation, the service providing server (520) according to the embodiment of the present invention may include, as illustrated in FIG. 2, a linking unit (522) for linking with the video providing device (510) and the user terminal (530) through a network, a service providing unit (524) for providing a meta live service based on the live video received through the linking unit (522), an authority control unit (526) for controlling the control authority status of the video providing device (510), and a layer changing unit (528) for calculating the degree of change between the live videos received from the video providing device (510) to determine whether to change the background area and then changing the background based on the determination.
[0075] The service provider (524) can create a background layer using a live video to provide a meta live service, and create a video for streaming using merged data that merges the background layer, the virtual reality layer, and the augmented reality layer, and then provide a meta live service based on this.
[0076] The authority control unit (526) provides a control authority notification to notify a user terminal (520) connected to the image providing device (510) of an uncontrollable state when the image providing device (510) is in an offline control state, and can display a message on the display of the image providing device (510) to notify that it is in an online control state when the image providing device (510) is in an online control state.
[0077] Meanwhile, in an embodiment of the present invention, the service providing server (520) can create a 360-degree background image using live images received from the image providing device (510) according to weather and season, and then store and manage the same in a database (525).
[0078] Through this, when the video providing device (510) receives a connection request from the user terminal (530) through a wired or wireless network in an offline control state, the service providing server (520) provides the user terminal (530) with an interface that can set a virtual camera viewpoint, which is a direction looking at a part of a 360-degree background image, and then creates a virtual camera machine having the set virtual camera viewpoint through the interface, and a background layer is created using the created virtual camera machine, and a meta live service can be provided using merged data in which the created background layer, virtual reality layer, and augmented reality layer are merged. At this time, the service providing server (520) provides the user terminal (530) with a control interface that can control up / down / left / right and zoom movement of the virtual camera machine from the virtual camera viewpoint of the created virtual camera machine, and when a user's control request is received through the control interface, the operation of the virtual camera machine can be controlled to change the background layer.
[0079] Meanwhile, the service providing server (520) according to an embodiment of the present invention provides a user terminal (530) with an environment setting interface that can set an environment, such as weather, time zone, season, etc., and creates an environment layer according to the environment set by the environment setting interface, and can then create merged data using the environment layer and the background layer and provide a meta live service based on this.
[0080] In an embodiment of the present invention, the image providing device (510) may be a telescope capable of PTZ (Pan Tilt Zoom) control and capable of up / down / left / right movement and zoom movement through PTZ control, which will be described with reference to FIGS. 3 to 12.
[0081] FIG. 3 is a drawing showing the overall structure of an image providing device (510) having a PTZ function according to an embodiment of the present invention, FIG. 4 is a perspective view showing a coupling structure between a position conversion unit and a photographing unit of an image providing device (510) having a PTZ function according to an embodiment of the present invention, FIG. 5 is a side view of a module mounted inside a housing in an image providing device (510) having a PTZ function according to an embodiment of the present invention, FIG. 6 is a drawing showing the structure of a stand applied to an embodiment of the present invention, FIG. 7 is a drawing showing the structure of a first support bracket in an image providing device (510) having a PTZ function according to an embodiment of the present invention, FIG. 8 is a drawing for explaining the structure of a second support bracket in an image providing device (510) having a PTZ function according to an embodiment of the present invention, FIG. 9 is a drawing for explaining the mounting state of a tilting module of an image providing device (510) having a PTZ function according to an embodiment of the present invention, and FIG. 10 is a drawing for explaining components of a photographing unit of an image providing device (510) having a PTZ function according to an embodiment of the present invention, FIG. 11 is a drawing for explaining the structure of a housing of an image providing device (510) having a PTZ function according to an embodiment of the present invention, and FIG. 12 is a drawing for explaining a configuration for limiting a panning angle and a tilting angle of an image providing device (510) having a PTZ function according to an embodiment of the present invention.
[0082] As illustrated in FIGS. 3 to 12, an image providing device (510) having a PTZ (Pan Tilt Zoom) function according to an embodiment of the present invention may be composed of a photographing unit (30) built into a housing (40), a support (110) for supporting the housing (40) and coupled to the housing (40) so as to be able to move up and down, and a control unit (50) for performing overall control of the photographing unit (30) formed inside the housing (40) and the image providing device (510).
[0083] Specifically, a video providing device (510) having a PTZ function according to an embodiment of the present invention may include a housing (40) supported at a position spaced apart from the ground by a support (110), a shooting unit (30) formed inside the housing (40) and having a zoom function, a position conversion unit (20) connected to the shooting unit (30) for pan and tilt control, a touch-type display (420) formed in the housing (40) and capable of receiving user input formed on an opposite surface to a direction in which the shooting unit shoots, and a control unit (50) that controls one of the shooting unit (30) and the position conversion unit (20) to perform zoom state control, pan control, or tilt control in response to a user input being input through the touch-type display (420).
[0084] In particular, the control unit (50) may further include a communication circuit (not shown) for communication with the service providing server (520) through a network, and may transmit live images to the service providing server (520).
[0085] In addition, the control unit (50) can transmit a live image to the service providing server (520) when the control is completed during pan, tilt, and zoom control and when the control is stopped for a preset period of time.
[0086] In addition, the control unit (50) can operate in an online control state in which up / down, left / right, and zoom control are performed by the service providing server (520) based on a control request from the user terminal (530), or in an offline control state in which the control is performed by the user at the installed location. Specifically, the control unit (50) can control the up / down movement and zoom movement of the shooting unit (30) through control of the position conversion unit (20) and the shooting unit (30) based on a user's input in the offline control state, and can control the up / down movement and zoom movement of the shooting unit (30) through control of the position conversion unit (20) and the shooting unit (30) based on a control request from the service providing server (520) in the online control state.
[0087] In addition, the control unit (50) performs pan, tilt, and zoom control according to a control request received from the service provision server (520) when in an online control state, and in this case, a message indicating that it is in an online control state can be displayed on the touch-type display (420).
[0088] The position conversion unit (20) according to an embodiment of the present invention may include a pan module (210) composed of a first step motor (212) that pans the photographing unit (30) in the left and right directions based on a control signal received from the control unit (50) according to a user input, and a tilting module (240) composed of a second step motor (242) that tilts the photographing unit (30) in the up and down directions based on a control signal received from the control unit (50) according to a user input.
[0089] Meanwhile, the image providing device (510) according to an embodiment of the present invention may include a mounting groove (122) into which a fan module (210) is inserted to support the rotation of the fan module (210), a first support bracket (220) that is mounted on the top of the fan module (210) and is rotated by the fan module (210), and includes a first horizontal fixing member (222) and a first vertical fixing member (224) formed at both ends of the first horizontal fixing member (222), a second support bracket (230) that is connected to the first vertical fixing member (224) at both ends and on which a photographing unit (30) is mounted so as to be tiltable, and to which a tilting module (240) is connected, etc.
[0090] The detailed configuration of the video providing device (510) having a PTZ function with the configuration described above is described below.
[0091] First, the shooting unit (30) of the image providing device (510) is supported by a support (10) so as to be positioned at a certain height from the ground, and the support (10) can be connected to a support (110) that supports the unit so as to be positioned at a certain height from the ground.
[0092] The support (110) can be configured as a fixed type in which the installation position is fixed, or as an elevating type in which the height is changed according to a control signal from the control unit (50).
[0093] When the support (110) is an elevating type, the support (110) may be composed of a fixed base (111) fixed to the ground, a lift platform (112) extended from the fixed base (111) but configured to be lifted based on the fixed base (111), and a lifting unit (113) in which the lift platform (112) provides a lifting force to perform a lifting action.
[0094] At this time, the elevator (113) may be configured with a rack and pinion method, a hydraulic or pneumatic cylinder method, etc., and any method that can provide lifting force to the elevator may be adopted.
[0095] In addition, a support (120) is formed on the top of the support (110) to form a flat surface to facilitate the placement of the position change unit (20).
[0096] In addition, a mounting groove (122) is formed at a certain depth on the base (120) to support the rotation of the position changing part (20) as a lower part of the position changing part (20) is mounted thereon.
[0097] In addition, the base (120) is configured to have a first through hole (124) that penetrates the center of the mounting groove (122) to a certain size so that an electrical connection with the control unit (50) can be made and power can be supplied, through which wires and the like can enter and exit.
[0098] Additionally, a stopper (126) is provided on the pedestal (120) to mechanically limit the rotation angle of the position changer (20).
[0099] The stopper (126) is configured to be erected at a certain height on the stand (120) so that the rotation angle of the position changer (20) cannot exceed, for example, the range of 120° to 180°. In this case, the stopper (126) is configured in the range of 60° to 90° on each side with the fixing groove (122) as the center.
[0100] In addition, at least one first fastening hole (128) is formed on the base (120) to support the housing (40) so that it is fixed by fastening means, such as a bolt and nut, or a fastening bolt.
[0101] Additionally, at least one second fastening hole (129) is formed on the base (120) so that the first step motor configured in the position change unit (20) can be fixed by a fastening means.
[0102] The position change unit (20) is configured on the upper part of the support (10) to pan or tilt the shooting unit (30) to maximize viewing efficiency.
[0103] The position change unit (20) includes a fan module (210), a first support bracket (220), a second support bracket (230), and a tilting module (240).
[0104] The fan module (210) rotates the shooting unit (30) at a certain angle in the left and right directions.
[0105] The fan module (210) is configured with a first step motor (212) that generates forward or reverse rotation power according to a control signal provided from the control unit (50). In this case, the first step motor (212) is configured on one side of the stand (120), and the first step motor (212) is fixed to the stand (120) as the fastening means is fastened to the second fastening hole (129).
[0106] In addition, the first step motor (212) is provided with a first driving gear (214) that is fixed to an extension shaft and rotates by forward or reverse rotation power generated from the first step motor (212).
[0107] In addition, the fan module (210) is configured at the top of the pedestal and is configured with a first driven gear (216) that receives forward or reverse rotational power provided from the first driving gear (214) and rotates at a certain angle.
[0108] In this case, an idle gear (215) is configured between the first driving gear (214) and the first driven gear (216) to support the first driven gear (216) so that it can rotate at a low speed by the gear ratio, and to protect the first driven gear (216) by preventing the rotational force of the first stepper motor (212) from being directly transmitted to the first driven gear (216).
[0109] That is, the idle gear (215) is configured between the first prime mover gear (214) and the first driven gear (216), so that the rotational force provided through the first prime mover gear (214) is not directly transmitted to the first driven gear (216), thereby protecting the first driven gear (216). Of course, it is not limited to this, and the idle gear (215) may be omitted, and the first driven gear (216) may be configured to be directly engaged with the first prime mover gear (214).
[0110] In addition, the first driven gear (216) is provided with a mounting boss (218) that is inserted into the mounting groove (122) of the support (120) to support the forward or reverse rotation of the first driven gear (216). In this case, the mounting boss (218) is formed to protrude from the lower end of the first driven gear (216) at a certain height.
[0111] In addition, a second through hole (219) is formed in the center of the first driven gear (216) to penetrate with a certain size so that an electrical connection line or a power supply wire provided through the control unit (50) can enter and exit. In this case, the second through hole (219) is configured to penetrate the first driven gear (216) and the mounting boss (218).
[0112] In addition, the first driven gear (216) is configured to protrude from the outer periphery by a certain length and is provided with at least one stop pin (216a) that supports the rotation angle by a stopper (126) configured on the support (120).
[0113] A first support bracket (220) can be configured to be mounted on the top of the fan module (210) in the position change unit (20).
[0114] The first support bracket (220) is mounted on the top of the fan module (210) and supports the second support bracket (230) while rotating along with the rotation of the fan module (210).
[0115] The first support bracket (220) is formed in a flat shape, and a first horizontal fixing member (222) that is mounted on the top of the first driven gear (216) can be formed.
[0116] In addition, a third through hole (2222) is formed in the center of the first horizontal fixing member (222) so that an electrical connection line or a power supply wire provided through the control unit (50) can enter and exit.
[0117] In addition, the first horizontal fixing member (222) is configured with at least one third fastening hole (2224) that supports the first driven gear (216) to rotate together with the first driven gear (216) by maintaining a state of being firmly fixed to the first driven gear (216) by a fastening means.
[0118] In addition, a first vertical fixing member (224) may be configured to be erected at a certain height at each end of the first horizontal fixing member (222) to support the second bracket (230) and the tilting module (240) so that they can be stably mounted.
[0119] The first vertical fixing member (224) is configured in multiple numbers to be erected at a certain height at each end of the first horizontal fixing member (222), thereby supporting the second support bracket (230) so that it is positioned at a certain height, thereby enabling the second support bracket (230) to rotate, i.e. tilt, smoothly within a certain range in the upper and lower directions.
[0120]
[0121] In addition, insertion holes (2242) are formed in the plurality of first vertical fixing members (224) to support smooth rotation of the second support bracket (230) by inserting each end of the second support bracket (230) into them.
[0122] Insertion holes (2242) are formed in each of the plurality of first vertical fixing members (224), so that both ends of the second support bracket (230) are inserted therein, thereby allowing the second support bracket (230) to rotate up and down at a certain angle around the insertion holes (2242). In this case, the insertion holes (2242) may be formed to penetrate the first vertical support member (224), or may be formed to a certain depth, and any structure or shape that allows the second support bracket (230) to rotate smoothly up and down may be adopted.
[0123] In addition, at least one fourth fastening hole (2244) is formed on one of the plurality of first vertical fixing members (224) so that the second step motor of the tilting module (240) can be fastened by a fastening means. In this case, the fourth fastening hole (2244) is formed below the insertion hole (2242).
[0124] In addition, a spacing member (226) is configured between the plurality of first vertical fixing members (224) to support and maintain the spacing between the first vertical fixing members (224). In this case, the spacing member (226) is configured at the upper end of the plurality of first vertical fixing members (224). Of course, it is not limited thereto, and at least one spacing member (226) may be configured at various locations depending on the configuration of the second support bracket (230) and the tilting module (240), and in some cases, the spacing member (226) may be omitted.
[0125] Meanwhile, the first support bracket (220) is configured by assembling the first horizontal fixing member (222), the first vertical fixing member (224), and the spacing member (226), thereby enabling smooth assembly of the second support bracket (230) and the tilting module (240).
[0126] The second support bracket (230) is configured to be supported by the first support bracket (220) and is rotated upward and downward by a certain angle by the tilting module (240).
[0127] The second support bracket (230) is formed flat, and a second horizontal fixing member (232) that supports the upper portion of the shooting unit (30) can be formed.
[0128] A fourth through hole (2322) is formed in the second horizontal fixed member (232) to allow electrical connection lines or power supply lines provided through the control unit (50) to enter and exit through a hole of a certain size.
[0129] Additionally, a plurality of second vertical fixing members (234) configured to be erected at a certain height at each end of the second horizontal fixing member (232) may be configured.
[0130] A plurality of second vertical fixing members (234) are configured to protrude to a certain length and are inserted into an insertion hole (2242) formed in the first vertical fixing member (224), thereby forming an insertion shaft (2342) that supports rotation at a certain angle in the upper and lower directions based on the insertion hole (2242).
[0131] The insertion shaft (2342) is formed to protrude from each of the plurality of second vertical fixing members (234) by a certain length, and is inserted into the insertion hole (2242) of the first vertical fixing member (224), thereby allowing the second support bracket (230) to smoothly rotate up and down at a certain angle with respect to the first support bracket (220). In this case, a bearing or bushing, etc. that supports the smooth rotation of the second support bracket (230) may be configured in the insertion shaft (2342), or a stop ring, etc. may be configured in the insertion hole (2242) to prevent the insertion shaft (2342) from being accidentally detached.
[0132] In addition, at least one fifth fastening hole (2344) is formed in one of the plurality of second vertical fixing members (234) to support the second driven gear of the tilting module (240) so that it can be fixed by a fastening means. In this case, the fifth fastening hole (2344) is formed in the position where the tilting module (240) is formed, that is, in the second vertical fixing member (234) adjacent to the first vertical fixing member (224) in which the fourth fastening hole (2244) is formed among the plurality of first vertical fixing members (224).
[0133] The tilting module (240) is configured on one side of the second support bracket (230) and rotates the second support bracket (230) upward or downward by a certain angle according to a control signal from the control unit (50).
[0134] The tilting module (240) can be configured with a second step motor (242) that generates a forward rotation force or a reverse rotation force according to a control signal of the control unit (50). In this case, the second step motor (242) is fixed to the first vertical fixing member (224) having a fourth fastening hole (2244) formed among the plurality of first vertical fixing members (224) by a fastening means.
[0135] In addition, a second driving gear (244) that is fixed to the extension shaft of the second stepper motor (242) and rotates forward and backward by the forward or reverse rotation power generated from the second stepper motor (242) may be configured. In this case, the second driving gear (244) is configured between the first and second vertical fixing members (224)(234).
[0136] In addition, the tilting module (240) may be configured with a second driven gear (246) that is fixed to a second vertical fixed member (234) having a fifth fastening hole (2344) formed among a plurality of second vertical fixed members (234) by a fastening means. In this case, the second driven gear (246) is configured between the first and second vertical fixed members (224) (234), and is configured to mesh with the second driving gear (244), thereby receiving forward or reverse rotation of the second driving gear (244) so that the second support bracket (230) can rotate upward and downward by a certain angle.
[0137] The shooting unit (30) is configured to be mounted on the second support bracket (230) and moves left and right or up and down by the action of the fan module (210) or tilting module (240) of the position change unit (20) to shoot the view.
[0138] The shooting unit (30) includes a camera (310), a lens (320), an adapter (330), and a fixing bracket (340).
[0139] The camera (310) can capture a view and obtain an image according to a control signal from the control unit (50). The obtained image can be input to the control unit (50) and displayed on the display (430).
[0140] It is more preferable that the camera (310) according to the embodiment of the present invention is an industrial camera that is easy to control by the control unit (50) and can easily obtain high resolution and clear images.
[0141] The lens (320) is configured on one end of the camera (310) and has a zoom in / out function, thereby facilitating the acquisition of images of distant views. In this case, the lens (320) is preferably a digital lens capable of controlling zoom in / out according to a control signal from the control unit (50). For example, the lens (320) is a lens for a DSLR.
[0142] An adapter (330) is configured between the camera (310) and the lens (320) to support easy mounting of the camera (310) and the lens (320).
[0143] The fixed bracket (340) supports the second support bracket (230) so that the photographing unit (30) can be stably fixed by a fastening means. For example, the fixed bracket (340) is configured on one side of the lens (320) and is fixed to the second horizontal fixing member (232) of the second support bracket (230) by a fastening means.
[0144] The housing (40) is configured on the support (10) and performs the function of protecting the position change unit (20) and the photographing unit (30) from being exposed to the outside, and maximizes viewing efficiency by allowing the user to easily check the snapshots taken by the photographing unit (30) with the naked eye from the outside.
[0145] The housing (40) may include a cover (410), a display (420), and a protective window (430).
[0146] The cover (410) is formed so that the position change unit (20) and the photographing unit (30) are positioned on the inside, but the top, bottom, left, and right sides are blocked to prevent the position change unit (20) and the photographing unit (30) from being exposed to the outside, and each end is opened. In this case, the lower end of the cover (410) is fixed by being fastened to the first fastening hole (128) of the support plate (120) by a fastening means.
[0147] The display (420) is configured on one end of the cover (410) to display images captured by the camera unit (30) in real time, thereby providing the effect of viewing the surrounding scenery. In this case, the display (420) is a display made of LED, OLED, or plasma capable of displaying images. Preferably, it is a touch screen that allows the user to adjust the viewing position direction or zoom in / out while viewing the screen.
[0148] That is, the display (420) can be used as an input interface for pan and tilt control of the shooting unit (30) through the position conversion unit (20) connected to the shooting unit (30).
[0149] Additionally, the display (420) can be used as an input interface for zoom-in / zoom-out control of the lens (320) of the shooting unit (30).
[0150] In an embodiment of the present invention, the input interface may mean a touch operation in the up, down, left, and right directions on the display (420), or a touch operation that reduces or increases the distance between two specific points.
[0151] User input according to such touch operation is input to the control unit (50), and the control unit (50) can control the position conversion unit (20) or the lens (320) according to the user input.
[0152] The protective window (430) is configured at the other end of the cover (410), i.e., in the direction corresponding to the display (420), to protect the photographing unit (30) and support the photographing unit (30) so that the photographing of the view can be performed smoothly. In this case, the protective window (430) is formed of transparent tempered glass.
[0153] Meanwhile, the display (420) and the protective window (430) are configured to be sealed against the cover (410), thereby preventing foreign substances from penetrating into the position changing unit (20) and the photographing unit (30) configured inside the housing (40), thereby protecting the position changing unit (20) and the photographing unit (30) and preventing malfunction due to foreign substances.
[0154] The control unit (50) transmits a control signal according to the user's direction control or zoom in / out control through the display (420) to the position change unit (20) or the shooting unit (30), thereby obtaining an image desired by the user and controlling the obtained image to be displayed on the display (420).
[0155] In addition, the control unit (50) controls the support (10) so that it can be raised and lowered according to the user's physical condition when the support (110) is configured to be liftable, thereby ensuring convenience of use.
[0156] In the case of an observation camera having a PTZ function as described above, the rotation angle of the position change unit (20) can be mechanically limited by using a stopper (126) as an example, but as shown in FIG. 12, not only can the rotation angle of the position change unit (20) be limited by using the first sensor (500), but also the tilting angle of the tilting module (240) can be limited by using the second sensor (502).
[0157] That is, the first sensor (500) is connected to the pan module (210) and senses the panning angle (rotation angle) as the camera unit (30) pans left and right, and the second sensor (502) is connected to the tilting module (240) and senses the tilting angle as the camera unit (30) tilts up and down. The panning angle and tilting angle sensed in this way can be provided to the control unit (50).
[0158] Accordingly, the control unit (50) can control the operation of the first and second step motors (212, 242) based on the panning angle and the tilting angle. That is, the control unit (50) can control the first and second step motors (212, 242) so that the shooting unit (30) is not panned or tilted beyond a certain angle.
[0159] Below, the process of providing a meta live service will be described with reference to FIGS. 13 to 15.
[0160] Figure 13 is a flowchart illustrating a process in which an image providing device according to an embodiment of the present invention transmits an image.
[0161] As illustrated in FIG. 13, when the image providing device (510) receives a meta live service request from a user terminal (530) through the service providing server (520), the image providing device (510) transmits a live image captured by the shooting unit (30) to the service providing server (520) (S600).
[0162] Then, when user control occurs offline (S602), that is, when a user input is generated for any one of up / down / left / right movement and zoom movement, the control unit (50) of the image providing device (510) controls the shooting unit (30) according to the user input.
[0163] After the control of the shooting unit (30) is completed and a preset threshold time has elapsed (S604), the control unit (50) of the image providing device (510) shoots an image through the operation of the shooting unit (30) and transmits a live image (S606).
[0164] Figure 14 is a flowchart illustrating a process in which a service providing server according to an embodiment of the present invention provides a meta live service.
[0165] As illustrated in FIG. 14, the service providing server (520) extracts guide points through analysis of the first live image as the first live image is received from the image providing device (510) (S610, S612).
[0166] Then, the service providing server (520) creates a background layer using the first live image, and also creates a virtual reality layer including the avatar and ground selected by the user and an augmented reality layer matching the content to be provided at the guide point (S614), and then creates merged data by merging the background layer, the virtual reality layer, and the augmented reality layer, and then provides the meta live service to the user terminal (520) through this (S616).
[0167] Thereafter, when a second live image is received from the image providing device (510) (S618), the service providing server (520) determines whether a change in the background area is necessary through a comparison between the first live image and the second live image (S620). Specifically, the service providing server (520) calculates a degree of change through a comparison between the first live image and the second live image, and can determine whether a change in the background area is necessary by checking whether the degree of change is outside a preset threshold range.
[0168] As a result of the judgment of S620, if a change in the background area is required, the service providing server (520) creates a background layer using the second live image, extracts guide points within the second live image, and then regenerates merged data by merging the augmented reality layer, background layer, and virtual reality layer to which the content to be provided is matched through the guide points, and then changes the image being served as a meta live service based on this (S622).
[0169] Although omitted in the embodiment of the present invention as described above, the layer for virtual reality may be changed and applied to suit the weather and season of the location where the video providing device (510) transmitting the live video is installed or the weather and season set by the user.
[0170] Additionally, the second live image may be received in a process in which the image providing device (510) is controlled in an offline control state as illustrated in FIG. 13, or may be received when the image providing device (510) is controlled according to a control request from a user terminal (530).
[0171] FIG. 15 is a flowchart illustrating a process for providing a meta live service when an image providing device according to an embodiment of the present invention is unable to be controlled online.
[0172] As illustrated in FIG. 15, the service providing server (520) provides a configuration image for setting a virtual camera viewpoint to the user terminal (530) in response to a meta live service request (S630). Here, the virtual camera viewpoint may refer to the direction viewed by the virtual camera machine generated by the service providing server (520), and the configuration image may be generated based on an image extracted from a specific portion of the 360-degree image generated by the service providing server (520).
[0173] Afterwards, when a virtual camera viewpoint is set on the user terminal (530), for example, a predetermined image is selected from the setting video, and the service providing server (520) creates a virtual camera machine having a virtual camera viewpoint looking at the selected image (S632).
[0174] Then, the service providing server (520) creates a background layer using an image corresponding to the virtual camera viewpoint (S634), and then provides the meta live service to the user terminal (530) using merged data that merges the background layer, the virtual reality layer, and the augmented reality layer (S636). At this time, the service providing server (520) may provide the user terminal (530) with a control interface that enables pan, tilt, and zoom control of the virtual camera machine.
[0175] Thereafter, when a control request for one of the pan, tilt, and zoom controls of the virtual camera machine is received (S638) through the control interface, the service providing server (520) controls the virtual camera machine according to the control request (S640), thereby changing the virtual camera viewpoint of the virtual camera machine.
[0176] Then, the service providing server (520) creates a background layer using the image according to the changed virtual camera viewpoint, that is, the image seen by the virtual camera machine according to the up / down or zoom movement (S642), and then regenerates merged data by merging the background layer, the virtual reality layer, and the augmented reality layer, and then changes the image served to the user terminal (530) as Meta Live based on this (S644).
[0177] The above description is merely one embodiment for implementation, and the present invention is not limited to the above-described embodiment. Those skilled in the art will understand that various modifications and implementations are possible without departing from the spirit and scope of the present invention.
[0178] [Explanation of symbols]
[0179] 10: Support
[0180] 20: Position conversion section
[0181] 30: Filming Department
[0182] 40: Housing
[0183] 50: Control unit
[0184] 510: Video providing device
[0185] 520: Service Provider Server
[0186] 525: Database
[0187] 530: User-side terminal
Claims
1. A method for providing a meta live service using an image providing device that is installed in a predetermined location and displays a live image in a viewing direction and can move up, down, left, right, and zoom based on the installed location, and changes the live image according to the up, down, left, right, and zoom movement and displays it on the display, and is equipped to transmit the live image through a network and includes a telescope that can move up, down, left, right, and zoom through PTZ (Pan Tilt Zoom) control, The above method, A step in which the above video providing device and the service providing server connected through a network receive a meta live service request from a user terminal connected through the network; A step in which the server receives a live video from the video providing device according to the meta live service request, A method for providing a meta live service, characterized in that the server comprises a step of providing a meta live service to a user terminal connected through the network using merged data in which the live video and the virtual reality layer are merged.
2. In paragraph 1, The above meta live service provision method is: A step of providing an environment setting interface for setting an environment to the user terminal, wherein the server can set an environment when the meta live service request is received from the user terminal, The steps for providing the above meta live service are: A method for providing a meta live service, characterized in that the server receives the live video from the video providing device, creates an environment layer according to the environment set by the environment setting interface, creates the merged data using the environment layer and the background layer, and then provides a meta live service based on the merged data.
3. In paragraph 1, The step in which the above server provides the above meta live service is: A step of creating a layer for virtual reality including the shape of the avatar and ground selected by the user received from the user terminal, A step of generating merged data by merging a background layer corresponding to the live image and an augmented reality layer for providing multiple contents based on the live image into the virtual reality layer, A method for providing a meta live service, characterized by including a step of providing a meta live service using the above merged data.
4. In paragraph 1, The method of providing the meta live service by the above server is as follows: A step of receiving a live video in real time from the above video providing device, Comprising a step of calculating the change between the received live images, The steps for providing the above meta live service are: A method for providing a meta live service, characterized in that it is determined whether to change the background layer in a video corresponding to the meta live service streamed to the user terminal based on the above change rate.
5. A video providing device that is installed in a designated location and displays a live image in the direction of view, and can move up, down, left, right, and zoom based on the installed location, and changes the live image according to the up, down, left, right, and zoom movement and displays it on a touch-type display, and transmits the live image through a network; A service providing server is connected to the above video providing device via a network, receives the above live video, and provides a meta live service to a user terminal connected via the network using merged data in which the above live video and a virtual reality layer are merged. The above video providing device is a telescope that can be controlled by PTZ (Pan Tilt Zoom) and can move up, down, left, right and zoom through PTZ control. A housing supported at a position spaced a predetermined distance from the ground by a support, A photographing unit formed inside the housing and having a zoom function, A position conversion unit for pan and tilt control connected to the above shooting unit, The touch-type display formed in the housing, but capable of user input, formed on the opposite side of the direction in which the photographing unit photographs, A position conversion unit including a pan module composed of a first step motor that pans the shooting unit in the left and right directions based on a control signal applied according to the user input, and a tilting module composed of a second step motor that tilts the shooting unit in the up and down directions based on a control signal applied according to the user input; A control unit that performs zoom state control, pan control, or tilt control by applying a control signal to one of the shooting unit and the position conversion unit as the user input is input through the touch-type display, A mounting groove into which the fan module is inserted to support the rotation of the fan module, A first support bracket that is mounted on the top of the fan module and rotated by the fan module, and includes a first horizontal fixing member and a first vertical fixing member formed at both ends of the first horizontal fixing member; A meta live service providing system characterized in that the two ends are connected to each of the first vertical fixing members, the shooting unit is fixed so as to be tiltable, and the tilting module is connected to a second support bracket.
Citation Information
Patent Citations
3D street view system using identification information.
KR1020120076175A
System and method for reality-linked crop cultivation game service based on image streaming
KR1020140072454A
Method, storage medium and electronic device for displaying images
KR1020180028796A
Zero diffusion break
KR1020230073973A
Method for mapping ar-based content and system using the same
KR102300285B1
