A panoramic live broadcast method and device

Abstract

The present invention relates to the field of video live broadcast technology, and in particular to a panoramic live broadcast method and device, wherein the method comprises obtaining angular position information and displacement information of a virtual line of sight of a user device; calculating coordinate variable data according to the real-time coordinate data and the historical coordinate data, and sending the coordinate variable data to a first image acquisition end; obtaining one of the images in the first image acquisition end that matches the stereoscopic orientation data, intercepting a corresponding display image according to the angle data and sending it to the user device. The present invention abandons the original idea of ​​setting up a virtual 3D environment, and does not need to project the real-time collected images to form a virtual live broadcast environment. By shooting the live broadcaster's movements on the screen and matting the images in real time, the live broadcaster can be given a live broadcast effect in a real environment.

Description

Technical Field

[0001] The present invention relates to the field of video live broadcast technology, and in particular to a panoramic live broadcast method and device. Background Art

[0002] At present, live video broadcasting is becoming more and more popular among the public, involving many fields. Especially with the support of 5G communication technology, the bandwidth of live broadcasting by streaming media technology has been enhanced, and the flat live broadcasting method has gradually developed towards the panoramic live broadcasting method.

[0003] Panoramic video uses computer technology and image processing as the core, synchronously obtains video images with different positions and angles and partial overlapping areas from multiple video image acquisition devices, and obtains panoramic video images through technologies such as registration and fusion.

[0004] However, current panoramic live broadcasts use fixed-point shooting, which cannot achieve real-time moving and following live broadcasts. Summary of the invention

[0005] In view of the above problems, embodiments of the present invention are proposed to provide a panoramic live broadcast method and device that overcome the above problems or at least partially solve the above problems.

[0006] The present invention provides a panoramic live broadcast method, comprising:

[0007] Acquire angular position information and displacement information of a virtual line of sight of a user device, wherein the angular position information includes three-dimensional orientation data and angle data of the virtual line of sight, and the displacement information includes coordinate data of the user device;

[0008] Calculate coordinate variable data according to the real-time coordinate data and the historical coordinate data, and send the coordinate variable data to the first image acquisition end;

[0009] Acquire one of the images in the first image acquisition end that matches the three-dimensional orientation data, intercept a corresponding display image according to the angle data, and send the image to the user device.

[0010] Furthermore, the calculating coordinate variable data according to the real-time coordinate data and the historical coordinate data, and sending the coordinate variable data to the first image acquisition end, includes:

[0011] Acquire coordinate data corresponding to the same time node as the current frame image acquired by the first image acquisition end to obtain real-time coordinate data;

[0012] Acquire coordinate data corresponding to the same time node as the previous frame image acquired by the first image acquisition end to obtain historical coordinate data;

[0013] The difference between the real-time coordinate data and the historical coordinate data is calculated to obtain the coordinate variable data.

[0014] Further, the acquiring of one of the images matching the stereoscopic orientation data in the first image acquisition end, intercepting a corresponding display image according to the angle data and sending it to the user device includes:

[0015] Acquire a mapping relationship between each stereoscopic orientation in the pre-established stereoscopic orientation data and each channel of images from the first image acquisition end;

[0016] Determine one of the images to be acquired through the mapping relationship;

[0017] The viewing angle information of the user device is obtained, the image display size of the user device is determined, and the corresponding display image is intercepted according to the angle value in the angle data.

[0018] Another method of panoramic live broadcast is also provided, including:

[0019] Acquire the curtain image data collected by the second image collection end;

[0020] According to the azimuth angle data of the second image acquisition end, acquiring one of the images of the first image acquisition end that matches the azimuth angle data;

[0021] One of the acquired images is superimposed with the curtain background in the curtain image by using the matting technology to obtain a live image and send it.

[0022] Further, the step of acquiring one of the images matching the azimuth angle data at the first image acquisition end according to the azimuth angle data at the second image acquisition end includes:

[0023] Constructing a correspondence between a preset curtain environment and each channel of images from the first image acquisition end;

[0024] Acquiring azimuth angle data of the second image acquisition device in the curtain environment;

[0025] According to the corresponding relationship and the azimuth angle data, one of the matching images in the first image acquisition end is obtained.

[0026] A panoramic live broadcast device is also provided, comprising:

[0027] A virtual sightline acquisition module, used to acquire angular position information and displacement information of a virtual sightline of a user device, wherein the angular position information includes three-dimensional orientation data and angle data of the virtual sightline, and the displacement information includes coordinate data of the user device;

[0028] A calculation module, used for calculating coordinate variable data according to the real-time coordinate data and the historical coordinate data, and sending the coordinate variable data to the first image acquisition end;

[0029] An image capture module, used to obtain one of the images matching the stereoscopic orientation data in the first image acquisition end, capture a corresponding display image according to the angle data, and send it to the user device;

[0030] A curtain image acquisition module, used to acquire curtain image data acquired by the second image acquisition end;

[0031] An environmental image acquisition module, configured to acquire, according to the azimuth angle data of the second image acquisition end, one of the images of the first image acquisition end that matches the azimuth angle data;

[0032] The matting module is used to overlap one of the acquired images with the curtain background in the curtain image through the matting technology to obtain the live broadcast image and send it.

[0033] Furthermore, the calculation module includes:

[0034] A first acquisition submodule is used to acquire coordinate data corresponding to the same time node as the current frame image acquired by the first image acquisition end, so as to obtain real-time coordinate data;

[0035] A second acquisition submodule is used to acquire coordinate data corresponding to the same time node as the previous frame image acquired by the first image acquisition end, so as to obtain historical coordinate data;

[0036] The calculation submodule is used to calculate the difference between the real-time coordinate data and the historical coordinate data to obtain the coordinate variable data.

[0037] Furthermore, the image capture module includes:

[0038] A relationship submodule, used for obtaining a mapping relationship between each stereoscopic orientation in the pre-established stereoscopic orientation data and each channel of images of the first image acquisition end;

[0039] A determination submodule, used for determining one of the images to be acquired according to the mapping relationship;

[0040] The interception submodule is used to obtain the viewing angle information of the user device, determine the image display size of the user device, and intercept the corresponding display image according to the angle value in the angle data.

[0041] The embodiments of the present invention include the following advantages:

[0042] The present invention forms a virtual image by collecting images in a real environment in real time through a first image collection end, replaces the user's perspective in the virtual image through a user device, collects the user's live broadcast screen in real time through a second image collection end, and processes the virtual image and the screen image with the help of matting technology, so that the user can live broadcast immersively, and also brings the audience the playback effect of the live broadcaster in a real environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flow chart of the steps of a panoramic live broadcast method of the present invention.

[0044] Figure 2 It is a step flow chart of another panoramic live broadcast method of the present invention.

[0045] Figure 3 It is a structural block diagram of a panoramic live broadcast device of the present invention.

[0046] Figure 4 It is a structural block diagram of a computer device for panoramic live broadcasting of the present invention. DETAILED DESCRIPTION

[0047] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0048] The executor of the panoramic live broadcast method provided in the embodiment of the present application is generally a computer device with certain computing capabilities. The computer device includes, for example: a terminal device or a server or other processing device. The terminal device can be a user device, a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital processing, a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc.

[0049] In the present invention, the live broadcaster wears a user device and stands in the background curtain, for example, wears AR glasses and stands in the green screen, and the drone collects the real environment image in real time from the outside. With the advantages of "large bandwidth and low latency" of the 5G communication network, the collected image is quickly transmitted to the cloud server in real time, and then the image is projected onto the AR glasses in real time, giving the live broadcaster an immersive live broadcast experience. At the same time, a mobile photography device is set up in the environment of the background curtain to shoot the live broadcaster's live broadcast action to form a live broadcast picture. With the help of current real-time matting technology, the curtain background in the live broadcast picture is replaced with the image collected by the drone.

[0050] The present invention abandons the original idea of ​​setting up a virtual 3D environment. It does not need to project the real-time collected images to form a virtual live broadcast environment. By shooting the live broadcaster's movements on the screen and cutting out the images in real time, it provides the audience with the broadcast effect of the live broadcaster in a real environment.

[0051] The following describes a method for panoramic live broadcast disclosed in this application by taking a computer device as an example of an execution subject.

[0052] From the perspective of the live streamer, Figure 1 As shown, the method comprises the following steps:

[0053] S110, obtaining angular position information and displacement information of a virtual line of sight of a user device, wherein the angular position information includes three-dimensional orientation data and angle data of the virtual line of sight, and the displacement information includes coordinate data of the user device;

[0054] It should be noted that the virtual line of sight of the user device can be understood as the line of sight of the user (live broadcaster) in the virtual environment, which includes the three-dimensional orientation data and angle data of the user's line of sight, that is, the pitch, elevation, left and right deflection and angle of the line of sight when the user remains motionless. The image seen by the user is different due to different angular information; it also includes the coordinate data of the user's line of sight, which can be understood as the angular position of the user's line of sight remains unchanged while the body moves, that is, the line of sight remains unchanged, but the image is pulled closer or farther away; or, the user's angular position information and displacement information change at the same time.

[0055] S120, calculating coordinate variable data according to the real-time coordinate data and the historical coordinate data, and sending the coordinate variable data to the first image acquisition end;

[0056] It should be noted that the first image acquisition end can be a drone device equipped with N cameras, which can synchronously capture N video images in real time in a real environment and send them to a cloud server via 5G communication. When the first image acquisition end receives the coordinate variable data, it moves accordingly according to the coordinate parameters in the coordinate variable data, and captures new video images after the movement and transmits them.

[0057] S130, obtaining one of the images matching the stereoscopic orientation data in the first image acquisition end, intercepting a corresponding display image according to the angle data, and sending it to the user device.

[0058] It should be noted that according to the stereoscopic orientation data of the virtual line of sight of the user device, for example, the virtual line of sight is forward and looking up 50°, then an image corresponding to the front is obtained in the first image acquisition end, the center line of the image is offset to looking up 50°, and then the display image matching the display window of the user device is intercepted and sent to the user device.

[0059] It can be understood that through the above implementation scheme, the user can see the real environment in the distance by wearing the user device, and the visual picture changes with the user's displacement and line of sight, giving the user an immersive experience.

[0060] In this embodiment, step S120, calculating coordinate variable data according to the real-time coordinate data and the historical coordinate data, and sending the coordinate variable data to the first image acquisition end, includes:

[0061] Acquire coordinate data corresponding to the same time node as the current frame image acquired by the first image acquisition end to obtain real-time coordinate data;

[0062] Acquire coordinate data corresponding to the same time node as the previous frame image acquired by the first image acquisition end to obtain historical coordinate data;

[0063] Calculate the difference between the real-time coordinate data and the historical coordinate data to obtain the coordinate variable data

[0064] It should be noted that the user device moves according to the user's body position, and the user actually moves in a plane, and the coordinate data accompanying the user device is only a change in the plane. In order to reflect the diverse changes of coordinate variables in space, the user device can also include a device that receives the rise and fall instructions issued by the user, such as a button that can control the rise and fall.

[0065] In this embodiment, step S130, the acquiring of one of the images matching the stereoscopic orientation data in the first image acquisition end, intercepting a corresponding display image according to the angle data and sending it to the user device, includes:

[0066] Acquire a mapping relationship between each stereoscopic orientation in the pre-established stereoscopic orientation data and each channel of images from the first image acquisition end;

[0067] Determine one of the images to be acquired through the mapping relationship;

[0068] The viewing angle information of the user device is obtained, the image display size of the user device is determined, and the corresponding display image is intercepted according to the angle value in the angle data.

[0069] It should be noted that each three-dimensional orientation of the virtual sight line of the user device corresponds to an image acquisition device, that is, one of the images of the first image acquisition end. Generally, the field of view angle of each image acquisition device is greater than the field of view angle of the user device, that is, the image from the user's perspective is not the entire image of one of the images collected.

[0070] From the audience's perspective, Figure 2 As shown, the method comprises the following steps:

[0071] S210, acquiring curtain image data acquired by the second image acquisition end;

[0072] It should be noted that the second image acquisition end can be a camera device that can be moved in the background curtain, or it can be a camera device for multi-channel fixed-point shooting. The images collected by the second image acquisition end are processed and displayed to the audience. The background curtain is a green screen or a blue screen. The live broadcast picture of the user (live broadcaster) in the curtain environment is photographed to form curtain image data.

[0073] S220, acquiring, according to the azimuth angle data of the second image acquisition end, one of the images of the first image acquisition end that matches the azimuth angle data;

[0074] It should be noted that the curtain environment in the present invention can be a sphere with a blue or green background color. The user stands inside the sphere to broadcast live. The first image acquisition end changes according to the change of the virtual line of sight of the user's device. The collected image forms the virtual live broadcast environment where the user is located. The second image acquisition end replaces the audience's perspective to collect the user's live broadcast status in the virtual live broadcast environment.

[0075] S230, superimposing one of the acquired images with the curtain background in the curtain image by using a matting technology to obtain a live broadcast image and send it.

[0076] The above-mentioned matting technology is used to superimpose the real environment image captured by the first image capture end in the distance on the screen environment image where the user is located, forming a live image of the user in the real environment from the audience's perspective, and the live image is sent to the audience client for playback.

[0077] In this embodiment, step S220, obtaining, according to the azimuth angle data of the second image acquisition end, one of the images of the first image acquisition end that matches the azimuth angle data, includes:

[0078] Constructing a correspondence between a preset curtain environment and each channel of images from the first image acquisition end;

[0079] Acquiring azimuth angle data of the second image acquisition device in the curtain environment;

[0080] According to the corresponding relationship and the azimuth angle data, one of the matching images in the first image acquisition end is obtained.

[0081] It can be understood that when the user is broadcasting live in a preset screen environment, there is a second image acquisition end that moves around it. The image captured by the second image acquisition end is actually the image in the screen environment. The screen image captured along the projection angle of the second image acquisition end is the image to be matted. The image in the real environment captured by the first image acquisition end and the screen image can be matted.

[0082] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0083] Reference Figure 3 , shows a structural block diagram of a panoramic live broadcast of the present invention, which may specifically include the following modules:

[0084] A virtual sightline acquisition module 100 is used to acquire angular position information and displacement information of a virtual sightline of a user device, wherein the angular position information includes three-dimensional orientation data and angle data of the virtual sightline, and the displacement information includes coordinate data of the user device;

[0085] The calculation module 200 is used to calculate the coordinate variable data according to the real-time coordinate data and the historical coordinate data, and send the coordinate variable data to the first image acquisition end;

[0086] An image capture module 300 is used to obtain one of the images matching the stereoscopic orientation data in the first image acquisition end, capture a corresponding display image according to the angle data, and send it to the user device;

[0087] The curtain image acquisition module 400 is used to acquire the curtain image data acquired by the second image acquisition end;

[0088] The environment image acquisition module 500 is used to acquire one of the images of the first image acquisition end that matches the azimuth angle data according to the azimuth angle data of the second image acquisition end;

[0089] The matting module 600 is used to overlap one of the acquired images with the curtain background in the curtain image by using the matting technology to obtain a live broadcast image and send it.

[0090] In this embodiment, the calculation module 200 includes:

[0091] A first acquisition submodule is used to acquire coordinate data corresponding to the same time node as the current frame image acquired by the first image acquisition end, so as to obtain real-time coordinate data;

[0092] A second acquisition submodule is used to acquire coordinate data corresponding to the same time node as the previous frame image acquired by the first image acquisition end, so as to obtain historical coordinate data;

[0093] The calculation submodule is used to calculate the difference between the real-time coordinate data and the historical coordinate data to obtain the coordinate variable data.

[0094] In this embodiment, the image capture module 300 includes:

[0095] A relationship submodule, used for obtaining a mapping relationship between each stereoscopic orientation in the pre-established stereoscopic orientation data and each channel of images of the first image acquisition end;

[0096] A determination submodule, used for determining one of the images to be acquired according to the mapping relationship;

[0097] The capture submodule is used to obtain the viewing angle information of the user device, determine the image display size of the user device, and capture the corresponding display image according to the angle value in the angle data.

[0098] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0099] like Figure 4 , showing a computer device of a panoramic live broadcast method of the present invention, which may specifically include the following:

[0100] In an embodiment of the present invention, the present invention further provides a computer device. The above-mentioned computer device 12 is manifested in the form of a general-purpose computing device. The components of the computer device 12 may include but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).

[0101] The bus 18 represents one or more of several types of bus 18 structures, including a memory bus 18 or memory controller, a peripheral bus 18, an accelerated graphics port, a processor or a local bus 18 using any of a variety of bus 18 architectures. These architectures include, by way of example, but are not limited to, an Industry Standard Architecture (ISA) bus 18, a Micro Channel Architecture (MAC) bus 18, an Enhanced ISA bus 18, an Audio Video Electronics Standards Association (VESA) local bus 18, and a Peripheral Component Interconnect (PCI) bus 18.

[0102] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0103] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 31 and / or cache memory 32. Computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write to non-removable, non-volatile magnetic media (commonly referred to as a "hard drive"). Although Figure 4 Not shown, a disk drive for reading and writing to a removable non-volatile disk (such as a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (such as a CD-ROM, DVD-ROM or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory may include at least one program product having a set (e.g., at least one) of program modules 42, which are configured to perform the functions of various embodiments of the present invention.

[0104] A program / utility 41 having a set (at least one) of program modules 42 may be stored in, for example, a memory, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules 42, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0105] The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, cameras, etc.), may communicate with one or more devices that enable a user to interact with the computer device 12, and / or may communicate with any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the computer device 12 may also communicate with one or more networks (e.g., local area networks (LANs)), wide area networks (WANs), and / or public networks (e.g., the Internet) via a network adapter 20. As shown, the network adapter 21 communicates with other modules of the computer device 12 via a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the computer device 12, including, but not limited to, microcode, device drivers, redundant processing units 16, external disk drive arrays, RAID systems, tape drives, and data backup storage systems 34, etc.

[0106] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing a panoramic live broadcast method provided by an embodiment of the present invention.

[0107] That is, when the processing unit 16 executes the above program, it realizes: obtaining the angular position information and displacement information of the virtual line of sight of the user device, wherein the angular position information includes the three-dimensional orientation data and angle data of the virtual line of sight, and the displacement information includes the coordinate data of the user device; calculating the coordinate variable data according to the real-time coordinate data and the historical coordinate data, and sending the coordinate variable data to the first image acquisition end; obtaining one of the images in the first image acquisition end that matches the three-dimensional orientation data, and intercepting the corresponding display image according to the angle data and sending it to the user device. And / or,

[0108] Acquire the curtain image data acquired by the second image acquisition end; acquire one of the images of the first image acquisition end that matches the azimuth angle data according to the azimuth angle data of the second image acquisition end; overlap one of the acquired images with the curtain background in the curtain image through the matting technology to obtain the live broadcast image and send it.

[0109] In an embodiment of the present invention, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a panoramic live broadcast method as provided in all embodiments of the present application.

[0110] That is, when the program is executed by the processor, it is implemented as follows: obtaining the angular position information and displacement information of the virtual line of sight of the user device, wherein the angular position information includes the three-dimensional orientation data and angle data of the virtual line of sight, and the displacement information includes the coordinate data of the user device; calculating the coordinate variable data based on the real-time coordinate data and the historical coordinate data, and sending the coordinate variable data to the first image acquisition end; obtaining one of the images in the first image acquisition end that matches the three-dimensional orientation data, and intercepting the corresponding display image based on the angle data and sending it to the user device. And / or,

[0111] Acquire the curtain image data acquired by the second image acquisition end; acquire one of the images of the first image acquisition end that matches the azimuth angle data according to the azimuth angle data of the second image acquisition end; overlap one of the acquired images with the curtain background in the curtain image through the matting technology to obtain the live broadcast image and send it.

[0112] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPOM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device or device.

[0113] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0114] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0115] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0116] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0117] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0118] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0119] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0120] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0121] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0122] The panoramic live broadcast method and device provided by the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A panoramic live broadcast method, characterized in that: include: Acquire angular position information and displacement information of a virtual line of sight of a user device, wherein the angular position information includes three-dimensional orientation data and angle data of the virtual line of sight, and the displacement information includes coordinate data of the user device; Calculate coordinate variable data according to the real-time coordinate data and the historical coordinate data, and send the coordinate variable data to a first image acquisition end, wherein the first image acquisition end is a drone device equipped with N cameras, which can synchronously acquire N video images in real time in a real environment and send them to a cloud server via 5G communication; Acquire one of the images matching the three-dimensional orientation data in the first image acquisition end, intercept a corresponding display image according to the angle data and send it to the user device, wherein one of the images in the first image acquisition end is an image acquired by one of the image acquisition devices corresponding to each three-dimensional orientation of the virtual sight line of the user device; Acquire the curtain image data collected by the second image collection end, wherein the second image collection end is a camera device movable in the background curtain or a camera device for multi-channel fixed-point shooting; According to the azimuth angle data of the second image acquisition end, one of the images of the first image acquisition end that matches the azimuth angle data is acquired, wherein the one of the images of the first image acquisition end that matches the azimuth angle data is an image in the real environment correspondingly acquired by the first image acquisition end; One of the images obtained by the first image acquisition end and matching the azimuth angle data is superimposed with the curtain background in the curtain image by using the matting technology to obtain a live image and send it.

2. The method according to claim 1, characterized in that The step of calculating coordinate variable data according to the real-time coordinate data and the historical coordinate data, and sending the coordinate variable data to the first image acquisition end, comprises: Acquire coordinate data corresponding to the same time node as the current frame image acquired by the first image acquisition end to obtain real-time coordinate data; Acquire coordinate data corresponding to the same time node as the previous frame image acquired by the first image acquisition end to obtain historical coordinate data; The difference between the real-time coordinate data and the historical coordinate data is calculated to obtain the coordinate variable data.

3. The method according to claim 1, characterized in that The step of acquiring one of the images matching the stereoscopic orientation data in the first image acquisition end, intercepting a corresponding display image according to the angle data and sending the image to the user device includes: Acquire a mapping relationship between each stereoscopic orientation in the pre-established stereoscopic orientation data and each channel of images from the first image acquisition end; Determine one of the images to be acquired through the mapping relationship; The viewing angle information of the user device is obtained, the image display size of the user device is determined, and the corresponding display image is intercepted according to the angle value in the angle data.

4. The method according to claim 1, characterized in that: The step of acquiring, according to the azimuth angle data of the second image acquisition end, one of the images of the first image acquisition end that matches the azimuth angle data comprises: Constructing a correspondence between a preset curtain environment and each channel of images from the first image acquisition end; Acquiring azimuth angle data of the second image acquisition device in the curtain environment; According to the corresponding relationship and the azimuth angle data, one of the matching images in the first image acquisition end is obtained.

5. A panoramic live broadcast device, characterized in that: include: A virtual sightline acquisition module, used to acquire angular position information and displacement information of a virtual sightline of a user device, wherein the angular position information includes three-dimensional orientation data and angle data of the virtual sightline, and the displacement information includes coordinate data of the user device; A calculation module is used to calculate coordinate variable data according to the real-time coordinate data and the historical coordinate data, and send the coordinate variable data to a first image acquisition end, wherein the first image acquisition end is a drone device equipped with N cameras, which can synchronously collect N video images in real time in a real environment and send them to a cloud server via 5G communication; An image capture module, used to obtain one of the images in the first image acquisition end that matches the three-dimensional orientation data, capture a corresponding display image according to the angle data and send it to the user device, wherein one of the images in the first image acquisition end is an image captured by one of the image acquisition devices corresponding to each three-dimensional orientation of the virtual sight line of the user device; A curtain image acquisition module, used to acquire curtain image data acquired by a second image acquisition end, wherein the second image acquisition end is a camera device movable in the background curtain or a camera device for multi-channel fixed-point shooting; An environmental image acquisition module, used to acquire, according to the azimuth angle data of the second image acquisition end, one of the images of the first image acquisition end that matches the azimuth angle data, wherein the one of the images of the first image acquisition end that matches the azimuth angle data is an image of the real environment correspondingly acquired by the first image acquisition end; The matting module is used to overlap one of the images obtained by the first image acquisition end and matching the azimuth angle data with the curtain background in the curtain image through the matting technology to obtain a live broadcast image and send it.

6. The device according to claim 5, characterized in that The computing module comprises: A first acquisition submodule is used to acquire coordinate data corresponding to the same time node as the current frame image acquired by the first image acquisition end, so as to obtain real-time coordinate data; A second acquisition submodule is used to acquire coordinate data corresponding to the same time node as the previous frame image acquired by the first image acquisition end, so as to obtain historical coordinate data; The calculation submodule is used to calculate the difference between the real-time coordinate data and the historical coordinate data to obtain the coordinate variable data.

7. The device according to claim 5, characterized in that The image capture module comprises: A relationship submodule, used for obtaining a mapping relationship between each stereoscopic orientation in the pre-established stereoscopic orientation data and each channel of images of the first image acquisition end; A determination submodule, used for determining one of the images to be acquired according to the mapping relationship; The capture submodule is used to obtain the viewing angle information of the user device, determine the image display size of the user device, and capture the corresponding display image according to the angle value in the angle data.

8. An electronic device, characterized in that The method comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the method according to any one of claims 1 to 4 when executed by the processor.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images