Pixel streaming method, device and pixel streaming system

Abstract

The present application discloses a pixel streaming method, device and pixel streaming system, which belong to the field of image processing technology. The pixel streaming method includes: obtaining the first image captured by each target camera; encoding each first image to generate at least one video stream signal; sending at least one video stream signal to a signaling server for the signaling server to send the processed video stream signal to the client. The pixel streaming method of the present application can effectively distinguish each video stream signal, thereby realizing the simultaneous transmission of multiple video stream signals, so as to realize image switching under multiple viewing angles and simultaneous viewing of multiple signals; and multiple signals can be dynamically added without adjusting the existing equipment architecture, with high dynamic adaptability, simple network topology and convenient deployment.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular, relates to a pixel streaming method, device and pixel streaming system. Background Art

[0002] In the related art, pixel streaming mainly adopts streaming and other methods, first rendering the picture on a server, and then transmitting the picture to other screens or web pages through streaming, etc. However, this method cannot achieve picture switching and cannot support multiple video channels. Summary of the invention

[0003] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a pixel streaming method, device and pixel streaming system, which can effectively distinguish each video stream signal, thereby realizing the simultaneous transmission of multiple video stream signals, so as to realize image switching under multiple viewing angles and simultaneous viewing of multiple signals; and multiple signals can be dynamically added without adjusting the existing device architecture, with high dynamic adaptability, simple network topology and convenient deployment.

[0004] In a first aspect, the present application provides a pixel streaming method, which is applied to a first server, wherein at least one target camera is disposed in the first server; the method comprises:

[0005] Acquire a first image captured by each of the target cameras;

[0006] Encoding each of the first images to generate at least one video stream signal;

[0007] The at least one video stream signal is sent to a signaling server, so that the signaling server can send the processed video stream signal to the client.

[0008] According to the pixel streaming method of the present application, by setting one or more target cameras for each first server, each target camera corresponds to unique identification information, so that each video stream signal is sent to the same signaling server for processing based on the identification information, and each video stream signal can be effectively distinguished, thereby realizing the simultaneous transmission of multiple video stream signals, so as to realize image switching under multiple viewing angles and simultaneous viewing of multiple signals; and multiple signals can be dynamically added without adjusting the existing equipment architecture, with high dynamic adaptability, and the network topology structure is simple and easy to deploy.

[0009] According to an embodiment of the present application, each of the target cameras corresponds to at least one stream channel, and each of the stream channels is correspondingly provided with an encoder; the stream channel corresponds to identification information of the target camera; and encoding each of the first images to generate at least one video stream signal includes:

[0010] The encoder corresponding to the stream channel is used to encode the first image transmitted to the stream channel to generate the video stream signal and identification information corresponding to the video stream signal.

[0011] According to an embodiment of the present application, the encoding of the first image transmitted to the stream channel by using the encoder corresponding to the stream channel to generate the video stream signal includes:

[0012] Performing differential compression processing on a first image corresponding to a current image frame and a first image corresponding to a historical image frame corresponding to the stream channel to obtain a differential partial code;

[0013] The video stream signal is generated based on the difference portion encoding.

[0014] According to one embodiment of the present application, the acquiring the first image captured by each of the target cameras includes:

[0015] Acquire image information collected by the target camera at a preset field of view angle;

[0016] The image information is processed based on the target size to obtain the first image.

[0017] In a second aspect, the present application provides a pixel streaming method, which is applied to a signaling server, and the method includes:

[0018] Receiving at least one video stream signal sent by at least one first server; the video stream signal is obtained by processing a first image captured by at least one target camera set in the first server;

[0019] The video stream signal is processed, and the processed video stream signal is sent to at least one client.

[0020] In a third aspect, the present application provides a pixel streaming device, which is applied to a first server, wherein at least one target camera is disposed in the first server; the device comprises:

[0021] A first processing module, used for acquiring a first image captured by each of the target cameras;

[0022] A second processing module, used for encoding each of the first images to generate at least one video stream signal;

[0023] The third processing module is used to send the at least one video stream signal to the signaling server, so that the signaling server can send the processed video stream signal to the client.

[0024] According to the pixel streaming device of the present application, by setting one or more target cameras for each first server, each target camera corresponds to unique identification information, so that each video stream signal is sent to the same signaling server for processing based on the identification information, and each video stream signal can be effectively distinguished, thereby realizing the simultaneous transmission of multiple video stream signals, so as to realize image switching under multiple viewing angles and simultaneous viewing of multiple signals; and multiple signals can be dynamically added without adjusting the existing equipment architecture, with high dynamic adaptability, and the network topology structure is simple and easy to deploy.

[0025] In a fourth aspect, the present application provides a pixel streaming device, which is applied to a signaling server, and the device includes:

[0026] A fourth processing module, used for receiving at least one video stream signal sent by at least one first server; the video stream signal is obtained by processing a first image acquired by at least one target camera set in the first server;

[0027] The fifth processing module is used to process the video stream signal and send the processed video stream signal to at least one client.

[0028] In a fifth aspect, the present application provides a pixel streaming system, including:

[0029] At least one first server, each of which is provided with at least one target camera, and each of which corresponds to at least one stream channel;

[0030] a signaling server, the signaling server being communicatively connected with each of the first servers through each of the stream channels;

[0031] The pixel streaming system operates based on the pixel streaming method as described in the first aspect or the second aspect.

[0032] In a sixth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the pixel streaming method as described in the first aspect above.

[0033] In a seventh aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the pixel streaming method as described in the first aspect above.

[0034] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:

[0035] By setting one or more target cameras for each first server, each target camera corresponds to unique identification information, so that each video stream signal is sent to the same signaling server for processing based on the identification information, and each video stream signal can be effectively distinguished, thereby realizing the simultaneous transmission of multiple video stream signals, so as to realize image switching under multiple viewing angles and simultaneous viewing of multiple signals; and multiple signals can be dynamically added without adjusting the existing equipment architecture, with high dynamic adaptability, simple network topology, and convenient deployment.

[0036] Furthermore, by setting one or more target cameras for each first server and setting a corresponding encoder for each target camera, the encoder encodes a video stream signal with unique identification information corresponding to the first image captured by the corresponding target camera. Signals from different channels can be distinguished according to the identification information to achieve image switching under multiple viewing angles and simultaneous viewing of multiple signals. The operation is simple and convenient, and it has high flexibility and dynamic adaptability.

[0037] Furthermore, by setting one or more target cameras for each first server and setting a corresponding encoder for each target camera, the encoder encodes a video stream signal with unique identification information corresponding to the first image captured by the corresponding target camera. Signals from different paths can be distinguished according to the identification information, and differential compression can be performed on the first images with the same identification information, thereby saving video stream performance, improving video stream transmission efficiency, and effectively improving the quality of picture generation.

[0038] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0040] Figure 1 This is one of the flowcharts of the pixel streaming method provided in the embodiment of the present application;

[0041] Figure 2 is a schematic diagram of the structure of a pixel streaming system provided in an embodiment of the present application;

[0042] Figure 3 This is the second flowchart of the pixel streaming method provided in the embodiment of the present application;

[0043] Figure 4 It is one of the structural schematic diagrams of the pixel streaming device provided in the embodiment of the present application;

[0044] Figure 5 This is the second structural schematic diagram of the pixel streaming device provided in the embodiment of the present application;

[0045] Figure 6 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0047] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0048] The pixel streaming method, pixel streaming device, electronic device and readable storage medium provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.

[0049] The pixel streaming method may be applied to a terminal, and may be specifically executed by hardware or software in the terminal.

[0050] In the following various embodiments, a terminal including a display and a touch-sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, a mouse and a joystick.

[0051] The pixel streaming method provided in the embodiment of the present application may be executed by an electronic device or a functional module or functional entity in the electronic device that can implement the pixel streaming method. The electronic devices mentioned in the embodiment of the present application include but are not limited to mobile phones, tablet computers, computers, cameras, and wearable devices. The pixel streaming method provided in the embodiment of the present application is described below using an electronic device as an example of an execution subject.

[0052] The process of pixel streaming involves a first server and a signaling server.

[0053] The number of the first servers is one or more.

[0054] Each first server is respectively connected to the same signaling server for communication.

[0055] The signaling server is connected to one or more clients for communication.

[0056] Clients include but are not limited to: electronic devices and other servers.

[0057] The first server is used for performing picture rendering and generating a video stream signal.

[0058] At least one target camera is arranged in the first server.

[0059] The field of view angles corresponding to the target cameras may be the same or different, and are used to collect images at different field of view angles.

[0060] The target camera may include a physical camera or a virtual camera.

[0061] The signaling server is used to forward the received video stream signal to other devices or web browsers for users to watch.

[0062] like Figure 1 As shown, the pixel streaming method is applied to a first server, and the pixel streaming method includes: step 110, step 120 and step 130.

[0063] Step 110, obtaining a first image captured by each target camera;

[0064] In this step, the first server is a server for generating image information.

[0065] like Figure 2 As shown, the number of the first servers may be one or more, and each first server is respectively connected to the same signaling server for communication.

[0066] Different first servers can be used to generate different first images.

[0067] Each first server may be provided with one or more target cameras to collect relevant images in the actual scene or virtual scene corresponding to the first server.

[0068] The first image includes some features of the actual scene or the virtual scene.

[0069] A target camera is used to capture one or more first images. For multiple target cameras set in the same first server, the viewing angles of the first images captured by different target cameras may be different, so that the first images captured by different target cameras may be different.

[0070] Different target cameras may be of different categories, including but not limited to: grayscale cameras and distance field cameras.

[0071] In some embodiments, taking the target camera as a virtual camera as an example, the first image captured by the virtual camera can be used to simulate what the agent sees. For example, in the field of virtual reality, the virtual camera simulates the eyes of the agent, and the first image captured by the virtual camera is used to simulate the signals captured by the two eyes of the agent.

[0072] Pixel Streaming, also known as streaming, video streaming or rendering cloud streaming, etc.

[0073] Pixel streaming is used to convert virtual rendering content generated in a server into a video stream signal and transmit it over the network in real time. Users can watch it on other devices or servers through a web browser or video player.

[0074] In some embodiments, step 110 may include:

[0075] Obtain image information collected by the target camera under a preset field of view angle;

[0076] The image information is processed based on the target size to obtain a first image.

[0077] In this embodiment, the value of the preset field of view angle may be customized by the user.

[0078] In the case where a plurality of target cameras are arranged in the same first server, the preset field of view angles corresponding to the target cameras may be different, so as to respectively capture images at different field of view angles to generate the first image.

[0079] In the actual execution process, the focal length of the target camera can be adjusted based on the preset field of view angle of the target camera, and the image information can be collected and intercepted to the target size to obtain the first image.

[0080] According to the pixel streaming method provided in the embodiment of the present application, by setting one or more target cameras for each first server, different target cameras are set with different preset field of view angles to perform multi-perspective image acquisition, which is used for the subsequent display and switching of multiple signals. Multiple signals can be dynamically added without adjusting the existing device architecture. It has high dynamic adaptability, simple network topology, and easy deployment.

[0081] Step 120: Encode each first image to generate at least one video stream signal;

[0082] In this step, the first image is encoded for scene rendering.

[0083] One frame of the first image corresponds to one video stream signal.

[0084] In some embodiments, at least one of color difference, illumination, and shadow may be rendered, and the rendering data format may be converted into a pixel stream signal to obtain a video stream signal.

[0085] In some embodiments, each target camera corresponds to at least one stream channel, and each stream channel is provided with an encoder; the stream channel corresponds to identification information of the target camera; step 120 may include:

[0086] The encoder corresponding to the stream channel is used to encode the first image transmitted to the stream channel to generate a video stream signal and identification information corresponding to the video stream signal.

[0087] In this embodiment, the identification information is a unique identification of the stream channel and the target camera, and is used to distinguish the video stream information corresponding to the first image captured by other stream channels and other target cameras.

[0088] The identification information may be in the form of an ID or a number, etc.

[0089] The encoder is used for picture rendering and encodes the first image into a video stream signal.

[0090] For example, a unique ID may be set for each stream channel, and an encoder corresponding to the stream channel encodes the first image transmitted by the stream channel to generate a video stream signal with unique identification information.

[0091] In the subsequent process of the client watching images or videos, by switching the identification information, it can switch to the video stream signal corresponding to the identification information to display the first image under the field of view angle corresponding to the identification information, thereby realizing flexible switching of multiple viewing angles.

[0092] During the actual execution process, the corresponding encoder can be created according to the video stream information during initialization.

[0093] In the subsequent decoding process, the corresponding encoder is used for decoding according to the frame format type.

[0094] In some embodiments, FTongPixelCaptureSource may be used instead of native CaptureSource to facilitate distinguishing identification information corresponding to video stream information during decoding.

[0095] According to the pixel streaming method provided in the embodiment of the present application, by setting one or more target cameras for each first server and setting a corresponding encoder for each target camera, the encoder encodes a video stream signal with unique identification information corresponding to the first image captured by the corresponding target camera. Signals of different channels can be distinguished according to the identification information to achieve image switching under multiple viewing angles and simultaneous viewing of multiple signals. The operation is simple and convenient, and it has high flexibility and dynamic adaptability.

[0096] In some embodiments, encoding the first image transmitted to the stream channel using an encoder configured corresponding to the stream channel to generate a video stream signal may include:

[0097] Performing differential compression processing on a first image corresponding to the current image frame and a first image corresponding to a historical image frame corresponding to the stream channel to obtain a differential partial code;

[0098] Based on the difference part encoding, a video stream signal is generated.

[0099] In this embodiment, the generation and acquisition method of the first image corresponding to the historical image frame is similar to the generation and acquisition method of the first image corresponding to the current image frame, and this application will not elaborate on it here.

[0100] The new first image currently obtained can be differentially processed with the historical image frame that has previously passed through the stream channel to obtain the difference features between the new frame and the previous frame, the difference features can be compressed to obtain the difference part encoding, and the difference part encoding can be transmitted as the video stream signal corresponding to the current first image to save the performance of the video stream and improve the transmission efficiency of the video stream.

[0101] Subsequently, the historical image frames and the difference parts may be encoded and decoded to obtain a complete first image.

[0102] During the research and development process, the inventors discovered that in the related technology, each server has only one encoder. If multiple stream signals are processed at the same time, the frame images of the multiple signals will be differentially compressed, causing the image to be chaotic and flickering, affecting the image generation effect.

[0103] In the present application, one or more target cameras are set for each first server, and a corresponding encoder is set for each target camera. The encoder encodes a video stream signal with unique identification information corresponding to the first image captured by the corresponding target camera, so that in the process of processing the first image and the video stream signal, signals of different paths can be distinguished according to the identification information, and differential compression can be performed on the first images with the same identification information, thereby reducing the probability of chaotic and flickering images and improving the generated quality of the images.

[0104] According to the pixel streaming method provided in the embodiment of the present application, by setting one or more target cameras for each first server and setting a corresponding encoder for each target camera, the encoder encodes a video stream signal with unique identification information corresponding to the first image captured by the corresponding target camera. Signals of different paths can be distinguished according to the identification information, and differential compression can be performed on the first images with the same identification information, thereby saving video stream performance, improving video stream transmission efficiency, and significantly improving the generated image quality.

[0105] Step 130: Send at least one video stream signal to a signaling server, so that the signaling server can send the processed video stream signal to the client.

[0106] In this step, the signaling server is used to receive the video stream signals of each channel sent by each first server, and distinguish the received multiple video stream signals based on the identification information corresponding to each video stream signal.

[0107] The signaling server is connected to the client for communication, and sends a corresponding video stream signal to the client based on a request sent by the client.

[0108] In the actual implementation process, the following procedures can be used to implement the above steps:

[0109] 1. Signaling Server

[0110] / / Set the signaling server address, format is ws: / / 127.0.0.1:8888

[0111] static void SetSignallingServerURL(FString URL);

[0112] 2. Streaming Control

[0113] / / Activate PixelCamera for streaming

[0114] static void StartPixelCameraStreaming(FString CameraName);

[0115] / / PixelCamera stops streaming

[0116] static void StopPixelCameraStreaming(FString CameraName);

[0117] / / Activate all PixelCameras for streaming

[0118] static void StartAllPixelCameraStreaming();

[0119] / / PixelCamera stops all streaming

[0120] static void StopAllPixelCameraStreaming();

[0121] / / Returns whether a PixelCamera is streaming

[0122] static bool IsPixelCameraActive(FString CameraName);

[0123] / / Get the names of all PixelCamera in the scene

[0124] static TArray <fstring>GetAllPixelCameraNames();

[0125] 3.Actor&Transform

[0126] / / Return to PixelCamera Actor

[0127] static ATongPixelCameraActor* GetPixelCameraActor(FStringCameraName);

[0128] / / Return PixelCamera Transform information

[0129] static FTransform GetPixelCameraTransform(FString CameraName).

[0130] like Figure 2 As shown, for example, the signaling server receives three video stream signals A, B and C sent by server 1, and two video stream signals D and E sent by server 2. User A sends a request to the signaling server through a computer web page to view video stream signal A. In response to the request, the signaling server sends video stream signal A to the computer web page for user A to watch. When user A needs to switch channels, he only needs to switch the identification information, such as switching A to B, to view video stream signal B.

[0131] The inventors also discovered during the research and development process that in the related technology, each server can only generate one video stream signal through a video encoder, and in order to avoid signal confusion, this video stream signal can only be sent to a single signaling server for processing; resulting in a server that can only generate one image, cannot switch viewing angles, and cannot support multiple video channels, and can only view one channel on the main screen; when encountering a scenario where multiple video stream signals need to be transmitted, it is necessary to add multiple servers and multiple signaling servers to solve the problem, resulting in a complex architecture and high design costs.

[0132] According to the pixel streaming method provided in the embodiment of the present application, by setting one or more target cameras for each first server, each target camera corresponds to unique identification information, so that each video stream signal is sent to the same signaling server for processing based on the identification information, and each video stream signal can be effectively distinguished, thereby realizing the simultaneous transmission of multiple video stream signals, so as to realize image switching under multiple viewing angles and simultaneous viewing of multiple signals; and multiple signals can be dynamically added without adjusting the existing equipment architecture, with high dynamic adaptability, and the network topology structure is simple and easy to deploy.

[0133] An embodiment of the present application provides a pixel streaming method.

[0134] like Figure 3 As shown, the pixel streaming method is applied to a signaling server, and the pixel streaming method includes: step 310 and step 320.

[0135] Step 310: receiving at least one video stream signal sent by at least one first server; the video stream signal is obtained by processing a first image captured by at least one target camera set in the first server;

[0136] In this step, the signaling server can be connected to multiple first servers, such as Figure 2 shown.

[0137] Different first servers are connected to the first server via different streaming channels.

[0138] In the case that the first server is provided with a plurality of target cameras, the plurality of target cameras are provided with stream channels in a one-to-one correspondence, and each target camera sends the acquired first image to the stream channel based on the corresponding stream channel.

[0139] Step 320: Process the video stream signal, and send the processed video stream signal to at least one client.

[0140] In this step, one encoder is set corresponding to one stream channel, which is used to perform scene rendering on the first image to generate a video stream signal. Each video stream signal corresponds to unique identification information. The generated video stream signal and the identification information corresponding to the video stream signal are transmitted to the signaling server through the stream channel for subsequent processing by the signaling server.

[0141] According to the pixel streaming method provided in the embodiment of the present application, by setting one or more target cameras for each first server, each target camera corresponds to unique identification information, so that each video stream signal is sent to the same signaling server for processing based on the identification information, and each video stream signal can be effectively distinguished, thereby realizing the simultaneous transmission of multiple video stream signals, so as to realize image switching under multiple viewing angles and simultaneous viewing of multiple signals; and multiple signals can be dynamically added without adjusting the existing equipment architecture, with high dynamic adaptability, and the network topology structure is simple and easy to deploy.

[0142] In some embodiments, step 320 may further include:

[0143] Processing video stream signals to obtain identification information corresponding to each video stream signal;

[0144] Based on the identification information, a processed target video stream signal is sent to a target client among at least one client.

[0145] In this embodiment, the target video stream signal may include one or more video stream signals obtained by encoding first images captured by different target cameras.

[0146] The target client may be any one or more clients that are in communication connection with the signaling server.

[0147] In the actual execution process, the target client can obtain the video stream signal corresponding to the identification information from the signaling server by switching the identification information, so as to be displayed on the target client.

[0148] According to the pixel streaming method provided in the embodiment of the present application, by setting one or more target cameras for each first server and setting a corresponding encoder for each target camera, the encoder encodes a video stream signal with unique identification information corresponding to the first image captured by the corresponding target camera. Signals of different channels can be distinguished according to the identification information to achieve image switching under multiple viewing angles and simultaneous viewing of multiple signals. The operation is simple and convenient, and it has high flexibility and dynamic adaptability.

[0149] The pixel streaming method provided in the embodiment of the present application can be executed by a pixel streaming device. In the embodiment of the present application, the pixel streaming device provided in the embodiment of the present application is described by taking the pixel streaming device executing the pixel streaming method as an example.

[0150] The embodiment of the present application also provides a pixel streaming device.

[0151] like Figure 4 As shown, the pixel streaming device is applied to a first server, and at least one target camera is arranged in the first server; the pixel streaming device includes: a first processing module 410, a second processing module 420 and a third processing module 430.

[0152] A first processing module 410 is used to obtain a first image captured by each target camera;

[0153] The second processing module 420 is used to encode each first image to generate at least one video stream signal;

[0154] The third processing module 430 is used to send at least one video stream signal to the signaling server, so that the signaling server can send the processed video stream signal to the client.

[0155] According to the pixel streaming device provided in the embodiment of the present application, by setting one or more target cameras for each first server, each target camera corresponds to unique identification information, so that each video stream signal is sent to the same signaling server for processing based on the identification information, and each video stream signal can be effectively distinguished, thereby realizing the simultaneous transmission of multiple video stream signals, so as to realize image switching under multiple viewing angles and simultaneous viewing of multiple signals; and multiple signals can be dynamically added without adjusting the existing equipment architecture, with high dynamic adaptability, and the network topology structure is simple and easy to deploy.

[0156] In some embodiments, each target camera corresponds to at least one stream channel, and each stream channel is provided with an encoder; the stream channel corresponds to identification information of the target camera; the second processing module 420 can also be used to:

[0157] The encoder corresponding to the stream channel is used to encode the first image transmitted to the stream channel to generate a video stream signal and identification information corresponding to the video stream signal.

[0158] In some embodiments, the second processing module 420 may also be used to:

[0159] Performing differential compression processing on a first image corresponding to the current image frame and a first image corresponding to a historical image frame corresponding to the stream channel to obtain a differential partial code;

[0160] Based on the difference part encoding, a video stream signal is generated.

[0161] In some embodiments, the first processing module 410 may also be used to:

[0162] Obtain image information collected by the target camera under a preset field of view angle;

[0163] The image information is processed based on the target size to obtain a first image.

[0164] The embodiment of the present application also provides a pixel streaming device.

[0165] like Figure 5 As shown, the pixel streaming device is applied to a signaling server; the pixel streaming device includes: a fourth processing module 510 and a fifth processing module 520.

[0166] The fourth processing module 510 is used to receive at least one video stream signal sent by at least one first server; the video stream signal is obtained by processing a first image captured by at least one target camera set in the first server;

[0167] The fifth processing module 520 is used to process the video stream signal and send the processed video stream signal to at least one client.

[0168] According to the pixel streaming device provided in the embodiment of the present application, by setting one or more target cameras for each first server, each target camera corresponds to unique identification information, so that each video stream signal is sent to the same signaling server for processing based on the identification information, and each video stream signal can be effectively distinguished, thereby realizing the simultaneous transmission of multiple video stream signals, so as to realize image switching under multiple viewing angles and simultaneous viewing of multiple signals; and multiple signals can be dynamically added without adjusting the existing equipment architecture, with high dynamic adaptability, and the network topology structure is simple and easy to deploy.

[0169] In some embodiments, the fifth processing module 520 may also be used to:

[0170] Processing video stream signals to obtain identification information corresponding to each video stream signal;

[0171] Based on the identification information, a processed target video stream signal is sent to a target client among at least one client.

[0172] The embodiment of the present application also provides a pixel streaming system.

[0173] like Figure 2 As shown, the pixel streaming system includes: at least one first server and a signaling server.

[0174] The first server is used to generate a first image, and perform screen rendering on the first image to generate a video stream signal.

[0175] The number of the first server is one or more.

[0176] Each first server is respectively connected to the same signaling server for communication.

[0177] At least one target camera is respectively arranged in each first server, and each target camera corresponds to at least one stream channel, and the stream channel is used for rendering and signal transmission of the first image captured by the target camera.

[0178] The field of view angles corresponding to the target cameras may be the same or different, and are used to collect the first images at different field of view angles.

[0179] The signaling server is communicatively connected with each first server through each stream channel.

[0180] The signaling server is connected to the client for forwarding the received video stream signal to other devices or web browsers for users to watch.

[0181] Clients include but are not limited to: electronic devices and other servers.

[0182] The pixel streaming system operates based on the pixel streaming method described in any of the above embodiments.

[0183] In some embodiments, the pixel streaming system may further include: at least one encoder.

[0184] In this embodiment, at least one encoder is arranged in a one-to-one correspondence with at least one stream channel.

[0185] The encoder is used to perform scene rendering on the first image and encode it to obtain video stream information.

[0186] In some embodiments, the pixel flow system may further include: a flow channel management module.

[0187] In this embodiment, the flow channel management modules are electrically connected to each flow channel respectively, and the flow channel management modules are used to manage each flow channel in a unified manner.

[0188] According to the pixel streaming system provided in the embodiment of the present application, by setting one or more target cameras for each first server, each target camera corresponds to unique identification information, so that each video stream signal is sent to the same signaling server for processing based on the identification information, and each video stream signal can be effectively distinguished, thereby realizing the simultaneous transmission of multiple video stream signals, so as to realize image switching under multiple viewing angles and simultaneous viewing of multiple signals; and multiple signals can be dynamically added without adjusting the existing equipment architecture, with high dynamic adaptability, and the network topology structure is simple and easy to deploy.

[0189] The pixel streaming device in the embodiment of the present application can be an electronic device or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc. It can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0190] The pixel streaming device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0191] The pixel streaming device provided in the embodiment of the present application can achieve Figure 1 or Figure 3 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0192] In some embodiments, Figure 6 As shown, an embodiment of the present application further provides an electronic device 600, including a processor 601, a memory 602, and a computer program stored in the memory 602 and executable on the processor 601. When the program is executed by the processor 601, each process of the above-mentioned pixel streaming method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0193] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0194] An embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned pixel streaming method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0195] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0196] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above-mentioned pixel streaming method when executed by a processor.

[0197] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0198] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned pixel streaming method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0199] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0200] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0201] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0202] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0203] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0204] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.< / fstring>

Claims

1. A pixel streaming method, characterized in that: Applied to a first server, wherein at least one target camera is disposed in the first server; the method comprises: Acquire a first image captured by each of the target cameras; Encoding each of the first images to generate at least one video stream signal; Sending the at least one video stream signal to a signaling server, so that the signaling server can send the processed video stream signal to the client; Each of the target cameras corresponds to at least one stream channel, and each of the stream channels is correspondingly provided with an encoder; the stream channel corresponds to identification information of the target camera; and encoding each of the first images to generate at least one video stream signal includes: Using an encoder corresponding to the stream channel, encode the first image transmitted to the stream channel to generate the video stream signal and identification information corresponding to the video stream signal; In the process of processing the first image and the video stream signal, differentiating signals from different channels according to the identification information; Each of the first servers is respectively connected to the same signaling server for communication; The signaling server is used to receive the video stream signals of each channel sent by each of the first servers, and distinguish the received multiple video stream signals based on identification information corresponding to each of the video stream signals.

2. The pixel streaming method according to claim 1, characterized in that: The step of using an encoder corresponding to the stream channel to encode the first image transmitted to the stream channel to generate the video stream signal includes: Performing differential compression processing on a first image corresponding to a current image frame and a first image corresponding to a historical image frame corresponding to the stream channel to obtain a differential partial code; The video stream signal is generated based on the difference portion encoding.

3. The pixel streaming method according to claim 2, characterized in that: The step of acquiring the first image captured by each of the target cameras comprises: Acquire image information collected by the target camera at a preset field of view angle; The image information is processed based on the target size to obtain the first image.

4. A pixel streaming method, characterized in that: Applied to a signaling server, the method comprises: Receiving at least one video stream signal sent by at least one first server; the video stream signal is obtained by processing a first image captured by at least one target camera set in the first server; Processing the video stream signal, and sending the processed video stream signal to at least one client; Each of the target cameras corresponds to at least one stream channel, and each of the stream channels is correspondingly provided with an encoder; the video stream signal is obtained by processing a first image acquired by at least one target camera provided in the first server, and includes: Using an encoder corresponding to the stream channel, encode the first image transmitted to the stream channel to generate the video stream signal and identification information corresponding to the video stream signal; In the process of processing the first image and the video stream signal, differentiating signals from different channels according to the identification information; Each of the first servers is respectively communicatively connected to the same signaling server.

5. The pixel streaming method according to claim 4, characterized in that: The step of processing the video stream signal and sending the processed video stream signal to at least one client comprises: Processing the video stream signals to obtain identification information corresponding to each of the video stream signals; Based on the identification information, a processed target video stream signal is sent to a target client among the at least one client.

6. A pixel streaming device, characterized in that: Applied to a first server, wherein at least one target camera is disposed in the first server; the device comprises: A first processing module, used for acquiring a first image captured by each of the target cameras; A second processing module, used for encoding each of the first images to generate at least one video stream signal; A third processing module, used for sending the at least one video stream signal to a signaling server, so that the signaling server can send the processed video stream signal to the client; Each of the target cameras corresponds to at least one stream channel, and each of the stream channels is correspondingly provided with an encoder; the stream channel corresponds to the identification information of the target camera; the second processing module is used to: Using an encoder corresponding to the stream channel, encode the first image transmitted to the stream channel to generate the video stream signal and identification information corresponding to the video stream signal; In the process of processing the first image and the video stream signal, differentiating signals from different channels according to the identification information; Each of the first servers is respectively connected to the same signaling server for communication; The signaling server is used to receive the video stream signals of each channel sent by each of the first servers, and distinguish the received multiple video stream signals based on identification information corresponding to each of the video stream signals.

7. A pixel streaming device, characterized in that: Applied to a signaling server, the device comprises: A fourth processing module, used for receiving at least one video stream signal sent by at least one first server; the video stream signal is obtained by processing a first image acquired by at least one target camera set in the first server; a fifth processing module, configured to process the video stream signal and send the processed video stream signal to at least one client; Each of the target cameras corresponds to at least one stream channel, and each of the stream channels is correspondingly provided with an encoder; the video stream signal is obtained by processing a first image acquired by at least one target camera provided in the first server, and includes: Using an encoder corresponding to the stream channel, encode the first image transmitted to the stream channel to generate the video stream signal and identification information corresponding to the video stream signal; In the process of processing the first image and the video stream signal, differentiating signals from different channels according to the identification information; Each of the first servers is respectively communicatively connected to the same signaling server.

8. A pixel streaming system, characterized in that: include: At least one first server, each of which is provided with at least one target camera, and each of which corresponds to at least one stream channel; a signaling server, the signaling server being communicatively connected with each of the first servers through each of the stream channels; The pixel streaming system operates based on the pixel streaming method according to any one of claims 1 to 5.

9. The pixel streaming system according to claim 8, wherein: It also includes at least one encoder, and the at least one encoder is arranged in a one-to-one correspondence with the at least one stream channel.

Images