An image processing method, apparatus, system, and storage medium

Abstract

This application discloses an image processing method, apparatus, system, and storage medium for a live streaming device, comprising: acquiring live video frames sent by a live streaming platform, wherein the live video frames are generated based on push video frames sent by the live streaming device to the live streaming platform; fusing the live video frames and push video frames to generate a fused video frame; and transmitting the fused video frame to a display screen, wherein the display screen is deployed at the live streaming site where the live streaming device is located.

Description

Technical Field

[0001] This application relates to the field of live streaming technology, and in particular to an image processing method, apparatus, system, and storage medium. Background Technology

[0002] Currently, with the advancement of network communication technology, live streaming has seen increasing development and application. A typical live streaming system includes video capture equipment, live streaming equipment, a live streaming platform, an app for viewing, monitor 1, and monitor 2. Each monitor corresponds to a different video source; monitor 1 is connected to live streaming equipment 100, and monitor 2 is connected to the app for viewing. Furthermore, the streamer can view the pushed video feed on monitor 1, allowing for timely adjustments to factors affecting the live stream quality, such as position and background. The streamer can also view the live video feed on monitor 2, enabling them to understand audience needs and opinions by reviewing viewer comments.

[0003] Therefore, as mentioned above, during actual live streaming, broadcasters typically need to simultaneously view the images displayed on two different monitors (i.e., Monitor 1 and Monitor 2). However, viewing two different monitors simultaneously has several drawbacks: 1. Because the human eye cannot focus on the details displayed on two monitors simultaneously, frequent switching of the broadcaster's gaze between the two monitors may lead to the omission of crucial information. 2. Due to the different image delays on the two monitors, the broadcaster's words and actions may appear inconsistent. 3. If the broadcaster's gaze shifts back and forth between the two monitors, viewers may perceive it as a lack of focus.

[0004] There is currently no effective solution to the technical problems existing in the above-mentioned technologies, which may lead to the omission of key information, inconsistencies in speech and behavior, and a drifting gaze when the live broadcaster's gaze frequently switches between two different monitors, thus appearing to have decreased concentration. Summary of the Invention

[0005] The embodiments of this disclosure provide an image processing method, apparatus, system, and storage medium to at least solve the technical problem in the prior art that when a live broadcaster frequently switches their gaze between two different displays, it may lead to the omission of key information, inconsistencies in speech and behavior, and drifting of eyes, thus resulting in decreased concentration.

[0006] According to one aspect of the present disclosure, an image processing method is provided for a live streaming device, comprising: acquiring live video frames sent by a live streaming platform, wherein the live video frames are generated based on push video frames sent by the live streaming device to the live streaming platform; fusing the live video frames and the push video frames to generate a fused video frame; and transmitting the fused video frame to a display screen, wherein the display screen is deployed at the live streaming site where the live streaming device is located.

[0007] According to another aspect of the present disclosure, a storage medium is also provided, the storage medium including a stored program, wherein, when the program is executed, a processor performs any of the methods described above.

[0008] According to another aspect of the present disclosure, an image processing system is also provided, including: a live streaming device and a display screen, wherein the display screen faces the live streaming personnel device, and the live streaming device is configured to perform the following operations: acquiring live video frames sent by a live streaming platform, wherein the live video frames are generated based on push video frames sent by the live streaming device to the live streaming platform; fusing the live video frames and the push video frames to generate a fused video frame; and transmitting the fused video frame to the display screen, wherein the display screen is deployed at the live streaming site where the live streaming device is located.

[0009] According to another aspect of the present disclosure, an image processing apparatus is also provided, comprising: a video frame acquisition module for acquiring live video frames sent by a live streaming platform, wherein the live video frames are generated based on push video frames sent from a live streaming device to the live streaming platform; a video frame fusion module for fusing the live video frames and push video frames to generate fused video frames; and a display module for transmitting the fused video frames to a display screen, wherein the display screen is deployed at the live streaming site where the live streaming device is located.

[0010] According to another aspect of the present disclosure, an image processing apparatus is also provided, comprising: a processor; and a memory connected to the processor, configured to provide the processor with instructions to process the following processing steps: acquiring live video frames sent by a live streaming platform, wherein the live video frames are generated based on push video frames sent by a live streaming device to the live streaming platform; fusing the live video frames and the push video frames to generate a fused video frame; and transmitting the fused video frame to a display screen, wherein the display screen is deployed at the live streaming site where the live streaming device is located.

[0011] This application provides an image processing method. First, a live streaming device acquires live video frames sent by a live streaming platform. Then, the live streaming device merges the live video frames with pushed video frames to generate a merged video frame. Finally, the processor of the live streaming device transmits the merged video frame to a display screen.

[0012] As described above, unlike existing technologies where live streamers need to frequently switch their gaze between two different displays to view the live and push video feeds, the live streamer in this application only needs to view a single display screen to see the fused video frame generated from the live and push video frames. This fused video frame contains information from both the live and push video frames. In other words, the live streamer only needs to view the live feed on one display screen to obtain all the information. Because the live streamer's gaze can focus on the details displayed on one screen, without needing to frequently switch between two different displays, key information is not missed, and viewers do not perceive the live streamer as unfocused. This solves the technical problem in existing technologies where frequent switching between two displays can lead to missed key information, inconsistent behavior, and shifty eyes, resulting in decreased focus. Attached Figure Description

[0013] The accompanying drawings, which are included to provide a further understanding of this disclosure and form part of this application, illustrate exemplary embodiments of this disclosure and are used to explain this disclosure, but do not constitute an undue limitation of this disclosure. In the drawings:

[0014] Figure 1A This is a schematic diagram of the image processing system according to Embodiment 1 of this application;

[0015] Figure 1B This is another schematic diagram of the image processing system according to Embodiment 1 of this application;

[0016] Figure 2 This is a schematic flowchart of the image processing method according to Embodiment 1 of this application;

[0017] Figure 3A This is a modular schematic diagram of an image processing method according to Embodiment 1 of this application;

[0018] Figure 3B This is a modular schematic diagram of another image processing method according to Embodiment 1 of this application;

[0019] Figure 4A This is a schematic diagram of pushing video frames according to Embodiment 1 of this application;

[0020] Figure 4B This is a schematic diagram of a live video frame according to Embodiment 1 of this application;

[0021] Figure 5A This is a schematic diagram of an image processing system displaying fused video frames according to Embodiment 1 of this application;

[0022] Figure 5B This is a schematic diagram of another image processing system displaying fused video frames according to Embodiment 1 of this application;

[0023] Figure 5C This is a schematic diagram of another image processing system displaying fused video frames according to Embodiment 1 of this application;

[0024] Figure 5D This is a schematic diagram of another image processing system displaying fused video frames according to Embodiment 1 of this application;

[0025] Figure 6 This is a schematic diagram of the image processing apparatus according to Embodiment 2 of this application; and

[0026] Figure 7 This is a schematic diagram of the image processing apparatus according to Embodiment 3 of this application. Detailed Implementation

[0027] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this disclosure.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] Example 1

[0030] According to this embodiment, an image processing method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Also, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0031] Figure 1A This is a schematic diagram of the image processing system according to this embodiment. Figure 1B This is another schematic diagram of the image processing system according to this embodiment. (Refer to...) Figure 1A as well as Figure 1B As shown, the system includes: a live streaming device 100, multiple input sources of different types 201-203, a display screen 300, and a live streaming platform 400. The live streaming device 100 is connected to the multiple input sources of different types 201-203 and is used to acquire raw video frames from the input sources 201-203 for generating push video frames. The raw video frames indicate video frames that have not yet undergone video processing.

[0032] The live streaming device 100 is connected to the live streaming platform 400 via the internet, and is used to acquire live video frames from the live streaming platform 400 and send push video frames to the live streaming platform 400. In addition, the live streaming device 100 is also used to fuse the live video frames and the push video frames to generate a merged video frame. The display screen 300 is connected to the live streaming device 100 and is used to display the merged video frame.

[0033] Further, refer to Figure 1B As shown, multiple different types of input sources 201-203 may include, for example, a first camera 201, a second camera 202, and a mobile phone 204. The first camera 201 and the second camera 202 are connected to deserializer 1 and deserializer 2 respectively via serial cables, so that deserializer 1 and deserializer 2 can deserialize the video image data transmitted by the first camera 201 and the second camera 202, and then transmit it to the processor of the live streaming device 100.

[0034] Furthermore, mobile phone 204 establishes a communication connection with the processor of live streaming device 100 via a mobile communication circuit, allowing the processor to receive video image data transmitted by mobile phone 204. Live streaming platform 400 connects to the processor of live streaming device 100 via a network communication circuit, allowing the processor to obtain live video data from live streaming platform 400. (The last sentence appears to be incomplete and unrelated to the preceding text.) Figure 2 Despite the limitations shown, mobile phone 204 can also communicate with the processor of live streaming device 100 via network communication circuit.

[0035] Furthermore, the display screen 300 is connected to the processor of the live streaming device 100 via an HDMI interface.

[0036] In addition, the live streaming equipment 100 and the display screen 300 are positioned at the broadcaster's position. The broadcaster's position is the location of the broadcaster during the live stream. The broadcaster then operates the live streaming equipment 100 to conduct the live stream. It is worth noting that the display screen 300 is positioned in front of the broadcaster's position (relative to the broadcaster) and faces the broadcaster's position.

[0037] It should be noted that the live streaming device 100 in the system is compatible with the hardware structure described above.

[0038] Under the above operating environment, according to the first aspect of this embodiment, an image processing method is provided, the method comprising: Figure 1A and Figure 1B The live streaming device 100 shown is implemented. Figure 2 A flowchart illustrating the method is shown in Figure 3. The method includes:

[0039] S202: Obtain live video frames sent by the live streaming platform, wherein the live video frames are generated based on the push video frames sent by the live streaming device to the live streaming platform;

[0040] S204: Generate a merged video frame by fusing live video frames and push video frames; and

[0041] S206: Transmit the merged video frames to the display screen, which is deployed at the live broadcast site where the live broadcast equipment is located.

[0042] Specifically, Figure 3A This is a modular schematic diagram of an image processing method according to an embodiment of this application. Figure 3B This is a modular schematic diagram of another image processing method according to an embodiment of this application. (Reference) Figure 3A and Figure 3B As shown, the live streaming device 100 runs a video processing module, an image fusion module, and a live streaming client. First, the video processing module of the live streaming platform 100 obtains the original video frames corresponding to the media files from a database or video capture device, and processes the original video frames to generate push video frames. The live streaming device 100 stores media files, such as video files 1-3 and PPT files 1-3. Then, the video processing module sends the push video frames to the live streaming client, thereby the live streaming device 100 sends the push video frames to the live streaming platform 400 via the live streaming client and a wireless network.

[0043] Furthermore, when the live streaming platform 400 generates live video frames based on pushed video frames, the live streaming device obtains the live video frames via a viewing client and a wireless network (S302). In this application, the viewing client can be installed in a terminal device that is communicatively connected to the live streaming device 100, or it can be directly installed in the live streaming device 100. (Reference) Figure 3A As shown, when the viewing client (i.e., the first client program) is set on a terminal device that is communicatively connected to the live streaming device 100, the image fusion module acquires live video frames from the live streaming platform 400 through the viewing client (i.e., the first client program) set on the terminal device. (Reference) Figure 3B As shown, when the viewing client (i.e., the second client program) is set on the live streaming device 100, the image fusion module directly acquires the live video frame corresponding to the live streaming platform 400 through the viewing client (i.e., the second client program). That is, regardless of which method is used, the image fusion module in the live streaming device 100 can acquire the live video frame corresponding to the live streaming platform 400.

[0044] also, Figure 4A This is a schematic diagram of pushing video frames according to an embodiment of this application. Figure 4B This is a schematic diagram of a live video frame according to an embodiment of this application. (Reference) Figure 4A and Figure 4B As shown, there are certain differences in the content contained in push video frames and live video frames. These differences include, but are not limited to, live video frames containing viewer comments while push video frames do not contain viewer comments, and live video frames containing the number of online users while push video frames do not contain the number of online users.

[0045] Furthermore, when the image fusion module acquires the live video frames sent by the live streaming platform 400, the image fusion module generates a fused video frame based on the live video frames and the pushed video frames (S304). In this embodiment, the fused video frame may be, for example, the display area that differs between the live video frame and the pushed video frame, and the fused display area with the pushed video frame to generate the fused video frame. The fused video frame may also be a split-screen display of the live video frame and the pushed video frame. That is, the fused video frame is displayed on a display screen 300, and the screen displayed on the display screen 300 includes both the information contained in the live video frame and the information contained in the pushed video frame. The above will be described in detail later, so it will not be repeated here.

[0046] Finally, the image fusion module transmits the fused video frames to the display screen 300 via the HDMI interface, so that the live broadcast personnel can watch and understand the full information by watching the fused video frames displayed on the display screen 300.

[0047] Figure 5A This is a schematic diagram of an image processing system displaying fused video frames according to an embodiment of this application. (Reference) Figure 5A As shown, the live streaming device 100 is deployed at the live streaming site, facing the live streamers, allowing them to control the live video stream by controlling the device. Furthermore, the live streaming device 100 is connected to a first camera 201 and a second camera 202, which, for example, are used to capture the live stream footage corresponding to the live streamers. The live streaming device 100 is also communicatively connected to a mobile phone 204 equipped with a viewing client (i.e., a first client program), enabling the device to capture live video frames corresponding to the live streaming platform 400 via the phone 204. Additionally, a display screen 300 connected to the live streaming device 100 is deployed at the live streaming site. This display screen 300 is used to display merged video frames, which may be, for example, split-screen live video frames and push video frames.

[0048] Figure 5B This is a schematic diagram of another image processing system displaying fused video frames according to an embodiment of this application. Figure 5A The difference is, Figure 5B The image processing system does not include a mobile phone 204 that is communicatively connected to the live streaming device 100. Thus, the live streaming device 100 can, for example, acquire live video frames from the live streaming platform 400 via a viewing client (i.e., a second client program) configured internally.

[0049] Figure 5C This is a schematic diagram of another image processing system displaying fused video frames according to an embodiment of this application. Figure 5A The difference is, Figure 5C The merged video frame displayed on the display screen 300 is a video frame generated by merging the display areas that differ between the live video frame and the push video frame with the push video frame.

[0050] Figure 5D This is a schematic diagram of another image processing system displaying fused video frames according to an embodiment of this application. Figure 5A The difference is, Figure 5D The image processing system does not include a mobile phone 204 that is communicatively connected to the live streaming device 100. Therefore, the live streaming device 100 can, for example, acquire live video frames from the live streaming platform 400 via an internally configured viewing client (i.e., a second client program). Furthermore, Figure 5D The merged video frame displayed on the display screen 300 is a video frame generated by merging the display areas that differ between the live video frame and the push video frame with the push video frame.

[0051] As described in the background section, during actual live broadcasts, broadcasters typically need to simultaneously view the images displayed on two different monitors (i.e., Monitor 1 and Monitor 2). However, viewing two different monitors simultaneously presents several drawbacks: 1. Because the human eye cannot simultaneously focus on the details displayed on two monitors, frequent switching of the broadcaster's gaze between the two monitors may lead to the omission of crucial information. 2. Due to the different image latency of the two monitors, inconsistencies in the broadcaster's words and actions may occur. 3. If the broadcaster's gaze shifts back and forth between the two monitors, viewers may perceive it as a lack of focus.

[0052] In view of this, this application provides an image processing method. Unlike existing technologies where live streamers need to frequently switch their gaze between two different displays to watch live and push-stream videos, the live streamer in this application only needs to view a fused video generated from the live video and the first push-stream video on one display. This fused video includes information from both the live video and the push-stream video frames. That is, the live streamer only needs to view the live stream on one display to obtain all the information. Because the live streamer can focus on the details displayed on one display without frequently switching between two different displays, key information is not missed, and viewers do not perceive the live streamer as unfocused. This solves the technical problem in existing technologies where frequent switching of the live streamer's gaze between two different displays can lead to missed key information, inconsistencies in speech and behavior, and shifting gaze, resulting in a perceived decrease in focus.

[0053] Optionally, the operation of acquiring live video frames includes: acquiring live video frames from a terminal device that is communicatively connected to the live streaming device, wherein the live video frames are received by the terminal device from the live streaming platform through a first client program corresponding to the live streaming device.

[0054] Specifically, refer to Figure 3A , Figure 5A and Figure 5CAs shown, the live streaming device 100 is communicatively connected to a mobile phone 204 equipped with a viewing client (i.e., the first client program), and the viewing client (i.e., the first client program) corresponds to the live streaming platform 400. Therefore, when the live streaming client in the live streaming device 100 sends a push video frame to the live streaming platform 400, the mobile phone 204, communicatively connected to the live streaming device 100, can receive the live video frame corresponding to the push video frame through the viewing client (i.e., the first client program). Further, the image fusion module in the live streaming device 100 obtains the live video frame from the viewing client (i.e., the first client program) in the mobile phone 204 and fuses the live video frame with the push video frame to generate a fused video frame.

[0055] Optionally, the operation of acquiring live video frames includes: receiving live video frames from the live streaming platform through a second client program corresponding to the live streaming device, wherein the second client program is deployed on the live streaming device.

[0056] Specifically, refer to Figure 3B , Figure 5B and Figure 5D As shown, the live streaming device 100 includes not only a live streaming client but also a viewing client (i.e., a second client program). Therefore, when the live streaming client in the live streaming device 100 sends a push video frame to the live streaming platform 400, the image fusion module can directly obtain the live video frame corresponding to the live streaming platform 400 through the viewing client (i.e., the second client program), and fuse the live video frame with the push video frame to generate a fused video frame.

[0057] Thus, the live streaming device 100 can obtain live video frames corresponding to the live streaming platform 400 from the viewing client (i.e., the first client program) of the mobile phone 204 with communication connection, and can also directly obtain live video frames corresponding to the live streaming platform 400 through the viewing client (i.e., the second client program) set inside, achieving the technical effect of providing the necessary foundation for the subsequent generation of fused video frames.

[0058] Optionally, the operation of fusing live video frames and push video frames to generate a fused video frame includes: determining the display areas where the live video frames and push video frames differ; and extracting the display areas from the live video frames and fusing the display areas with the push video frames to generate a fused video frame.

[0059] Specifically, it referenced Figure 5C or Figure 5DAs shown, when the image fusion module acquires both live video frames and push video frames, it determines the display areas where the live video frames and push video frames differ. For example, if the live video frames have an additional display area corresponding to viewer comments compared to the push video frames, then the display area corresponding to viewer comments is the display area where the live video frames and push video frames differ.

[0060] The image fusion module then extracts the display areas that differ from the live video frames and merges these areas with the pushed video frames to generate a fused video frame. For example, the image fusion module extracts the display area corresponding to the viewer's comment from the live video frame, merges this area with the pushed video frame, and generates a fused video frame that displays the viewer's comment.

[0061] Thus, when the fused video frames generated by the image fusion module are displayed on the display screen 300, the live broadcast personnel can obtain the information contained in the live video frames and the information contained in the push video frames based on the fused video frames displayed on the display screen 300.

[0062] Optionally, the operation of merging live video frames and push video frames to generate a merged video frame includes: split-screen display of live video frames and push video frames, and generating a merged video frame.

[0063] Specifically, refer to Figure 5A and Figure 5B As shown, when the image fusion module acquires both live video frames and push video frames, it displays both in a split-screen manner and generates a fused video frame. Specifically, the first area of ​​the display screen 300 displays the image corresponding to the live video frame, and the second area of ​​the display screen 300 displays the image corresponding to the push video frame, thus allowing both live video frames and push video frames (i.e., fused video frames) to be displayed on the display screen 300. Live broadcast operators can obtain information contained in both the live video frames and the push video frames based on the fused video frames displayed on the display screen 300. The first and second areas can be the same size or different.

[0064] As can be seen from the above, by simply watching the fused video frames displayed on a single screen 300, the live streamer can obtain the information contained in the live video frames and the information contained in the push video frames, thus achieving the technical effect of facilitating the live streamer's broadcast.

[0065] Optionally, the operation of transmitting the merged video frames to the display screen includes transmitting the merged video frames to the display screen via an HDMI cable.

[0066] Thus, according to the first aspect of this embodiment, it is possible to enable the live broadcaster's gaze to focus on the details displayed on a single screen without frequently switching between two different screens, thereby preventing the omission of key information and avoiding the perception that the live broadcaster is not focused.

[0067] Furthermore, according to a second aspect of this embodiment, a storage medium is provided. The storage medium includes a stored program, wherein, when the program is executed, a processor performs the method described in any of the above embodiments.

[0068] Thus, according to this embodiment, the live broadcaster's gaze can be focused on the details displayed on a single screen, without the need to frequently switch between two different screens. Therefore, key information will not be missed, and viewers will not perceive the live broadcaster as unfocused.

[0069] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0070] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0071] Example 2

[0072] Figure 6 An image processing apparatus 600 according to this embodiment is shown, which corresponds to the method described according to Embodiment 1. Reference Figure 6As shown, the device 600 includes: a video frame acquisition module 610, used to acquire live video frames sent by the live streaming platform, wherein the live video frames are generated based on the push video frames sent by the live streaming device to the live streaming platform; a video frame fusion module 620, used to fuse the live video frames and the push video frames to generate fused video frames; and a display module 630, used to transmit the fused video frames to a display screen, wherein the display screen is deployed at the live streaming site where the live streaming device is located.

[0073] Optionally, the video frame acquisition module 610 includes: a first video frame acquisition submodule, used to acquire live video frames from a terminal device that is communicatively connected to the live streaming device, wherein the live video frames are received by the terminal device from the live streaming platform through a first client program corresponding to the live streaming device.

[0074] Optionally, the video frame acquisition module 610 includes: a second video frame acquisition submodule, used to receive live video frames from the live streaming platform through a second client program corresponding to the live streaming device, wherein the second client program is deployed on the live streaming device.

[0075] Optionally, the video frame fusion module 620 includes: a display area determination module for determining the display areas where there are differences between the live video frame and the pushed video frame; and a first fused video frame generation module for extracting the display areas from the live video frame and fusion of the display areas with the pushed video frame to generate a fused video frame.

[0076] Optionally, the video frame fusion module 620 is a second fused video frame generation module, used for split-screen display of live video frames and push video frames, and for generating fused video frames.

[0077] Thus, according to this embodiment, the live broadcaster's gaze can be focused on the details displayed on a single screen, without the need to frequently switch between two different screens. Therefore, key information will not be missed, and viewers will not perceive the live broadcaster as unfocused.

[0078] Example 3

[0079] Figure 7 An image processing apparatus 700 according to this embodiment is shown, which corresponds to the method described according to Embodiment 1. Reference Figure 7 As shown, the device 700 includes: a processor 710; and a memory 720 connected to the processor 710, for providing the processor 710 with instructions to process the following steps: acquiring live video frames sent by a live streaming platform, wherein the live video frames are generated based on push video frames sent from the live streaming device to the live streaming platform; fusing the live video frames and the push video frames to generate a fused video frame; and transmitting the fused video frame to a display screen, wherein the display screen is deployed at the live streaming site where the live streaming device is located.

[0080] Optionally, the operation of acquiring live video frames includes: acquiring live video frames from a terminal device that is communicatively connected to the live streaming device, wherein the live video frames are received by the terminal device from the live streaming platform through a first client program corresponding to the live streaming device.

[0081] Optionally, the operation of acquiring live video frames includes: receiving live video frames from the live streaming platform through a second client program corresponding to the live streaming device, wherein the second client program is deployed on the live streaming device.

[0082] Optionally, the operation of fusing live video frames and push video frames to generate a fused video frame includes: determining the display areas where the live video frames and push video frames differ; and extracting the display areas from the live video frames and fusing the display areas with the push video frames to generate a fused video frame.

[0083] Optionally, the operation of merging live video frames and push video frames to generate a merged video frame includes: split-screen display of live video frames and push video frames, and generating a merged video frame.

[0084] Optionally, the operation of transmitting the merged video frames to the display screen includes transmitting the merged video frames to the display screen via an HDMI cable.

[0085] Thus, according to this embodiment, the live broadcaster's gaze can be focused on the details displayed on a single screen, without the need to frequently switch between two different screens. Therefore, key information will not be missed, and viewers will not perceive the live broadcaster as unfocused.

[0086] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0087] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0088] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0089] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0090] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0091] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0092] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An image processing method for a live device, the method comprising: The method comprises: obtaining a live video frame sent by a live platform, wherein the live video frame is generated according to a push video frame sent by a live device to the live platform; fusing to generate a fused video frame based on the live video frame and the push video frame, wherein there is a difference between the live video frame and the push video frame, and wherein the specific operation comprises: determining a display area in which the live video frame and the push video frame differ; extracting the display area in the live video frame and fusing the display area with the push video frame to generate the fused video frame; and transmitting the fused video frame to a display screen, wherein the display screen is deployed at a live site where the live device is located, and wherein the operation of obtaining the live video frame comprises: obtaining the live video frame from a terminal device in communication connection with the live device, wherein the live video frame is received by the terminal device from the live platform through a first client program corresponding to the live device, or the operation of obtaining the live video frame comprises: receiving the live video frame from the live platform through a second client program corresponding to the live device, wherein the second client program is deployed on the live device. The operation of fusing to generate a fused video frame based on the live video frame and the push video frame comprises:

2. The method of claim 1, wherein, split-screen displaying the live video frame and the push video frame and generating the fused video frame. The operation of transmitting the fused video frame to the display screen comprises: transmitting the fused video frame to the display screen through an HDMI cable.

3. The method of claim 1, wherein, The method comprises:

4. An image processing system, characterized by a live device and a display screen, wherein the display screen faces a live person device, and the live device is configured to perform the following operations: obtaining a live video frame sent by a live platform, wherein the live video frame is generated according to a push video frame sent by a live device to the live platform; fusing to generate a fused video frame based on the live video frame and the push video frame, wherein there is a difference between the live video frame and the push video frame, and wherein the specific operation comprises: determining a display area in which the live video frame and the push video frame differ; and extracting the display area in the live video frame and fusing the display area with the push video frame to generate the fused video frame; and transmitting the fused video frame to a display screen, wherein the display screen is deployed at a live site where the live device is located, and wherein the operation of obtaining the live video frame comprises: obtaining the live video frame from a terminal device in communication connection with the live device, wherein the live video frame is received by the terminal device from the live platform through a first client program corresponding to the live device, or the operation of obtaining the live video frame comprises: receiving the live video frame from the live platform through a second client program corresponding to the live device, wherein the second client program is deployed on the live device. ​ 5. A storage medium, characterized by The storage medium comprises a stored program, wherein the method of any one of claims 1 to 3 is executed by a processor when the program is running.

6. An image processing apparatus characterized by comprising: Comprise: The video frame acquisition module is used for acquiring the live video frame sent by the live platform, wherein the live video frame is generated according to the push video frame sent by the live device to the live platform; The video frame fusion module is based on the live video frame and the push video frame, and the fusion video frame is generated by fusion, wherein there is a difference between the live video frame and the push video frame, wherein the video frame fusion module comprises: a display area determination module, which is used for determining the display area of the live video frame and the push video frame; and The first fusion video frame generation module is used for extracting the display area in the live video frame, and fusing the display area with the push video frame to generate the fusion video frame; and The display screen display module is used for transmitting the fusion video frame to the display screen, wherein the display screen is deployed in the live scene where the live device is located, and wherein the video frame acquisition module comprises: The first video frame acquisition submodule is used for acquiring the live video frame from the terminal device in communication connection with the live device, wherein the live video frame is received by the terminal device from the live platform through the first client program corresponding to the live device, or the video frame acquisition module comprises: The second video frame acquisition submodule is used for receiving the live video frame from the live platform through the second client program corresponding to the live device, wherein the second client program is deployed in the live device.

7. An image processing apparatus characterized by comprising: Comprise: The processor; And The memory is connected with the processor, and is used for providing the processor with instructions for processing the following processing steps: Acquire the live video frame sent by the live platform, wherein the live video frame is generated according to the push video frame sent by the live device to the live platform; Based on the live video frame and the push video frame, the fusion video frame is generated by fusion, wherein there is a difference between the live video frame and the push video frame, wherein the specific operation comprises: Determine the display area of the live video frame and the push video frame; and Extract the display area in the live video frame, and fuse the display area with the push video frame to generate the fusion video frame; and The fusion video frame is transmitted to the display screen, wherein the display screen is deployed in the live scene where the live device is located, and wherein the operation of acquiring the live video frame comprises: The first video frame acquisition submodule is used for acquiring the live video frame from the terminal device in communication connection with the live device, wherein the live video frame is received by the terminal device from the live platform through the first client program corresponding to the live device, or the video frame acquisition module comprises: The second video frame acquisition submodule is used for receiving the live video frame from the live platform through the second client program corresponding to the live device, wherein the second client program is deployed in the live device.

Images