Camera horizontal and vertical screen adaptive network streaming media real-time output system

Abstract

The invention discloses a camera horizontal and vertical screen adaptive network streaming media real-time output system, and belongs to the field of video processing. The direction sensor can detect the rotation angle of the camera in real time, and the video processing module rotates or cuts the video signal according to the detected direction data so as to adapt to different screen directions. And the streaming media coding module is used for coding and compressing the processed video signal to generate streaming media data conforming to a network transmission standard. And the network transmission module is responsible for transmitting the streaming media data to target equipment or a server in real time. The invention designs an efficient, flexible and reliable video camera horizontal and vertical screen adaptive network streaming media real-time output system, can be widely applied to multiple fields of online education, remote monitoring, live broadcast entertainment and the like, and provides a more convenient and efficient solution for video transmission.

Description

Technical Field

[0001] The invention relates to the technical field of video processing, and in particular to a camera horizontal and vertical screen adaptive network streaming media real-time output system. Background Art

[0002] With the rapid development of mobile Internet, the short video and live broadcast industries have experienced explosive growth. However, traditional cameras only output videos in a single direction (horizontal), which limits their flexibility and user experience in many application scenarios of mobile Internet. For example, when watching vertical videos on mobile devices, the horizontal videos shot by traditional cameras need to be rotated or cropped, resulting in reduced image quality or information loss. Vertical shooting requires the camera to be rotated 90° with a specific bracket or manually flipped 90° for shooting, which is inconvenient to operate. Therefore, it is particularly important to develop a system that can adaptively switch between horizontal and vertical screens of cameras and output network streaming media in real time. Summary of the invention

[0003] The present invention provides a camera horizontal and vertical screen adaptive network streaming media real-time output system, the system can intelligently analyze the camera posture, automatically adjust the video output format according to the camera shooting direction, realize seamless switching between horizontal and vertical screens, and ensure the real-time and high quality of network streaming media, perfectly compatible with traditional video shooting and new media vertical screen shooting requirements, and aims to solve the problem of incompatible video output of traditional network cameras when switching between horizontal and vertical screens.

[0004] The embodiment of the present invention provides a camera horizontal and vertical screen adaptive network streaming media real-time output system, comprising:

[0005] The camera module has a built-in direction sensor, which is used to collect video signals and detect the rotation angle of the camera in real time;

[0006] A video processing module, used to rotate or crop the collected video signal according to the rotation angle to adapt to different screen directions;

[0007] The streaming media encoding module is used to encode and compress the processed video signal to generate streaming media data;

[0008] The network transmission module is used to transmit the streaming media data to the target device in real time.

[0009] Optionally, in one embodiment of the present invention, the system further includes:

[0010] The control module is used to provide a user interface and a control interface so that the user can manually or automatically switch the video output format or adjust the camera parameters according to the needs.

[0011] Optionally, in an embodiment of the present invention, the camera module uses a CMOS image sensor. The zoom lens of the camera and the PCB board with a sensor are combined into a camera module in an integrated design. The camera module is connected to the internal rotating device of the camera in a screwed form and positioned in the form of a pin at the same time. The rotating device is connected to the camera mainframe frame.

[0012] Optionally, in an embodiment of the present invention, the PCB board inside the camera module integrates an inertial measurement unit to obtain the attitude information of the camera module.

[0013] A camera horizontal and vertical screen adaptive network streaming media real-time output system according to an embodiment of the present invention has the following beneficial effects:

[0014] Strong adaptability: It can automatically adjust the video output format according to the direction of the output device, realizing seamless switching between horizontal and vertical screens.

[0015] Good real-time performance: It adopts efficient video processing and network transmission technologies to ensure the real-time performance and high quality of streaming media data.

[0016] Good user experience: Whether the user uses a horizontal screen device or a vertical screen device, they can obtain the best video viewing experience.

[0017] Wide application range: It is applicable to various scenarios requiring real-time video transmission, such as online education, remote monitoring, live entertainment, etc.

[0018] The additional aspects and advantages of the present invention will be partly given in the following description, partly will become obvious from the following description, or will be understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and / or additional aspects and advantages of the present invention will become obvious and easy to understand from the following description of the embodiments in conjunction with the drawings, where:

[0020] Figure 1 FIG. is a schematic structural diagram of a camera horizontal and vertical screen adaptive network streaming media real-time output system according to an embodiment of the present invention;

[0021] Figure 2 FIG. is a schematic diagram of the camera in the horizontal screen shooting state according to an embodiment of the present invention;

[0022] Figure 3 FIG. is a schematic diagram of the camera in the vertical screen shooting state according to an embodiment of the present invention;

[0023] Figure 4 FIG. is a flowchart of a camera horizontal and vertical screen adaptive network streaming media real-time output system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] Embodiments of the present invention will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are intended to explain the present invention, and should not be construed as a limitation of the present invention.

[0025] Figure 1 It is a schematic structural diagram of a camera horizontal and vertical screen adaptive network streaming media real-time output system provided according to an embodiment of the present invention.

[0026] As Figure 1 shown, the camera horizontal and vertical screen adaptive network streaming media real-time output system includes:

[0027] A camera module, which is built-in with a direction sensor and is used to collect video signals while real-time detecting the rotation angle of the camera. A video processing module, which is used to rotate or crop the collected video signals according to the rotation angle to adapt to different screen directions. A streaming media encoding module, which is used to encode and compress the processed video signals to generate streaming media data that meets the network transmission standard. A network transmission module, which is used to transmit the streaming media data to the target device or server in real time.

[0028] The camera uses a high-performance CMOS image sensor and supports high-definition video recording. The zoom lens of the camera and the PCB board with a sensor are combined in an integrated design to form a camera module. The camera module is connected to the internal rotating device of the camera host in a screwed form and is positioned in the form of a pin at the same time. The rotating device is connected to the camera host frame, providing structural support for the accurate realization of the subsequent horizontal and vertical screen conversion function. The PCB board inside the camera module integrates a Gyro sensor as the direction sensor, which can obtain the attitude information of the camera module covering the angle with extremely high precision and high sensitivity.

[0029] The video processing module receives the original video signal transmitted by the camera module and judges the current device direction according to the rotation data of the direction sensor. The rotation angle is collected by the direction sensor. When the rotation angle is greater than a certain value, it is considered that the direction of the camera has changed (such as from horizontal screen to vertical screen), and then the video signal is automatically rotated or cropped to adapt to the new output format.

[0030] The streaming media encoding module encodes and compresses the processed video signals to generate streaming media data that meets the network transmission standard. This module supports multiple encoding formats (such as H.264, H.265, AV1) and resolution settings (4k, 1920x1080) to meet the needs of different network environments and playback devices.

[0031] The network transmission module is responsible for real-time transmission of the encoded streaming media data to the server or the client. This module adopts advanced network transmission protocols (such as RTSP, RTMP, HLS, etc.) to ensure the stability and real-time nature of the data.

[0032] In an embodiment of the present invention, the system further includes: a control module, which is used to provide a user interface and a control interface, so that the user can manually or automatically switch the video output format or adjust the camera parameters according to requirements.

[0033] The control module: provides a user interface and a control interface, allowing the user to manually or automatically switch the video output format or adjust the relevant parameters as needed. At the same time, the control module is also responsible for monitoring the operating state of the entire system to ensure the stability and reliability of the system.

[0034] When the present invention is in use, the system for real-time output of camera horizontal and vertical screen adaptive network streaming media is as follows:

[0035] System initialization: After starting the system, each module performs initialization settings, including parameter configuration of the camera module, mode selection of the video processing module, encoding format and resolution settings of the streaming media encoding module, etc.

[0036] Video acquisition: The camera module starts to acquire video signals and transmits the original video data to the video processing module.

[0037] Orientation detection and video processing: The video processing module receives the data from the orientation sensor to judge the current device orientation. If the orientation changes, the video signal is rotated or cropped according to the preset rules. The processed video signal is transmitted to the streaming media encoding module again.

[0038] Streaming media encoding and transmission: The streaming media encoding module encodes and compresses the processed video signal to generate streaming media data that meets the network transmission standard. The network transmission module transmits the encoded streaming media data to the target device or server in real time.

[0039] User control and monitoring: The user manually or automatically switches the video output format or adjusts the relevant parameters through the control module as needed. At the same time, the control module monitors the operating state of the entire system to ensure the stability and reliability of the system.

[0040] See Figures 2 to 4 , the specific implementation steps of the system for real-time output of camera horizontal and vertical screen adaptive network streaming media of the present invention are as follows:

[0041] I. Video input and processing

[0042] 1. Video input: Shoot horizontally with a 4K resolution (3840×2160).

[0043] 2. Preprocessing: Decode the input 4K video to obtain the original frame data.

[0044] II. Landscape Video Output

[0045] 1. Bitstream Output: Keep the original 16:9 aspect ratio and output a 4K resolution video bitstream, or perform scaling processing as needed (such as scaling to a resolution suitable for network transmission).

[0046] 2. Composition Unchanged: Ensure that the composition of the landscape video is the same as when shooting, without additional cropping or rotation.

[0047] III. Portrait Video Output

[0048] 1. Human Figure Recognition and Secondary Composition:

[0049] S1, Full-frame Human Figure Recognition:

[0050] In the case of unknown objects, perform full-frame human figure recognition on the video frames.

[0051] Adjust the composition according to the recognized human figure size and position information.

[0052] Preset the protection range (such as the left, right, upper, and lower boundaries) to ensure that the person does not exceed the frame.

[0053] Under the composition requirements of portrait 9:16, the maximum horizontal side is 1215, so the frame size is dynamically adjusted between 1080x1920 and 1215×2160.

[0054] Move the composition left and right according to the person's size to keep the frame beautiful.

[0055] S2, Individual Object Recognition and Cropping:

[0056] When a specific person object is recognized in the frame, select the object.

[0057] Perform individual composition cropping according to the position of the object's head to ensure that the person's head is in a suitable position in the frame.

[0058] 2. Bitstream Output: Output the adjusted portrait video as an independent bitstream.

[0059] Vertical screen output includes secondary composition of the original content within the image and also indicates it in the metadata during the push of the original data. For example, if the transmitted content remains in landscape (3840x2160), the system will write the rotation angle in each frame of the encoded image in the metadata. During the rotation process, the terminal can create a rotation animation based on this angle (or not), and finally, it can determine whether the stable state is in vertical screen based on the angle and then adjust the display and composition ratio.

[0060] IV. Simultaneous output of landscape and vertical screens

[0061] 1. Picture-in-picture composition:

[0062] Use the landscape video as the main screen and embed the vertical screen video as a small screen to form a picture-in-picture effect.

[0063] Adjust the position and size of the small screen to ensure a harmonious overall composition.

[0064] 2. Stitching composition:

[0065] Stitch the landscape video and the vertical screen video left and right or up and down to form a stitching composition.

[0066] Adjust the stitching position and ratio as needed to ensure the continuity of the picture.

[0067] Through the above solutions, the requirement of simultaneously outputting landscape and vertical screen videos from a horizontally shot camera can be achieved, meeting the video processing requirements in various application scenarios.

[0068] It can be understood that the parameters listed in the above embodiments are only used as an example. In actual applications, the specific parameters can be adjusted according to the actual situation, such as adjusting the resolution or the landscape / vertical screen ratio, etc.

[0069] According to the camera landscape / vertical screen adaptive network streaming media real-time output system proposed in the embodiments of the present invention, this system realizes the real-time detection and adaptive adjustment of the camera output direction through a camera module integrated with a direction sensor, a video processing module, a streaming media encoding module, a network transmission module, and a control module. Among them, the direction sensor can detect the rotation angle of the camera in real time, and the video processing module rotates or crops the video signal according to the detected direction data to adapt to different screen directions. The streaming media encoding module encodes and compresses the processed video signal to generate streaming media data that meets the network transmission standard. The network transmission module is responsible for transmitting these streaming media data to the target device or server in real time. The camera landscape / vertical screen adaptive network streaming media real-time output system designed by the present invention is an efficient, flexible, and reliable system that can be widely applied to multiple fields such as online education, remote monitoring, live entertainment, etc., providing a more convenient and efficient solution for video transmission.

[0070] In the description of this specification, the descriptions with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc., mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of this application. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0071] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "N" is at least two, such as two, three, etc., unless otherwise specifically defined.

Claims

1. A camera horizontal and vertical screen adaptive network streaming media real-time output system, characterized in that: include: The camera module has a built-in direction sensor, which is used to collect video signals and detect the rotation angle of the camera in real time; A video processing module, used to rotate or crop the collected video signal according to the rotation angle to adapt to different screen directions; The streaming media encoding module is used to encode and compress the processed video signal to generate streaming media data; The network transmission module is used to transmit the streaming media data to the target device in real time.

2. The system according to claim 1, characterized in that The system further comprises: The control module is used to provide a user interface and a control interface so that the user can manually or automatically switch the video output format or adjust the camera parameters according to the needs.

3. The system according to claim 1, characterized in that The camera module adopts a CMOS image sensor. The zoom lens of the camera and the PCB board with the sensor are integrated into a camera module. The camera module is connected to the internal rotating device of the camera in a screw connection and positioned in a pin form. The rotating device is connected to the camera host frame.

4. The system according to claim 3, characterized in that The PCB board inside the camera module integrates an inertial measurement unit to obtain the attitude information of the camera module.

Images