Method and device for intelligent screen recording of high-resolution live broadcast picture

Abstract

The invention discloses an intelligent screen recording method and equipment for a high-resolution live broadcast picture, and the method comprises the steps: obtaining the camera information of a terminal after determining the start of screen recording, and enabling the camera information to comprise any one or more of the following information: the position of a camera, the serial number of the camera, the type of the camera, and the pixel of the camera; and acquiring first screen data corresponding to a screen picture of the mobile equipment at a preset first frame frequency by using an application program interface. According to the screen recording method and device, after it is determined that screen recording is started, the camera information of the terminal is firstly obtained, then the camera information is used for controlling the target camera to be started, and the picture collected by the target camera is displayed in the preset window on the terminal screen, so that the terminal screen comprises the interface of the application to be subjected to screen recording and the picture collected by the target camera.

Description

Technical Field

[0001] The present invention relates to the technical field of screen recording methods, and more specifically, to a method and device for intelligently recording high-resolution live broadcast images. Background Art

[0002] With the development of Internet technology, more and more people have started to pay attention to webcasting. Webcasting is a way of information network publishing with a two-way communication process, which synchronously produces and publishes information at the scene as the event occurs and develops. Currently, screen recording is usually performed on a playback terminal (such as a set-top box), and the recording is carried out from the playback terminal. After the video is downloaded to the playback terminal, it is downloaded, played, and recorded at the same time. This recording solution requires high hardware resources of the playback terminal. In addition, the traditional screen recording method is a recording method that is consistent with the time sequence of video playback. However, for live video, users often have a recording requirement for the video that has already been played. The traditional method requires dragging the progress bar back to the specified position, and while playing back, the pre-recorded video stream is played, recorded, and edited for the second time, which is complex in operation and consumes a large amount of resources. Summary of the Invention

[0003] (1) Technical Problems to be Solved

[0004] In view of the problems existing in the prior art, the present invention provides a method and device for intelligently recording high-resolution live broadcast images to solve the technical problems mentioned in the background art.

[0005] (2) Technical Solutions

[0006] To achieve the above object, the present invention provides the following technical solution: A method for intelligently recording high-resolution live broadcast images, comprising the following steps:

[0007] Step 1: After determining the start of screen recording, obtain the camera information of the terminal. Among them, the camera information includes any one or more of the following information: the position of the camera, the serial number of the camera, the model, and the pixel. Use the application programming interface Opengl ES to collect the first screen data corresponding to the screen image of the mobile device at a preset first frame rate;

[0008] Step 2: Receive the mobile terminal identifier sent by the mobile terminal; obtain the playback terminal identifier pre-bound to the mobile terminal identifier, and obtain the channel identifier of the playback terminal corresponding to the playback terminal identifier; wherein, obtaining the channel identifier of the playback terminal corresponding to the playback terminal identifier includes: sending a channel identifier acquisition instruction to the playback terminal corresponding to the playback terminal identifier, obtaining the channel identifier that the playback terminal is currently playing, encoding the collected first screen data to obtain the encoded first video data; encapsulating the first video data into a first RTMP data packet.

[0009] Step 3: Perform screen recording on the video corresponding to the channel identifier; before performing screen recording on the video corresponding to the channel identifier, it further includes: obtaining the screen recording time and the screen recording duration, opening the live broadcast software for screen recording live broadcast, and switching to the main live broadcast interface, where the main live broadcast interface is used to play the live broadcast content; during the live broadcast process, the control device of the system controls the main live broadcast interface to play any one of the first shooting content of the front camera, the second shooting content of at least one of the multiple cameras, at least one of the multiple pre-playback files, and the content information pre-recorded by the multiple cameras.

[0010] Step 4: When the control device controls the main live broadcast interface to play at least one of the multiple pre-playback files, it specifically includes: obtaining the multiple pre-playback files and numbering the multiple pre-playback files; after opening the live broadcast software for screen recording live broadcast, switching to the main live broadcast interface; obtaining the switching instruction for the user to switch the pre-playback file number, and playing the pre-playback file corresponding to the corresponding number to the main live broadcast interface.

[0011] Step 5: Use the camera information to control the activation of multiple target cameras, where the target cameras are determined according to the type of the application to be screen-recorded and / or the type of the terminal; display the different pictures collected by the multiple target cameras in multiple preset windows on the terminal screen, so that the terminal screen includes both the interface of the application to be screen-recorded and the different pictures collected by the multiple target cameras; wherein, the size ratio of the preset windows is determined by the type of the application to be screen-recorded and the picture content collected by the target cameras.

[0012] The present invention is further configured to create the virtual screen; use the application programming interface Opengl ES to collect the first screen data from the virtual screen at a preset first frame rate.

[0013] The present invention is further configured such that when the user selects to use the mobile device for live streaming, the first RTMP data packet is sent to the content delivery network server; when the user selects to use the terminal computer for live streaming, the first RTMP data packet is sent to the terminal computer.

[0014] The present invention is further configured such that the screen recording of the video corresponding to the channel identifier includes: after receiving the user's personalized screen recording instruction, verifying according to the mobile terminal identifier to obtain the user's personalized label; extracting the features of the video screen and uniformly encoding and identifying them with the feature library feature labels to obtain the feature labels corresponding to the video; wherein, the feature library feature labels are generated by uniformly encoding the collected feature labels in advance; matching the user's personalized label with the feature labels corresponding to the video; if the number of matching personalized labels and feature labels reaches a preset number, calibrating the screen recording duration according to the feature labels, and performing screen recording on the video corresponding to the channel identifier according to the calibrated screen recording duration; if the number of matching personalized labels and feature labels does not reach the preset number, performing screen recording on the video corresponding to the channel identifier according to the screen recording time and the screen recording duration.

[0015] The present invention is further configured such that obtaining the playback terminal identifier pre-bound to the mobile terminal identifier and obtaining the channel identifier of the playback terminal corresponding to the playback terminal identifier includes: searching for the playback terminal identifier pre-bound to the mobile terminal identifier; obtaining the channel identifier fed back by the playback terminal corresponding to the playback terminal identifier, or searching for the channel identifier played by the playback terminal corresponding to the playback terminal identifier.

[0016] The present invention is further configured such that using the camera information to control the target camera to turn on includes: calling the camera startup function to control the camera corresponding to the target camera information to turn on.

[0017] The present invention is further configured such that before using the camera information to control the target camera to turn on, it further includes: determining the target camera information according to the type of the terminal, the identifier of the application to be screen-recorded, and / or the user's settings.

[0018] The present invention is further configured such that before displaying the image captured by the target camera in a preset window on the terminal screen, it further includes: creating the preset window on the terminal screen according to a preset rule.

[0019] The present invention further includes a device applying the method for intelligent screen recording of high-resolution live streaming images described above.

[0020] (III) Beneficial effects

[0021] Compared with the prior art, the present invention provides a method and device for intelligent screen recording of high-resolution live broadcast images, having the following beneficial effects:

[0022] After determining the start of screen recording, the present invention first obtains the camera information of the terminal, and then uses the camera information to control the target camera to turn on, and displays the image captured by the target camera in a preset window on the terminal screen, so that the terminal screen includes both the interface of the application to be screen-recorded and the image captured by the target camera. Thus, the obtained screen recording file includes both the interface of the application to be screen-recorded and the image captured by the camera, enriching the information content of the screen recording file, meeting the needs of users for different uses of the screen recording file, improving the user experience, using the mobile terminal to control the cloud server to play the screen recording in the playback device, without the need for the playback terminal to perform screen recording, not occupying the resources of the playback terminal, and being more convenient to share through the mobile terminal after screen recording. In addition, by controlling the recording method of the cloud server, compared with the prior art, the two-step operation of first playing back and then recording the screen recording is simplified to one-step operation, and compared with the traditional real-time screen recording, it can better meet the limitation that the past screen recording of the live broadcast image does not need to be played back first and then recorded, and can directly trace back from the current time to the past image in reverse time and complete the screen recording, simplifying the user operation, and further improving the user experience of screen recording. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is the overall flowchart of a method and device for intelligent screen recording of high-resolution live broadcast images. DETAILED DESCRIPTION OF THE INVENTION

[0024] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other. The present invention will be described in detail below with reference to the drawings and in conjunction with the embodiments.

[0025] It should be pointed out that, unless otherwise specified, all technical and scientific terms used in the present application have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present application belongs.

[0026] Please refer to Figure 1 , a method for intelligent screen recording of high-resolution live broadcast images, comprising the following steps:

[0027] Step 1: After determining the start of screen recording, obtain the camera information of the terminal, wherein the camera information includes any one or more of the following information: the position of the camera, the serial number of the camera, the model and the pixel, and use the application programming interface Opengl ES to collect the first screen data corresponding to the screen image of the mobile device at a preset first frame rate;

[0028] Step 2: Receive the mobile terminal identifier sent by the mobile terminal; obtain the playback terminal identifier pre-bound to the mobile terminal identifier, and obtain the channel identifier of the playback terminal corresponding to the playback terminal identifier; wherein, obtaining the channel identifier of the playback terminal corresponding to the playback terminal identifier includes: sending a channel identifier acquisition instruction to the playback terminal corresponding to the playback terminal identifier, obtaining the channel identifier that the playback terminal is currently playing, encoding the collected first screen data to obtain encoded first video data; encapsulating the first video data into a first RTMP data packet.

[0029] Step 3: Perform screen recording on the video corresponding to the channel identifier; before performing screen recording on the video corresponding to the channel identifier, it further includes: obtaining the screen recording time and the screen recording duration, opening the live broadcast software for screen recording live broadcast, and switching to the main live broadcast interface, where the main live broadcast interface is used to play the live broadcast content; during the live broadcast process, the control device of the system controls the main live broadcast interface to play any one of the first shooting content of the front camera, the second shooting content of at least one of the multiple cameras, at least one of the multiple pre-playback files, and the content information pre-recorded by the multiple cameras.

[0030] Step 4: When the control device controls the main live broadcast interface to play at least one of the multiple pre-playback files, it specifically includes: obtaining multiple pre-playback files, and numbering the multiple pre-playback files; after opening the live broadcast software for screen recording live broadcast, switching to the main live broadcast interface; obtaining the switching instruction for the user to switch the pre-playback file number, and playing the pre-playback file corresponding to the corresponding number to the main live broadcast interface.

[0031] Step 5: Use the camera information to control the activation of multiple target cameras, where the target cameras are determined according to the type of the application to be screen-recorded and / or the type of the terminal; display the different images collected by the multiple target cameras in multiple preset windows on the terminal screen, so that the terminal screen includes both the interface of the application to be screen-recorded and the different images collected by the multiple target cameras; wherein, the size ratio of the preset windows is determined by the type of the application to be screen-recorded and the image content collected by the target cameras.

[0032] In a further embodiment of the present invention, create the virtual screen; use the application programming interface Opengl ES to collect the first screen data from the virtual screen at a preset first frame rate.

[0033] In a further embodiment of the present invention, when the user selects to use the mobile device for live streaming, the first RTMP data packet is sent to the content delivery network server; when the user selects to use the terminal computer for live streaming, the first RTMP data packet is sent to the terminal computer.

[0034] In a further embodiment of the present invention, the screen recording of the video corresponding to the channel identifier includes: after receiving the user's personalized screen recording instruction, verifying according to the mobile terminal identifier to obtain the user's personalized label; extracting the features of the video screen and uniformly encoding and identifying them with the feature library feature labels to obtain the feature labels corresponding to the video; wherein, the feature library feature labels are generated by uniformly encoding the collected feature labels in advance; matching the user's personalized label with the feature labels corresponding to the video; if the number of identical matches between the personalized label and the feature labels reaches a preset number, calibrating the screen recording duration according to the feature labels, and performing screen recording on the video corresponding to the channel identifier according to the calibrated screen recording duration; if the number of identical matches between the personalized label and the feature labels does not reach the preset number, performing screen recording on the video corresponding to the channel identifier according to the screen recording time and the screen recording duration.

[0035] Embodiment 2:

[0036] A method for intelligent screen recording of high-resolution live broadcast images, comprising the following steps:

[0037] Step 1: After determining the start of screen recording, obtain the camera information of the terminal, wherein the camera information includes any one or more of the following information: the position of the camera, the serial number of the camera, the model and the pixel, and use the application programming interface Opengl ES to collect the first screen data corresponding to the screen image of the mobile device at a preset first frame rate;

[0038] Step 2: Receive the mobile terminal identifier sent by the mobile terminal; obtain the playback terminal identifier pre-bound with the mobile terminal identifier, and obtain the channel identifier of the playback terminal corresponding to the playback terminal identifier; wherein, obtaining the channel identifier of the playback terminal corresponding to the playback terminal identifier includes: sending a channel identifier acquisition instruction to the playback terminal corresponding to the playback terminal identifier, obtaining the channel identifier currently being played by the playback terminal, encoding the collected first screen data to obtain the encoded first video data; encapsulating the first video data into a first RTMP data packet;

[0039] Step 3: Take a screen recording of the video corresponding to the channel identifier; before taking a screen recording of the video corresponding to the channel identifier, it further includes: obtaining the screen recording time and duration, opening the live broadcast software for screen recording live broadcast, and switching to the main live broadcast interface, where the main live broadcast interface is used to play the live broadcast content; during the live broadcast, the control device of the system controls the main live broadcast interface to play any one of the first shooting content of the front camera, the second shooting content of at least one of the multiple cameras, at least one of the multiple pre-playback files, and the content information pre-recorded by the multiple cameras;

[0040] Step 4: When the control device controls the main live broadcast interface to play at least one of the multiple pre-playback files, it specifically includes: obtaining the multiple pre-playback files, and numbering the multiple pre-playback files; after opening the live broadcast software for screen recording live broadcast, switching to the main live broadcast interface; obtaining the switching instruction for the user to switch the pre-playback file number, and playing the pre-playback file corresponding to the corresponding number to the main live broadcast interface;

[0041] Step 5: Using the camera information, control multiple target cameras to turn on, where the target cameras are determined according to the type of the application to be screen-recorded and / or the type of the terminal; respectively display the different pictures collected by the multiple target cameras in multiple preset windows on the terminal screen, so that the terminal screen includes both the interface of the application to be screen-recorded and the different pictures collected by the multiple target cameras; where the size ratio of the preset windows is determined by the type of the application to be screen-recorded and the picture content collected by the target cameras.

[0042] In a further embodiment of the present invention, the obtaining the playback terminal identifier pre-bound with the mobile terminal identifier and obtaining the channel identifier of the playback terminal corresponding to the playback terminal identifier includes: searching for the playback terminal identifier pre-bound with the mobile terminal identifier; obtaining the channel identifier fed back by the playback terminal corresponding to the playback terminal identifier, or searching for the channel identifier played by the playback terminal corresponding to the playback terminal identifier; the using the camera information to control the target camera to turn on includes: calling the camera startup function to control the camera corresponding to the target camera information to turn on; before using the camera information to control the target camera to turn on, it further includes: determining the target camera information according to the type of the terminal, the identifier of the application to be screen-recorded, and / or the user's settings; before displaying the picture collected by the target camera in the preset window on the terminal screen, it further includes: creating the preset window on the terminal screen according to the preset rules.

[0043] The present invention further includes a device applying the method for intelligent screen recording of high-resolution live broadcast pictures as described above.

[0044] Although embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for intelligent screen recording of high-resolution live broadcast images, characterized in that, Including the following steps: Step 1: After determining the start of screen recording, obtain the camera information of the terminal. The camera information includes any one or more of the following: the position of the camera, the serial number of the camera, the model, and the pixel count. Use the OpenGL ES application programming interface to collect the first screen data corresponding to the screen image of the mobile device at a preset first frame rate; Step 2: Receive the mobile terminal identifier sent by the mobile terminal; obtain the playback terminal identifier pre-bound to the mobile terminal identifier, and obtain the channel identifier of the playback terminal corresponding to the playback terminal identifier. Among them, obtaining the channel identifier of the playback terminal corresponding to the playback terminal identifier includes: sending a channel identifier acquisition instruction to the playback terminal corresponding to the playback terminal identifier, obtaining the channel identifier that the playback terminal is currently playing, encoding the collected first screen data to obtain the encoded first video data; encapsulating the first video data into a first RTMP data packet; Step 3: Perform screen recording on the video corresponding to the channel identifier. Before performing screen recording on the video corresponding to the channel identifier, it also includes: obtaining the screen recording time and the screen recording duration, opening the live broadcast software for live screen recording, and switching to the main live broadcast interface, where the main live broadcast interface is used to play the live broadcast content; during the live broadcast process, the control device of the system controls the main live broadcast interface to play any one of the first shooting content of the front camera, the second shooting content of at least one of the multiple cameras, at least one of the multiple pre-playback files, and the content information pre-recorded by the multiple cameras; Step 4: When the control device controls the main live broadcast interface to play at least one of the multiple pre-playback files, it specifically includes: obtaining the multiple pre-playback files and numbering the multiple pre-playback files; after opening the live broadcast software for live screen recording, switching to the main live broadcast interface; obtaining the switching instruction for the user to switch the pre-playback file number, and playing the pre-playback file corresponding to the corresponding number to the main live broadcast interface; Step 5: Use the camera information to control the activation of multiple target cameras, where the target cameras are determined according to the type of the application to be screen-recorded and / or the type of the terminal; display the different images collected by the multiple target cameras in multiple preset windows on the terminal screen, so that the terminal screen includes both the interface of the application to be screen-recorded and the different images collected by the multiple target cameras; among them, the size ratio of the preset windows is determined by the type of the application to be screen-recorded and the image content collected by the target cameras.

2. A method for intelligent screen recording of high-resolution live broadcast images according to claim 1, characterized in that: Create the virtual screen; use the OpenGL ES application programming interface to collect the first screen data from the virtual screen at a preset first frame rate.

3. A method for intelligent screen recording of high-resolution live broadcast images according to claim 2, characterized in that: In the case where the user selects to use the mobile device for live broadcast, send the first RTMP data packet to the content delivery network server; In the case where the user selects to use the terminal computer for live broadcast, send the first RTMP data packet to the terminal computer.

4. A method for intelligent screen recording of high-resolution live broadcast images according to claim 1, characterized in that: The screen recording of the video corresponding to the channel identifier includes: after receiving a user personalized screen recording instruction, verifying according to the mobile terminal identifier to obtain the user's personalized label; extracting the features of the video picture and uniformly encoding and identifying them with the feature library feature labels to obtain the feature labels corresponding to the video, wherein the feature library feature labels are generated by uniformly encoding the collected feature labels in advance; matching the user's personalized label with the feature labels corresponding to the video; if the number of matching personalized labels and feature labels reaches a preset number, calibrating the screen recording duration according to the feature labels, and performing screen recording on the video corresponding to the channel identifier according to the calibrated screen recording duration; if the number of matching personalized labels and feature labels does not reach the preset number, performing screen recording on the video corresponding to the channel identifier according to the screen recording time and the screen recording duration.

5. A method for intelligent screen recording of high-resolution live broadcast images according to claim 1, characterized in that: The obtaining of the playback terminal identifier pre-bound with the mobile terminal identifier and the obtaining of the channel identifier of the playback terminal corresponding to the playback terminal identifier include: searching for the playback terminal identifier pre-bound with the mobile terminal identifier; obtaining the channel identifier fed back by the playback terminal corresponding to the playback terminal identifier, or searching for the channel identifier played by the playback terminal corresponding to the playback terminal identifier.

6. A method for intelligent screen recording of high-resolution live broadcast images according to claim 1, characterized in that: The using of the camera information to control the target camera to turn on includes: calling the camera startup function to control the camera corresponding to the target camera information to turn on.

7. A method for intelligent screen recording of high-resolution live broadcast images according to claim 1, characterized in that: Before the using of the camera information to control the target camera to turn on, it further includes: determining the target camera information according to the type of the terminal, the identifier of the application to be screen recorded, and / or the user's settings.

8. A method for intelligent screen recording of high-resolution live broadcast images according to claim 1, characterized in that: Before the displaying of the picture captured by the target camera in the preset window on the terminal screen, it further includes: creating the preset window on the terminal screen according to the preset rules.

9. The present invention further includes a device applying the method of intelligent screen recording of high-resolution live broadcast pictures described above.

Images