A screen recording and playback method for multiple displays, computer and readable medium

By acquiring and encoding audio and video data of multiple monitors, generating audio and video files and achieving synchronous playback, the problem of multi-display operation recording and playback is solved, and flexible playback control is provided to meet users' multi-display review needs.

CN114615539BActive Publication Date: 2024-11-05SHANDONG CHAOYUE DATA CONTROL ELECTRONICS CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210318735.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-29
Publication Date
2024-11-05
Estimated Expiration
2042-03-29

AI Technical Summary

Technical Problem

The prior art is difficult to effectively record and playback of operating videos on multiple displays, and cannot meet users' needs for review of multiple displays.

Method used

By obtaining the audio and video data of multiple monitors, encoding and generating audio and video files, playing according to the monitor's timestamp, it supports synchronous playback and independent playback of multiple monitors, and provides playback progress, speed double, pause and exit playback functions.

Benefits of technology

It realizes recording and fixed-point playback of multiple monitor operations, meets the needs of users' multi-display review and provides flexible playback control.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114615539B_ABST
    Figure CN114615539B_ABST
Patent Text Reader

Abstract

The present invention discloses a screen recording and playback method for multiple displays, the method comprising: obtaining m frames of audio and video data of a first display, the m frames of audio and video data corresponding to m display timestamps respectively; encoding the m frames of audio and video data to obtain m frames of coded data corresponding to the m frames of audio and video data respectively; obtaining n frames of audio and video data of a second display, the n frames of audio and video data corresponding to n display timestamps respectively, n equals m, and n is an integer greater than 1; encoding the n frames of audio and video data to obtain n frames of coded data corresponding to the n frames of audio and video data respectively; generating an audio and video file according to the m frames of coded data, the m display timestamps, the n frames of coded data, and the n display timestamps. The present invention also discloses a computer and a computer-readable storage medium. Through the scheme of the present invention, after a user records operations on multiple displays, the video can be played back at fixed points corresponding to multiple displays, which can meet the user's need for reviewing multiple displays.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of video technology, and in particular to a screen recording and playback method for multiple displays, a computer and a readable medium. Background Art

[0002] With the rapid development of computer technology, more and more users are using split-screen technology in their work, life and study. For example, using multiple monitors for extended display, users can operate on multiple monitors separately. How to record the operations on multiple monitors and then play back the video at the corresponding multiple monitors is a technical problem to be solved by those skilled in the art. Summary of the invention

[0003] The present invention proposes a screen recording and playback method for multiple displays, a computer and a computer-readable storage medium. After a user records operations on multiple displays, the video can be played back at fixed points corresponding to the multiple displays, which can meet the user's multi-display review needs.

[0004] Based on the above purpose, an embodiment of the present invention provides a method for replaying screen recordings on multiple displays, which specifically includes the following steps:

[0005] Obtain m frames of audio and video data of the first display, where the m frames of audio and video data correspond to m display timestamps respectively, and m is an integer greater than 1;

[0006] Encode m frames of audio and video data to obtain m frames of encoded data corresponding to the m frames of audio and video data respectively;

[0007] Obtain n frames of audio and video data of the second display, where the n frames of audio and video data correspond to n display timestamps respectively, where n is equal to m and n is an integer greater than 1, and the second display and the first display are extended for display;

[0008] Encode n frames of audio and video data to obtain n frames of encoded data corresponding to the n frames of audio and video data respectively;

[0009] Generate audio and video files according to m frames of coded data, m display timestamps, n frames of coded data, and n display timestamps.

[0010] In some embodiments, after generating the audio and video file according to the m frames of coded data, the m display timestamps, the n frames of coded data, and the n display timestamps, the method further includes:

[0011] The audio and video files are played on the first display and the second display.

[0012] In a specific implementable manner, playing an audio or video file on a first display and a second display includes:

[0013] According to the audio and video file, playing m frames of audio and video data corresponding to the m frames of encoded data on the first display in the time sequence of the m display timestamps;

[0014] According to the audio and video file, n frames of audio and video data corresponding to the n frames of encoded data are played on the second display in the time sequence of the n display timestamps.

[0015] In a specific implementable manner, playing an audio or video file on a first display and a second display includes:

[0016] When playing the audio and video files on the first display and the second display, the playing mode of the audio and video files is controlled.

[0017] In a specific implementation manner, the play mode includes at least one of the following: play progress, play speed, pause play, and exit play.

[0018] In some specific implementations, before acquiring the m frames of audio and video data of the first display, the method further includes:

[0019] Acquire basic information of the first display and the second display, the basic information including identification information;

[0020] Encoding the m frames of audio and video data, including: encoding the m frames of audio and video data according to the identification information of the first display;

[0021] Encoding n frames of audio and video data includes: encoding n frames of audio and video data according to identification information of the second display.

[0022] In a specific implementable manner, the basic information further includes attribute information; generating an audio and video file according to m frames of coded data, m display timestamps, n frames of coded data, and n display timestamps, including:

[0023] An audio and video file is generated according to the attribute information of the first display, the attribute information of the second display, m frames of coded data, m display timestamps, n frames of coded data, and n display timestamps.

[0024] In a specific implementation, the audio and video data includes screenshot image data of a display.

[0025] In another aspect of the embodiments of the present invention, a computer is provided, comprising: a first acquisition unit, a first encoding unit, a second acquisition unit, a second encoding unit and a generation unit, wherein:

[0026] The first acquisition unit is configured to acquire m frames of audio and video data of the first display, the m frames of audio and video data respectively corresponding to m display timestamps, where m is an integer greater than 1;

[0027] The first encoding unit is configured to encode m frames of audio and video data to obtain m frames of encoded data corresponding to the m frames of audio and video data respectively;

[0028] The second acquisition unit is configured to acquire n frames of audio and video data of the second display, the n frames of audio and video data respectively correspond to n display timestamps, n is equal to m, and n is an integer greater than 1, and the second display and the first display are extended for display;

[0029] The second encoding unit is configured to encode n frames of audio and video data to obtain n frames of encoded data corresponding to the n frames of audio and video data respectively;

[0030] The generating unit is configured to generate an audio and video file according to m frames of coded data, m display timestamps, n frames of coded data, and n display timestamps.

[0031] According to another aspect of the embodiments of the present invention, a computer-readable storage medium is provided, which stores a computer program that implements the above method steps when executed by a processor.

[0032] In the embodiment of the present application, an audio and video file is generated according to the m frames of coded data and m display timestamps corresponding to the first display, and the n frames of coded data and n display timestamps corresponding to the second display. In this way, after the user records the operations on multiple displays, the video can be played back at the corresponding multiple displays, which can meet the user's multi-display review needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can be obtained based on these drawings without paying creative work.

[0034] Figure 1 A schematic flow chart of a screen recording and playback method for multiple displays provided by an embodiment of the present invention;

[0035] Figure 2 A schematic diagram of the structure of a computer provided by an embodiment of the present invention;

[0036] Figure 3 Schematic diagram of a computer-readable storage medium provided for embodiments of the present invention. DETAILED DESCRIPTION

[0037] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the embodiments of the present invention are further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0038] It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are for distinguishing two non-identical entities with the same name or non-identical parameters. It can be seen that "first" and "second" are only for the convenience of expression and should not be understood as limitations on the embodiments of the present invention. The subsequent embodiments will not explain this one by one.

[0039] The first aspect of the embodiments of the present invention provides an embodiment of a screen recording playback method for multiple displays. Figure 1 As shown, it includes:

[0040] S100: Obtain basic information of the first display and the second display.

[0041] The first display and the second display perform extended display. Of course, in the embodiment of the present invention, the number of displays can be multiple, and these displays perform extended display. For example, three displays, five displays, etc. The embodiment of the present application is not specifically limited. The embodiment of the present application is described by taking two displays as an example.

[0042] The basic information may include identification information of the first display and identification information of the second display. Of course, the basic information may also include attribute information of the first display and attribute information of the second display. Among them, the attribute information may include the resolution of the display, the display size of the display, etc., which is not specifically limited in the embodiments of the present application.

[0043] In a specific implementation manner, the mss screenshot function is called to obtain basic information of the first display and the second display.

[0044] S101. Obtain m frames of audio and video data of a first display, where the m frames of audio and video data correspond to m display timestamps respectively, and m is an integer greater than 1.

[0045] The audio and video data may include audio data and video data, wherein the video data may include screenshot image data of a display.

[0046] In a specific implementation manner, the mss screenshot function is called to obtain m frames of screenshot image data of the first display.

[0047] S102 . Encode m frames of audio and video data to obtain m frames of encoded data corresponding to the m frames of audio and video data.

[0048] S102 may be specifically implemented as follows: encoding m frames of audio and video data according to the identification information of the first display.

[0049] In a specific implementable manner, the cv2 library of Python is used to encode m frames of audio and video data to obtain m frames of encoded data corresponding to the m frames of audio and video data.

[0050] S103: Obtain n frames of audio and video data of the second display, where the n frames of audio and video data correspond to n display timestamps respectively, where n is equal to m, and n is an integer greater than 1.

[0051] Here, n is equal to m, which can be understood as the first display and the second display recording the screen synchronously.

[0052] In a specific implementation manner, the mss screenshot function is called to obtain n frames of screenshot image data of the second display.

[0053] S104 , encoding n frames of audio and video data to obtain n frames of encoded data corresponding to the n frames of audio and video data.

[0054] Among them, S104 can be specifically implemented as: encoding n frames of audio and video data according to the identification information of the second display.

[0055] In a specific implementable manner, the cv2 library of Python is used to encode n frames of audio and video data to obtain n frames of encoded data corresponding to the n frames of audio and video data.

[0056] S105 . Generate an audio and video file according to m frames of coded data, m display timestamps, n frames of coded data, and n display timestamps.

[0057] In a specific implementation, an audio and video file is generated according to attribute information of the first display, attribute information of the second display, m frames of coded data, m display timestamps, n frames of coded data, and n display timestamps.

[0058] Exemplarily, a first file is generated according to the attribute information of the first display, m frames of coded data, and m display timestamps. A second file is generated according to the attribute information of the second display, n frames of coded data, and n display timestamps. The first file and the second file are synthesized into an audio and video file according to the m display timestamps and the n display timestamps.

[0059] In a specific implementable manner, the cv2 library of Python is used to generate an audio and video file by converting the attribute information of the first display, the attribute information of the second display, m frames of encoded data, m display timestamps, n frames of encoded data, and n display timestamps.

[0060] In the embodiment of the present application, an audio and video file is generated according to the m frames of coded data and m display timestamps corresponding to the first display, and the n frames of coded data and n display timestamps corresponding to the second display. In this way, after the user records the operations on multiple displays, the video can be played back at the corresponding multiple displays, which can meet the user's multi-display review needs.

[0061] In some embodiments, after S105, a screen recording playback method for multiple displays provided by an embodiment of the present application further includes:

[0062] S106: Play the audio and video files on the first display and the second display.

[0063] Specifically, S106 can be implemented as follows: according to the audio and video file, m frames of audio and video data corresponding to m frames of coded data are played on the first display in the time sequence of m display timestamps. According to the audio and video file, n frames of audio and video data corresponding to n frames of coded data are played on the second display in the time sequence of n display timestamps. That is, at the same timestamp, the first display plays m frames of audio and video data, and the second display plays n frames of audio and video data, that is, the first display and the second display play their corresponding recorded data synchronously.

[0064] In a specific implementation, when the first display and the second display play the audio or video file, the play mode of the audio or video file is controlled, wherein the play mode may include at least one of the following: play progress, play speed, pause play, and exit play.

[0065] In a specific implementable method, Python's cv2 library is used to obtain audio and video files and play the audio and video files on the first display and the second display. When the audio and video files are played on the first display and the second display, the progress bar control, speed increase, pause and exit of the video playback on all displays can be performed.

[0066] Based on the same inventive concept, according to another aspect of the present invention, Figure 2As shown, an embodiment of the present invention further provides a computer 200, which includes: a first acquisition unit 201, a first encoding unit 202, a second acquisition unit 203, a second encoding unit 204 and a generation unit 205, wherein the first acquisition unit 201 is configured to acquire m frames of audio and video data of the first display, and the m frames of audio and video data correspond to m display timestamps respectively, and m is an integer greater than 1. The first encoding unit 202 is configured to encode the m frames of audio and video data to obtain m frames of encoded data corresponding to the m frames of audio and video data. The second acquisition unit 203 is configured to acquire n frames of audio and video data of the second display, and the n frames of audio and video data correspond to n display timestamps respectively, and n is equal to m, and n is an integer greater than 1. The second encoding unit 204 is configured to encode the n frames of audio and video data to obtain n frames of encoded data corresponding to the n frames of audio and video data. The generation unit 205 is configured to generate an audio and video file according to the m frames of encoded data, the m display timestamps, the n frames of encoded data, and the n display timestamps.

[0067] Based on the same inventive concept, according to another aspect of the present invention, Figure 3 As shown, an embodiment of the present invention further provides a computer-readable storage medium 30, which stores a computer program 310 for executing the above method when executed by a processor.

[0068] Finally, it should be noted that a person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium of the program can be a disk, an optical disk, a read-only storage memory (ROM) or a random access memory (RAM), etc. The above-mentioned computer program embodiments can achieve the same or similar effects as the corresponding above-mentioned arbitrary method embodiments.

[0069] The above are exemplary embodiments disclosed in the present invention, but it should be noted that various changes and modifications may be made without departing from the scope disclosed in the embodiments of the present invention as defined in the claims. The functions, steps and / or actions of the method claims according to the disclosed embodiments described herein do not need to be performed in any particular order. In addition, although the elements disclosed in the embodiments of the present invention may be described or required in individual form, they may also be understood as multiple unless explicitly limited to the singular.

[0070] It should be understood that, as used herein, the singular forms "a", "an" are intended to include the plural forms as well, unless the context clearly supports an exception. It should also be understood that, as used herein, "and / or" refers to any and all possible combinations including one or more of the associated listed items.

[0071] The serial numbers of the embodiments disclosed in the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0072] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a hard disk, or a CD, etc.

[0073] A person skilled in the art should understand that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the disclosure of the embodiments of the present invention (including the claims) is limited to these examples; under the concept of the embodiments of the present invention, the technical features in the above embodiments or different embodiments can also be combined, and there are many other changes in different aspects of the embodiments of the present invention as above, which are not provided in detail for the sake of simplicity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the embodiments of the present invention.

Claims

1. A screen recording and playback method for multiple displays, characterized in that: include: Obtain m frames of audio and video data of the first display, where the m frames of audio and video data correspond to m display timestamps respectively, and m is an integer greater than 1; Encode m frames of audio and video data to obtain m frames of encoded data corresponding to the m frames of audio and video data respectively; Obtain n frames of audio and video data of the second display, where the n frames of audio and video data correspond to n display timestamps respectively, where n is equal to m and n is an integer greater than 1, and the second display and the first display are extended for display; Encode n frames of audio and video data to obtain n frames of encoded data corresponding to the n frames of audio and video data respectively; Generate an audio and video file according to m frames of coded data, m display timestamps, n frames of coded data, and n display timestamps; After generating the audio and video file according to the m frames of coded data, the m display timestamps, the n frames of coded data, and the n display timestamps, the method further includes: Playing audio and video files on a first display and a second display, including: According to the audio and video file, playing m frames of audio and video data corresponding to the m frames of encoded data on the first display in the time sequence of the m display timestamps; According to the audio and video file, playing n frames of audio and video data corresponding to n frames of encoded data on the second display in the time sequence of n display timestamps; Before acquiring the m frames of audio and video data of the first display, the method further includes: Acquire basic information of the first display and the second display, wherein the basic information includes attribute information; The step of generating an audio and video file according to m frames of coded data, m display timestamps, n frames of coded data, and n display timestamps includes: An audio and video file is generated according to the attribute information of the first display, the attribute information of the second display, m frames of coded data, m display timestamps, n frames of coded data, and n display timestamps, wherein the first file is generated according to the attribute information of the first display, m frames of coded data, and m display timestamps, and the second file is generated according to the attribute information of the second display, n frames of coded data, and n display timestamps, and the first file and the second file are synthesized into an audio and video file according to the m display timestamps and the n display timestamps.

2. The method according to claim 1, characterized in that Playing audio and video files on the first display and the second display includes: When playing the audio and video files on the first display and the second display, the playing mode of the audio and video files is controlled.

3. The method according to claim 2, characterized in that The play mode includes at least one of the following: play progress, play speed, pause play, and exit play.

4. The method according to claim 1, characterized in that: Before acquiring the m frames of audio and video data of the first display, the method further includes: Basic information includes identification information; Encoding the m frames of audio and video data, including: encoding the m frames of audio and video data according to the identification information of the first display; Encoding n frames of audio and video data includes: encoding n frames of audio and video data according to identification information of the second display.

5. The method according to claim 1, characterized in that The audio and video data includes the screenshot image data of the display.

6. A computer, characterized in that: include: A first acquisition unit, a first encoding unit, a second acquisition unit, a second encoding unit and a generation unit, wherein: The first acquisition unit is configured to acquire m frames of audio and video data of the first display, the m frames of audio and video data respectively corresponding to m display timestamps, where m is an integer greater than 1; The first encoding unit is configured to encode m frames of audio and video data to obtain m frames of encoded data corresponding to the m frames of audio and video data respectively; The second acquisition unit is configured to acquire n frames of audio and video data of the second display, the n frames of audio and video data respectively correspond to n display timestamps, n is equal to m, and n is an integer greater than 1, and the second display and the first display are extended for display; The second encoding unit is configured to encode n frames of audio and video data to obtain n frames of encoded data corresponding to the n frames of audio and video data respectively; The generating unit is configured to generate an audio and video file based on m frames of coded data, m display timestamps, n frames of coded data, and n display timestamps; Also included are units for performing the following functions: Play audio and video files on the first and second displays for: According to the audio and video file, playing m frames of audio and video data corresponding to the m frames of encoded data on the first display in the time sequence of the m display timestamps; According to the audio and video file, playing n frames of audio and video data corresponding to n frames of encoded data on the second display in the time sequence of n display timestamps; and units for performing the following functions: Acquire basic information of the first display and the second display, wherein the basic information includes attribute information; The generating unit is further configured to: An audio and video file is generated according to the attribute information of the first display, the attribute information of the second display, m frames of coded data, m display timestamps, n frames of coded data, and n display timestamps, wherein the first file is generated according to the attribute information of the first display, m frames of coded data, and m display timestamps, and the second file is generated according to the attribute information of the second display, n frames of coded data, and n display timestamps, and the first file and the second file are synthesized into an audio and video file according to the m display timestamps and the n display timestamps.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps included in the method according to any one of claims 1 to 5 are implemented.