Multi-platform collaborative screenshot and screen recording review method and system based on UE

Abstract

The invention discloses a UE-based multi-platform collaborative review screenshot and screen recording method and system, and the method comprises the steps: S1, carrying out the adaptation of equipment and the triggering of functions, enabling a user to enter a collaborative review system, enabling the collaborative review system to automatically detect the types of the equipment, enabling the types of the equipment to be divided into VR equipment and a PC terminal, and enabling the VR equipment to carry out the automatic review of the equipment; mapping an input signal of the VR equipment into a unified screenshot and screen recording trigger instruction; s2, triggering a screenshot function and a screen recording function, and after the screenshot function is triggered, acquiring corresponding screen image data by the system according to the visual angle information of the current review scene; s3, storing and synchronizing the data; and S4, compared with the prior art, the method and the system for multi-platform collaborative review screenshot and screen recording based on the UE can realize multi-platform compatible adaptation: a universal adaptation framework is developed, the screenshot and screen recording function can be smoothly realized on various VR brand devices and PCs, and wide device support is provided for collaborative review.

Description

Technical Field

[0001] This application relates to the technical field of relay protection. More specifically, this application relates to a method and system for collaborative review screenshot recording based on UE across multiple platforms. Background Art

[0002] With the development of the industry, remote collaborative review has become an important means to improve R & D efficiency and integrate global resources. The collaborative review system based on Unreal Engine (UE) has been widely used due to its powerful 3D rendering ability and cross-platform characteristics. In terms of the screenshot recording function, the existing technologies are also constantly evolving.

[0003] In the prior art, the publication number CN119376598A discloses "A method for screenshot recording in a virtual world and a virtual reality device", which provides a method for screenshot recording in a virtual world and a virtual reality device for realizing partial screenshot recording. When taking a screenshot or recording a screen, by displaying an operation box that changes with the user's viewing point in the current virtual screen, the target area for screenshot or screen recording can be flexibly selected to achieve partial screenshot recording; by creating a layer, the picture of the target area is superimposed and displayed on the current virtual screen, which is convenient for users to preview the effect of partial screenshot recording and ensures accuracy; after the screenshot recording operation is completed, the layer is hidden to avoid interfering with the current virtual screen; directly performing screenshot or screen recording operations on the target area of the current virtual screen to ensure the original resolution of the partial screenshot image or partial screen recording image and improve the image quality; and the publication number CN117615248A discloses "A method, device, equipment and storage medium for photographing the content of a VR display screen", which collects the light intensity signal of the VR display screen and preprocesses it, and determines the shooting time point according to the preprocessed light intensity signal, and controls the shooting device to shoot the content of the VR display screen at this time point. By calculating the light intensity change of the VR display screen to trigger shooting, the requirements for the shooting device can be reduced, it is easy to select models, the cost is low, the installation size is small, and the situation of frame loss can be effectively avoided;

[0004] However, there are still certain defects in the prior art, such as:

[0005] Poor multi-device adaptability: It is impossible to achieve a unified and stable screenshot recording function on various VR brand devices and PCs, which limits the range of device selection for collaborative review;

[0006] Insufficient data storage and management: In terms of data storage, the needs of local and server storage cannot be well balanced, it is difficult to ensure data consistency, and the reliability of data transmission is low in a cross-border network environment. In terms of data management, there is a lack of effective retrieval and backup strategies, and data conflicts are likely to occur during multi-user concurrent access.

[0007] Limited interaction function: There is a lack of three-dimensional spatial interaction display function for screenshots on VR devices, and the data synchronization and interaction between the PC side and the VR side are not smooth, affecting the communication effect and collaboration efficiency during the review process.

[0008] In summary, the present invention overcomes the above problems and provides a screenshot and screen recording function module that can operate stably on multiple platforms (multiple VR brands and PCs), efficiently store and manage data, and has rich interaction functions, so as to improve the quality and efficiency of collaborative review. Summary of the Invention

[0009] In order to overcome the above-mentioned defects of the prior art, the present application provides a method and system for multi-platform collaborative review screenshot and screen recording based on UE to solve the problems proposed in the above background technology.

[0010] To achieve the above object, the present application provides the following technical solutions: A method for multi-platform collaborative review screenshot and screen recording based on UE, including:

[0011] S1. First, perform device adaptation and function triggering.

[0012] Among them, when the user enters the collaborative review system, the collaborative review system automatically detects the device type, and the device type is divided into VR devices and PC terminals.

[0013] On the VR device, map the input signal of the VR device to a unified screenshot and screen recording trigger instruction.

[0014] On the PC side, trigger the screenshot and screen recording function through keyboard shortcuts and mouse operations.

[0015] S2. Trigger the screenshot function and the screen recording function.

[0016] After the screenshot function is triggered, the system obtains the corresponding screen image data according to the perspective information of the current review scene.

[0017] After the screen recording function is started, the system continuously captures the video stream and audio stream of the current review scene.

[0018] S3. Store and synchronize the data, and store and synchronize the processed screenshot and screen recording data to both the local and the server at the same time.

[0019] S4. Perform interactive display on the VR device.

[0020] On the VR device, the user can view the saved screenshots in three-dimensional space through the handle operation, and the system presents the screenshots in the form of cards in the VR scene. The user can interactively display by grabbing the cards and scaling and rotating them in the hand.

[0021] Further, the input signals on the VR device are input into the collaborative review system through handle button operations or gesture recognition signals. On the VR device, when the user presses a specific handle button, a screenshot can be triggered, and when pressed twice consecutively, screen recording starts.

[0022] Further, on the PC side, the user takes a screenshot by pressing the "Ctrl + Shift + S" key combination, and starts screen recording by pressing the "Ctrl + Shift + R" key combination.

[0023] Further, in the screenshot function in S2, on the VR device, the image of the current viewport is obtained using the rendering pipeline of UE; on the PC side, the image of the current window or a specified area is captured through the graphics API, where the graphics API includes DirectX and OpenGL. For the need to take screenshots from different angles, the system records the user's operation of adjusting the viewing angle, and obtains the corresponding image according to the final viewing angle parameters. The obtained image data is compressed using an efficient image compression algorithm to reduce the data volume and improve the storage and transmission efficiency.

[0024] Further, the efficient image compression algorithm uses WebP or JPEG-XR.

[0025] Further, in the screen recording function in S2, in terms of video stream capture, the VR device and the PC side use their respective cameras to obtain continuous image frames; the audio stream is collected through the device's microphone, and then the video stream and the audio stream are encoded separately. Among them, the video uses the H.265 or AV1 high-efficiency video coding standard, and the audio uses AAC or Opus coding. The encoded audio and video data are encapsulated in a certain format.

[0026] Further, in S3, in terms of local storage, according to the operating systems and storage permissions of different devices, the corresponding storage paths and file formats are selected for saving. In terms of server storage, a distributed storage architecture is adopted, and the data is stored on multiple server nodes to improve the reliability and scalability of storage; at the same time, the AES encryption algorithm is used to encrypt the data stored on the server, and a two-way synchronization mechanism is established between the local and the server. Among them, when the local data is updated, the system uploads the data to the server through the network; when the server data changes, the system synchronizes the data to the local in a timely manner. In a cross-border network environment, an adaptive network transmission protocol is adopted to dynamically adjust the data transmission rate and retransmission strategy according to the network status to ensure the integrity of data transmission.

[0027] Further, in S4, to achieve synchronous display, when the user operates the screenshot on the VR device, the system synchronizes the operation information to the devices of other participants through the network, so that all participants can see the same display effect. In terms of background data management, the server records the relevant information of each screenshot, including the screenshot time, screenshot perspective, and the review meeting it belongs to, facilitating retrieval and viewing by the user when needed.

[0028] Furthermore, a multi-platform collaborative review screenshot and screen recording system based on UE includes:

[0029] Device identification module: When the user enters the collaborative review system using a VR device or a PC, the collaborative review system automatically detects the device type through the device identification module, enabling the collaborative review system to record the corresponding adaptation plugins according to different brands of VR devices;

[0030] Screenshot function module: Used to take screenshots on VR devices and PC terminals and store the screenshot data;

[0031] Screen recording function module: Used to perform screen recording on VR devices and PC terminals and store the screen recording data;

[0032] Data storage and synchronization module: Used to store the screenshot data obtained through VR devices and PC terminals in the screenshot function module, and at the same time store the screen recording data obtained by screen recording on VR devices and PC terminals. The data storage and synchronization module is also used to synchronize the screenshot data and screen recording data during local storage and service;

[0033] Interactive display module: Used to display the screenshot information in the form of VR.

[0034] Technical effects and advantages of this application:

[0035] Compared with the prior art, the method and system for multi-platform collaborative review screenshot and screen recording based on UE can be compatible and adapted on multiple platforms: Develop a general adaptation framework to ensure that the screenshot and screen recording functions can be smoothly implemented on various VR brand devices and PCs, providing extensive device support for collaborative review.

[0036] The method and system for multi-platform collaborative review screenshot and screen recording based on UE have efficient data storage and management capabilities: By constructing a secure and reliable data storage and management system, realize the synchronous storage of screenshot and screen recording data on the local and the server, ensuring data consistency. By optimizing the storage structure and transmission protocol, improve the transmission reliability of data in a cross-border network environment, and at the same time establish an efficient data retrieval and backup mechanism to meet the needs of multi-user concurrent access.

[0037] The method and system for screenshot recording and screen casting in multi-platform collaborative review based on UE have rich interaction functions: realizing three-dimensional spatial interactive display of screenshots on VR devices, facilitating intuitive display and communication of participants in an immersive environment, strengthening data synchronization and interaction between the PC side and the VR side, and improving the collaboration and efficiency of collaborative review. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic diagram of the system flow of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0039] Next, the technical solutions in the embodiments of this application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts shall fall within the protection scope of this application.

[0040] Embodiment 1

[0041] The method for screenshot recording and screen casting in multi-platform collaborative review based on UE includes:

[0042] S1. First, perform device adaptation and function triggering.

[0043] Among them, when the user enters the collaborative review system, the collaborative review system automatically detects the device type, and the device type is divided into VR devices and the PC side.

[0044] On the VR device, map the input signal of the VR device to a unified screenshot recording and screen casting triggering instruction.

[0045] On the PC side, trigger the screenshot recording and screen casting function through keyboard shortcuts and mouse operations.

[0046] S2. Trigger the screenshot function and the screen casting function.

[0047] After the screenshot function is triggered, the system obtains the corresponding screen image data according to the perspective information of the current review scene.

[0048] After the screen casting function is started, the system continuously captures the video stream and audio stream of the current review scene.

[0049] S3. Store and synchronize the data, and store and synchronize the processed screenshot and screen casting data to both the local and the server at the same time.

[0050] S4. Perform interactive display on the VR device.

[0051] On the VR device, the user operates the handle to view the saved screenshots in the three-dimensional space. The system presents the screenshots in the form of cards in the VR scene, and the user interacts and displays them by grabbing the cards and scaling and rotating them in the hand.

[0052] The input signal on the VR device is input into the collaborative review system through the handle button operation or gesture recognition signal. On the VR device, when the user presses a specific handle button, the screenshot can be triggered, and when pressed twice continuously, the screen recording starts.

[0053] On the PC side, the user takes a screenshot by pressing the "Ctrl+Shift+S" key combination, and starts screen recording by pressing the "Ctrl+Shift+R" key combination.

[0054] In the screenshot function of S2, on the VR device, the image of the current viewport is obtained using the rendering pipeline of UE; on the PC side, the image of the current window or a specified area is intercepted through the graphics API, where the graphics API includes DirectX and OpenGL. For the need to take screenshots at different angles, the system records the user's operation of adjusting the viewing angle, and obtains the corresponding image according to the final viewing angle parameters. The obtained image data is compressed using an efficient image compression algorithm to reduce the data volume and improve the storage and transmission efficiency.

[0055] The efficient image compression algorithm uses WebP or JPEG-XR.

[0056] In the screen recording function of S2, in terms of video stream capture, the VR device and the PC side use their respective cameras to obtain continuous image frames; the audio stream is collected through the device's microphone, and then the video stream and the audio stream are encoded separately. Among them, the video uses the H.265 or AV1 high-efficiency video coding standard, and the audio uses the AAC or Opus coding. The encoded audio and video data are encapsulated in a certain format.

[0057] In S3, in terms of local storage, according to the operating systems and storage permissions of different devices, corresponding storage paths and file formats are selected for saving. For example, on the PC side of the Windows system, the data is saved to the folder "My Documents\Collaborative Review\Screenshots and Screen Recordings"; on the VR device of the Android system, it is saved to the specified private storage directory of the application. In terms of server storage, a distributed storage architecture is adopted, and the data is stored on multiple server nodes to improve the reliability and scalability of storage; at the same time, the AES encryption algorithm is used to encrypt the data stored on the server, and a two-way synchronization mechanism is established between the local and the server. Among them, when the local data is updated, the system uploads the data to the server through the network; when the server data changes, the system synchronizes the data to the local in a timely manner. In a cross-border network environment, an adaptive network transmission protocol is adopted to dynamically adjust the data transmission rate and retransmission strategy according to the network status to ensure the integrity of data transmission.

[0058] In S4, to achieve synchronous display, when the user operates on the screenshots on the VR device, the system synchronizes the operation information to the devices of other participants through the network, so that all participants can see the same display effect. In terms of background data management, the server records the relevant information of each screenshot, including the screenshot time, screenshot perspective, and the affiliated review meeting, which is convenient for users to retrieve and view when needed.

[0059] Embodiment 2

[0060] Please refer to Figure 1 , a multi-platform collaborative review screenshot and screen recording system based on UE, including:

[0061] Device identification module: When the user enters the collaborative review system using a VR device or a PC, the collaborative review system automatically detects the device type through the device identification module, enabling the collaborative review system to record the corresponding adaptation plug-ins according to different brands of VR devices;

[0062] Screenshot function module: Used for taking screenshots on VR devices and PC sides and storing the screenshot data;

[0063] Screen recording function module: Used for performing screen recording on VR devices and PC sides and storing the screen recording data;

[0064] Data storage and synchronization module: Used for storing the screenshot data obtained from VR devices and PC sides in the screenshot function module, and at the same time storing the screen recording data obtained from VR devices and PC sides. The data storage and synchronization module is also used for synchronizing the screenshot data and the screen recording data during local storage and service;

[0065] Interactive display module: Used for displaying the screenshot information in the form of VR.

[0066] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the technical principle of the present invention, several improvements and modifications can be made, and these improvements and modifications should also be regarded as the protection scope of the present invention.

Claims

1. A method for collaborative review screenshot and screen recording of multiple platforms based on UE, characterized in that: Including: S1. First, perform device adaptation and function triggering, wherein, when the user enters the collaborative review system, the collaborative review system automatically detects the device type, and the device types are divided into VR devices and PC terminals. On the VR device, map the input signal of the VR device into a unified screenshot and screen recording trigger instruction; On the PC terminal, trigger the screenshot and screen recording functions by means of keyboard shortcuts and mouse operations; S2. Trigger the screenshot function and the screen recording function, After the screenshot function is triggered, the system obtains the corresponding screen image data according to the perspective information of the current review scene; After the screen recording function is started, the system continuously captures the video stream and audio stream of the current review scene; S3. Store and synchronize the data, and store and synchronize the processed screenshot and screen recording data to both the local and the server simultaneously; S4. Perform interactive display on the VR device, On the VR device, the user can view the saved screenshots in the three-dimensional space through the handle operation. The system presents the screenshots in the form of cards in the VR scene. The user can interactively display by grabbing the cards and zooming and rotating them in the hand.

2. The method for collaborative review screenshot and screen recording based on UE according to claim 1, wherein The input signal on the VR device is input into the collaborative review system through the handle button operation or gesture recognition signal. On the VR device, when the user presses a specific handle button, the screenshot can be triggered, and when pressed twice continuously, the screen recording starts.

3. The method for UE-based multi-platform collaborative review screenshot and screen recording according to claim 1, characterized in that: On the PC terminal, the user takes a screenshot by pressing the "Ctrl+Shift+S" combination key, and starts the screen recording by pressing the "Ctrl+Shift+R" combination key.

4. The method for UE-based multi-platform collaborative review screenshot and screen recording according to claim 1, wherein: In the screenshot function in S2, on the VR device, use the rendering pipeline of UE to obtain the image of the current viewport; on the PC terminal, intercept the image of the current window or the specified area through the graphics API, where the graphics API includes DirectX and OpenGL. For the need to take screenshots from different angles, the system records the user's operation of adjusting the perspective, and obtains the corresponding image according to the final perspective parameters. The obtained image data is compressed by an efficient image compression algorithm to reduce the data volume and improve the storage and transmission efficiency.

5. The method for UE-based multi-platform collaborative review screenshot and screen recording according to claim 4, characterized in that: The efficient image compression algorithm uses WebP or JPEG-XR.

6. The method for UE-based multi-platform collaborative review screenshot and screen recording according to claim 1, wherein: In the screen recording function in S2, in terms of video stream capture, the VR device and the PC terminal use their respective cameras to obtain continuous image frames; the audio stream is collected through the microphone of the device, and then the video stream and the audio stream are respectively encoded. Among them, the video uses the H.265 or AV1 high-efficiency video coding standard, and the audio uses AAC or Opus coding. The encoded audio and video data are encapsulated in a certain format.

7. The method for collaborative review screenshot recording and screen recording based on UE according to claim 1, wherein: In S3, in terms of local storage, according to the operating systems and storage permissions of different devices, corresponding storage paths and file formats are selected for saving. In terms of server storage, a distributed storage architecture is adopted, and data is stored on multiple server nodes to improve the reliability and scalability of storage. At the same time, the AES encryption algorithm is used to encrypt the data stored on the server, and a two-way synchronization mechanism is established between the local and the server. Among them, when the local data is updated, the system uploads the data to the server through the network; when the server data changes, the system synchronizes the data to the local in a timely manner. In a cross-border network environment, an adaptive network transmission protocol is adopted to dynamically adjust the data transmission rate and retransmission strategy according to the network status to ensure the integrity of data transmission.

8. The method for UE-based multi-platform collaborative review screenshot and screen recording according to claim 1, characterized in that: In S4, to achieve synchronous display, when the user operates the screenshot on the VR device, the system synchronizes the operation information to the devices of other participants through the network, so that all participants can see the same display effect. In terms of background data management, the server records the relevant information of each screenshot, including the screenshot time, screenshot perspective, and the affiliated review meeting, which is convenient for users to retrieve and view when needed.

9. A system for collaborative review screenshot recording based on UE, which is applied to the method for collaborative review screenshot recording based on UE according to any one of claims 1-8, is characterized in that, Including: Device identification module: When the user enters the collaborative review system using a VR device or a PC, the collaborative review system automatically detects the device type through the device identification module, enabling the collaborative review system to record the corresponding adaptation plugins according to different brands of VR devices; Screenshot function module: Used for taking screenshots on VR devices and PC sides and storing the screenshot data; Screen recording function module: Used for recording the screen on VR devices and PC sides and storing the screen recording data; Data storage and synchronization module: Used for storing the screenshot data obtained from VR devices and PC sides in the screenshot function module, and at the same time storing the screen recording data obtained from VR devices and PC sides by screen recording. The data storage and synchronization module is also used for synchronizing the screenshot data and the screen recording data during local storage and service; Interactive display module: Used for displaying the screenshot information in the form of VR.

Images