Virtual production-based device monitoring method, device, apparatus, medium and product

Abstract

The application discloses a virtual production-based equipment monitoring method and device, equipment, medium and product, and relates to the technical field of virtual production. The method comprises the following steps: acquiring running state data and running video data corresponding to a plurality of hardware devices respectively; matching the running state data of the hardware devices with monitoring indexes of the device types corresponding to the hardware devices, displaying the matching results between the running state data of the hardware devices and the monitoring indexes; displaying video pictures corresponding to the running video data of the hardware devices; and performing data statistical analysis on the running performance data of the hardware devices, and displaying the statistical analysis results of the running performance data in the form of a chart. The method realizes the automation of data detection of each device in the shooting field of virtual production in the hardware aspect, improves the monitoring efficiency of the hardware devices in the shooting field of virtual production, and reduces the workload of the on-site staff.

Description

Virtual film production-based equipment monitoring methods, devices, equipment, media, and products Technical Field

[0001] This application relates to the field of virtual filmmaking technology, and in particular to a device monitoring method, apparatus, equipment, medium and product based on virtual filmmaking. Background Technology

[0002] Virtual production refers to a series of computer-aided production and visual filmmaking methods. On a virtual production set, a light-emitting diode (LED) screen displays virtual content. In front of the LED screen are actual set props. Cameras in the virtual production simultaneously capture images of the LED screen and the set props, resulting in a video that blends the images from both. The virtual production process involves many hardware components; for example, a rendering machine is needed on the virtual production set to render the captured, blended video in real time.

[0003] In related technologies, once it is confirmed that the rendering machine can work properly, staff can operate the rendering machine to render the captured fused video in real time. After the shooting is completed, staff can check the rendered video material.

[0004] However, because the rendering machine renders in real time, staff cannot detect problems with the rendering machine during filming. The rendered video footage can only be checked during post-production. If problems arise with the video footage at this point, and the video needs to be reshot, it requires a lot of manpower and resources, resulting in low filming efficiency for virtual production. Summary of the Invention

[0005] This application provides a method, apparatus, device, medium, and product for monitoring equipment based on virtual filmmaking, which improves the efficiency of monitoring hardware equipment on the shooting set of virtual filmmaking. The technical solution is as follows:

[0006] On the one hand, a device monitoring method based on virtual filmmaking is provided, the method comprising:

[0007] The system acquires running status data and running video data corresponding to multiple hardware devices, which are devices located in a virtual film production shooting scene. The multiple hardware devices are used to build and collect shooting scenes of the virtual film production shooting scene. The multiple hardware devices correspond to at least two device types. The running status data includes running performance data, which is used to represent the hardware performance of the hardware devices during operation.

[0008] The operating status data of the hardware device is matched with the monitoring indicators of the corresponding device type, and the matching result between the operating status data of the hardware device and the monitoring indicators is displayed. The monitoring indicators are used to indicate the operating conditions that the hardware device corresponding to the device type needs to meet during operation, and the matching result is used to indicate the operating fault status of the hardware device in the virtual film production shooting site.

[0009] The video screen that displays the operating video data of the hardware device refers to the working scene screen of the hardware device on the virtual film production shooting site;

[0010] The operating performance data of the hardware device is statistically analyzed, and the results of the statistical analysis are displayed in the form of charts.

[0011] On the other hand, a device monitoring apparatus based on virtual film production is provided, the apparatus comprising:

[0012] The acquisition module is used to acquire the running status data and running video data corresponding to multiple hardware devices. The multiple hardware devices are devices located in the virtual film production shooting scene. The multiple hardware devices are used to build and collect the shooting scene of the virtual film production shooting scene. The multiple hardware devices correspond to at least two types of devices. The running status data includes running performance data, which is used to represent the hardware performance of the hardware devices during operation.

[0013] The first display module is used to match the operating status data of the hardware device with the monitoring indicators of the device type corresponding to the hardware device, and display the matching result between the operating status data of the hardware device and the monitoring indicators. The monitoring indicators are used to indicate the operating conditions that the hardware device corresponding to the device type needs to meet during operation, and the matching result is used to indicate the operating fault status of the hardware device in the virtual film production shooting site.

[0014] The first display module is further configured to display video footage corresponding to the operating video data of the hardware device, wherein the video footage refers to the working scene footage of the hardware device on the virtual film production shooting site;

[0015] The first display module is also used to perform statistical analysis on the operating performance data of the hardware device and display the statistical analysis results of the operating performance data in the form of charts.

[0016] On the other hand, a device monitoring method based on virtual film production is provided, the method comprising:

[0017] The system receives device monitoring operations on multiple hardware devices, which are devices located in a virtual film production shooting location. The multiple hardware devices are used to build and collect shooting scenes from the virtual film production shooting location, and the multiple hardware devices correspond to at least two device types.

[0018] In response to the device monitoring operation, the matching result between the hardware device's operating status data and the monitoring indicators of the corresponding device type is displayed. The monitoring indicators are used to indicate the operating conditions that the hardware device corresponding to the device type needs to meet during operation. The matching result is used to indicate the operating fault status of the hardware device in the virtual film production shooting site. The operating status data includes operating performance data, which is used to represent the hardware performance of the hardware device during operation.

[0019] The video screen that displays the operating video data of the hardware device refers to the working scene of the hardware device on the virtual film production shooting site.

[0020] In response to receiving an analysis operation on the operating performance data of the hardware device, the statistical analysis results of the operating performance data are displayed in the form of a chart. The analysis operation is used to perform statistical analysis on the specified operating status data.

[0021] On the other hand, a device monitoring apparatus based on virtual film production is provided, the apparatus comprising:

[0022] A receiving module is used to receive device monitoring operations on multiple hardware devices, which are devices located in a virtual film production shooting scene. The multiple hardware devices are used to build and collect the shooting scene of the virtual film production shooting scene, and the multiple hardware devices correspond to at least two device types.

[0023] The second display module is used to respond to the device monitoring operation and display the matching result between the hardware device's operating status data and the monitoring indicators of the device type corresponding to the hardware device. The monitoring indicators are used to indicate the operating conditions that the hardware device corresponding to the device type needs to meet during operation. The matching result is used to indicate the operating fault status of the hardware device corresponding to the virtual film production shooting site. The operating status data includes operating performance data, which is used to represent the hardware performance of the hardware device during operation.

[0024] The second display module is also used to display video footage corresponding to the operating video data of the hardware device, wherein the video footage refers to the working scene of the hardware device on the virtual film production shooting site;

[0025] The second display module is further configured to respond to receiving an analysis operation on the operating performance data of the hardware device, and to display the statistical analysis results of the operating performance data in the form of a chart, wherein the analysis operation is used to perform statistical analysis on the specified operating status data.

[0026] On the other hand, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement the device monitoring method based on virtual fabrication as described in any of the above embodiments.

[0027] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, code set, or instruction set is stored therein, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the device monitoring method based on virtual fabrication as described in any of the above embodiments.

[0028] On the other hand, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the virtual fabrication-based device monitoring methods described in the above embodiments.

[0029] The beneficial effects of the technical solutions provided in this application include at least the following:

[0030] On the one hand, by matching the operational status data of various hardware devices with the monitoring indicators corresponding to the device types, the matching results between the operational status data and monitoring indicators are displayed on the monitoring terminal device. The device also displays the working scene of multiple hardware devices on the virtual production set. This allows staff to manage multiple hardware devices in a unified manner and monitor them remotely, automating the data detection of hardware aspects of various devices on the virtual production shooting set. This improves the efficiency of hardware device monitoring on the virtual production shooting set and reduces the workload of on-site staff. On the other hand, by performing statistical analysis on the operational performance data of the hardware devices and displaying the results in chart form, the performance of the hardware devices during operation is more intuitively displayed. This allows staff to understand the current operating status of the hardware devices in real time and promptly resolve any potential hardware operation problems. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 is a schematic diagram of a device monitoring system provided in an exemplary embodiment of this application;

[0033] Figure 2 is a flowchart of a device monitoring method based on virtual fabrication provided in an exemplary embodiment of this application;

[0034] Figure 3 is a schematic diagram of the interface for real-time display of running status data provided in an exemplary embodiment of this application;

[0035] Figure 4 is a flowchart of a device monitoring method based on virtual film production provided in another exemplary embodiment of this application;

[0036] Figure 5 is an interactive flowchart of a device monitoring method based on virtual fabrication provided in an exemplary embodiment of this application;

[0037] Figure 6 is a flowchart of a device monitoring method based on virtual fabrication provided in yet another exemplary embodiment of this application;

[0038] Figure 7 is a schematic diagram of the interface of the hardware status check module in the device monitoring software provided in an exemplary embodiment of this application;

[0039] Figure 8 is a schematic diagram of the interface of the hardware real-time status module in the device monitoring software provided in an exemplary embodiment of this application;

[0040] Figure 9 is a schematic diagram of the interface of the data alarm module in the device monitoring software provided in an exemplary embodiment of this application;

[0041] Figure 10 is a schematic diagram of the interface of the historical log module in the device monitoring software provided in an exemplary embodiment of this application;

[0042] Figure 11 is a schematic diagram of the interface of the on-site video monitoring module in the equipment monitoring software provided in an exemplary embodiment of this application;

[0043] Figure 12 is a schematic diagram of the interface of the performance data acquisition module in the device monitoring software provided in an exemplary embodiment of this application;

[0044] Figure 13 is a structural block diagram of a device monitoring apparatus based on virtual fabrication provided in an exemplary embodiment of this application;

[0045] Figure 14 is a structural block diagram of a device monitoring apparatus based on virtual fabrication provided in another exemplary embodiment of this application;

[0046] Figure 15 is a structural block diagram of a monitoring terminal device provided in an exemplary embodiment of this application. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] In this application, the terms "first" and "second" are used to distinguish between identical or similar items that have essentially the same function. It should be understood that there is no logical or temporal dependency between "first" and "second", nor is there any limitation on the quantity or execution order.

[0049] Virtual production refers to a series of computer-aided production and visual filmmaking methods. On a virtual production set, a light-emitting diode (LED) screen displays virtual content. In front of the LED screen are actual set props. Cameras in the virtual production simultaneously capture images of the LED screen and the set props, resulting in a video that blends the images from both. The virtual production process involves many hardware components. For example, a rendering machine is needed on set to render the captured blended video in real time. In related technologies, once the rendering machine is confirmed to be working properly, staff can operate it to render the captured blended video in real time. After filming, staff can inspect the rendered video footage. However, because the rendering is done in real time, staff cannot detect problems with the rendering machine during filming. Inspection of the rendered video footage is only possible during post-production. If problems arise at this point, requiring reshooting, it incurs significant manpower and resources, resulting in low efficiency for virtual production.

[0050] This application provides a device monitoring method based on virtual filmmaking. For illustration, please refer to Figure 1, which shows a schematic diagram of a device monitoring system provided in an exemplary embodiment. The device monitoring system includes a monitoring terminal device 110, multiple hardware devices, and a server 130. The computer device 100 and the server 130, as well as the multiple hardware devices and the server 130, are connected via wired or wireless networks.

[0051] Optionally, the multiple hardware devices 120 include: a rendering machine 121, an LED processor 122, an operator unit 123, a camera for shooting 124, a monitoring camera 125, and other hardware devices used to build and capture shooting scenes of a virtual film production shooting location. Optionally, each of the above-mentioned hardware devices is equipped with a first application program. Schematic, this first application program can be a traditional application program, a cloud application program, a mini-program or application module within a host application program, or a web platform; this embodiment does not limit this. The first application program provides data acquisition and transmission functions, used to collect the operating status data of the hardware devices and send it to the monitoring terminal device 110.

[0052] Optionally, the monitoring terminal device 110 is equipped with a second application, which provides data display and detection functions to display the acquired hardware device's operating status data and operating video data; at the same time, it analyzes the operating status data and displays the analysis results.

[0053] Server 130 provides backend services for monitoring terminal device 110 and multiple hardware devices 120. Optionally, server 130 includes data server 131 and web server 132.

[0054] As illustrated in Figure 1, the shooting location also includes a first switch 140. Each of the aforementioned hardware devices sends its corresponding operational status data to the first switch 140, which then forwards it to a data server 131. Upon receiving the operational status data, the data server 131 forwards it to a network server 132, which then sends it to a monitoring device 110. Upon receiving the operational status data, the monitoring device 110 can display it on its screen.

[0055] Optionally, after receiving the operating status data, the monitoring terminal device 110 can match the operating status data of the hardware device based on the monitoring indicators, and display the matching result between the operating status data of the hardware device and the monitoring indicators on the device screen; Optionally, the operating status data of the hardware device also includes operating performance data, which is used to represent the hardware performance of the hardware device during operation. After receiving the operating performance data, the monitoring terminal device 110 can perform data statistical analysis on the operating performance data of the hardware device and display the statistical analysis results of the operating performance data in the form of charts.

[0056] In some optional embodiments, the first application described above is also used to collect video data from the hardware device and send it to the monitoring terminal device 110. The video data includes: (1) the screen display of the operator 123, which is used to record the software operation process during the shooting process; (2) the footage captured by the camera 124 (i.e., the video used to make the movie); and (3) the working scene of each hardware device captured by the monitoring camera 125.

[0057] As illustrated in Figure 1, the shooting location also includes a second switch 150 and a streaming device 160, and the server 130 includes a video server 133. The operator unit 123, the shooting camera 124, and the monitoring camera 125 send their respective video data to the streaming device 160. The streaming device 160 pushes the data to the second switch 150, which then sends the video data to the video server 133. Upon receiving the video data, the video server 133 forwards it to the network server 132, which then sends it to the monitoring device 110. After receiving the video data, the monitoring device 110 displays the corresponding video image on its screen.

[0058] It is worth noting that server 130 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0059] Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. Based on the cloud computing business model, cloud technology encompasses network technology, information technology, integration technology, management platform technology, and application technology. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to a backend system for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing. Optionally, server 130 can also function as a node in a blockchain system.

[0060] It should be noted that this application may display prompt interfaces, pop-ups, or output voice prompts before and during the collection of user-related data (e.g., operational status data, operational video data). These prompt interfaces, pop-ups, or voice prompts are used to inform the user that their relevant data is being collected. This ensures that the application only begins the steps for collecting user-related data after receiving confirmation from the user regarding the prompt interface or pop-up; otherwise (i.e., without user confirmation), the steps for collecting user-related data end, meaning no user-related data is collected. In other words, all user data collected in this application is collected with the user's consent and authorization, and the collection, use, and processing of related user data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0061] Based on the above description, the device monitoring method based on virtual fabrication provided in this application will be described. Figure 2 shows a flowchart of a device monitoring method based on virtual fabrication provided in an exemplary embodiment of this application. Taking the application of this method in the monitoring terminal device 110 shown in Figure 1 as an example, the method includes:

[0062] Step 201: Obtain the operating status data and operating video data corresponding to multiple hardware devices respectively.

[0063] Among them, multiple hardware devices are devices located on the virtual film production shooting site. Multiple hardware devices are used to build and collect shooting scenes of the virtual film production shooting site. Multiple hardware devices correspond to at least two types of devices. The running status data includes running performance data, which is used to represent the hardware performance of the hardware devices during operation.

[0064] Optionally, the multiple hardware devices include at least two of the following: a rendering machine, an LED processor, a control unit, a camera for shooting, and a camera for monitoring, with each hardware device corresponding to a device type.

[0065] The rendering machine is a computer device equipped with a rendering engine, used to handle rendering tasks during the shooting process, such as rendering the required footage on an LED display screen in a virtual production location; the LED processor is used to convert images into signals that the LED display screen can receive; the operator console is equipped with various software control systems for controlling the equipment on site, such as controlling the lighting on site through the lighting control system in the operator console; the shooting camera is used to shoot videos; and the monitoring camera is used to capture the working scenes of various equipment on site and the manual operation scenes of the equipment by the staff.

[0066] Optionally, the operational status data corresponding to the hardware device is used to indicate the operating status of each hardware component within the device. Each hardware device has its own corresponding operational status data. For example, for rendering machines and operating machines, the operational status data includes: Central Processing Unit (CPU) utilization, CPU temperature, Graphics Processing Unit (GPU) utilization, GPU temperature, frame rate, synchronization signal, hard drive status, memory usage, process focus, graphics card resolution, graphics card color depth, and graphics card color exposure value. For LED processors, the operational status data includes: input signal resolution, output signal resolution, input signal frame rate, output signal frame rate, color gamut setting, color space, exposure parameters, brightness setting, Gamma setting, color temperature, display status, LED shutter refresh rate, LED synchronization status, and LED phase shift. For cameras, the operational status data includes: shooting status, hard drive status, shooting resolution, shooting frame rate, shooting color gamut, camera battery level, timecode, synchronization status, color temperature, and aperture.

[0067] Optionally, the method for the monitoring terminal device to acquire the operating status data corresponding to multiple hardware devices includes at least one of the following methods:

[0068] 1. In response to receiving a device monitoring operation, acquire the operating status data corresponding to multiple hardware devices.

[0069] Among them, equipment monitoring operation refers to the operation performed on the monitoring terminal device. For example, the monitoring terminal device is equipped with a target application, which is used to detect the operating status of multiple hardware devices. When the staff at the shooting site clicks the "Start Detection" control on the target application, the monitoring terminal device sends a data acquisition command to the server. The server forwards the command to each hardware device, and each hardware device begins to collect real-time operating status data and sends it to the server. The server then sends the acquired operating status data to the monitoring terminal device.

[0070] 2. Based on preset time intervals, acquire the operating status data corresponding to multiple hardware devices.

[0071] Optionally, multiple hardware devices will periodically send their respective real-time operating status data to the server, and the server will then send the acquired operating status data to the monitoring terminal device. Alternatively, multiple hardware devices may send real-time operating status data to the server at fixed time intervals (e.g., every 5 seconds), and the server will then send the acquired operating status data to the monitoring terminal device.

[0072] 3. In response to a change in the specified operating status data corresponding to the hardware device, obtain the specified operating status data corresponding to the hardware device.

[0073] Optionally, multiple hardware devices periodically send their respective real-time operating status data to the server. The server analyzes this operating status data, and if a specific operating status data changes, it sends that specific operating status data to the monitoring device. For example, camera 1 periodically sends the acquired shooting resolution, shooting frame rate, camera battery level, etc., to the server. When camera 1 starts running, the monitoring device acquires a shooting resolution of 1920*1080, a shooting frame rate of 30 frames per second, and a camera battery level of 100%. After camera 1 has been running for 5 minutes, it sends the acquired shooting resolution of 1920*1080, a shooting frame rate of 30 frames per second, and a camera battery level of 99% to the server. The server analyzes the acquired data and finds that, except for the camera battery level, other operating status data for camera 1 has not changed. Therefore, it sends the 99% camera battery level data to the monitoring device, which then updates the camera battery level data of camera 1 to 99%.

[0074] It should be noted that the above-described method for obtaining the operating status data corresponding to multiple hardware devices is merely an illustrative example, and the embodiments of this application do not limit it.

[0075] The aforementioned video data is used to indicate the working scene of the hardware equipment on the virtual film production shooting site.

[0076] Optionally, the method for the monitoring terminal device to acquire the running video data corresponding to multiple hardware devices includes at least one of the following methods:

[0077] 1. In response to receiving a device monitoring operation, acquire the running video data corresponding to multiple hardware devices.

[0078] Optionally, after receiving a device monitoring operation on the monitoring terminal device, the monitoring terminal device sends a running video data acquisition instruction to the server. After receiving the instruction, the server sends a running video data acquisition request to the operator and cameras (including monitoring cameras and shooting cameras) of the virtual production site. The server obtains the running video data and forwards it to the monitoring terminal device.

[0079] 2. Based on preset time intervals, acquire the operating status data corresponding to multiple hardware devices.

[0080] Optionally, the operating units and cameras (including monitoring cameras and shooting cameras) on the virtual production set periodically send runtime video data to the server, and the server sends the acquired runtime video data to the monitoring terminal device. Optionally, multiple hardware devices send real-time runtime video data to the server at fixed time intervals (e.g., once every 5 seconds), and the server sends the acquired runtime video data to the monitoring terminal device.

[0081] Step 202: Match the operating status data of the hardware device with the monitoring indicators of the corresponding device type, and display the matching results between the operating status data of the hardware device and the monitoring indicators.

[0082] Among them, the monitoring indicators are used to indicate the operating conditions that the hardware equipment corresponding to the equipment type needs to meet during operation, and the matching results are used to indicate the operating fault status of the hardware equipment at the virtual filming shooting site.

[0083] Optionally, the operating conditions may be reference operating parameters corresponding to hardware components during the operation of the hardware device; or, the range of reference operating parameters corresponding to hardware components during the operation of the hardware device; or, the reference operating state corresponding to hardware components during the operation of the hardware device (e.g., the reference operating state of the camera is standby state and shooting state).

[0084] The monitoring indicators include at least one of the following:

[0085] 1. Monitoring indicators refer to the indicator data that the specified operating status data of hardware devices need to achieve.

[0086] The indicator data can be the target data to be achieved, or it can be the target range to be achieved.

[0087] To illustrate, the brightness specification for the LED processor is 1900 cd / m². 2 Therefore, the brightness of the LED processor needs to be set to 1900 cd / m². 2 Alternatively, if the GPU utilization rate of the camera is 97%-100%, then a GPU utilization rate of 97%-100% is normal for the camera.

[0088] 2. Monitoring indicators refer to the status indicators of specified operating status data in hardware devices during normal operation.

[0089] For illustrative purposes, the camera's shooting status indicator is "standby" or "shooting," indicating that the camera is in standby or shooting mode and is operating normally; or, the rendering machine's synchronization signal indicator is "synchronizing," indicating that the rendering machine's synchronization signal is in synchronization mode and is operating normally.

[0090] Optionally, the monitoring indicators are indicator data preset by the staff; or, the monitoring indicators are indicator data determined based on historical operating status data. For example, in the historical operating data of the rendering machine, when the GPU utilization rate is 50%, the number of failures is the least, so the GPU utilization rate of the rendering machine can be set to 50%.

[0091] Optionally, if the monitoring indicators are stored in a designated memory in the monitoring terminal device, then the monitoring indicators corresponding to at least two device types can be retrieved directly from the designated memory; or, if the monitoring indicators are stored in a server, then the monitoring terminal device can obtain the monitoring indicators corresponding to at least two device types from the server.

[0092] Optionally, the monitoring indicators that need to be matched for the operating status data of hardware devices of the same type are the same; or, the monitoring indicators that need to be matched for the operating status data of hardware devices of the same type are different.

[0093] This is illustrative of the scenario where all LED processors on set are tested based on the same monitoring metrics; alternatively, each LED processor may have its own specific monitoring metrics, which differ between different LED processors. Optionally, the monitoring metrics of the LED processors can be set according to the parameters of the corresponding LED display screen.

[0094] Optionally, the operating status data of each hardware device is compared with its corresponding monitoring indicators in turn. For example, for multiple operating status data points of a single hardware device, each data point is compared with its corresponding monitoring indicators in turn. If the operating status data meets the monitoring indicators, it indicates that the hardware component corresponding to that operating status data is operating normally; if the operating status data does not meet the monitoring indicators, it indicates that the hardware component corresponding to that operating status data is operating abnormally.

[0095] As an illustration, after the hardware is powered on but before officially starting work, staff can test multiple rendering machines on set, checking if the graphics card resolution of each machine is 1920*1080. If all of them are 1920*1080, the resolution test is passed. If the graphics card resolution of rendering machine 1 is 1080*720, the message "Rendering machine 1 graphics card resolution is 1080*720" will be displayed as the matching result for the multiple rendering machines, indicating that the graphics card resolution of rendering machine 1 is abnormal.

[0096] Optionally, if multiple hardware devices periodically send their respective real-time operating status data to the monitoring terminal device, the monitoring terminal device will display the operating status data of the multiple hardware devices in real time.

[0097] Optionally, the operating status data of multiple hardware devices can be displayed in the monitoring terminal device in the form of a statistical table. For illustration, please refer to Figure 3, which shows a schematic diagram of a real-time display interface for operating status data. As shown in Figure 3, interface 300 is the real-time operating status data display interface for the rendering machine, displaying the frame rate, synchronization signal, etc., corresponding to the rendering machine.

[0098] Step 203: Display the video screen corresponding to the operating video data of the hardware device.

[0099] Among them, video footage refers to the working scene of hardware equipment on a virtual film production and shooting set.

[0100] Optionally, a first video frame corresponding to the operating video data of the hardware device is displayed. The first video frame is the footage captured by the camera at the virtual production shooting location.

[0101] In illustrative terms, the aforementioned first video frame could be footage shot by a camera on a virtual filming set, specifically a composite image of the LED display screen and the scene in front of it. Alternatively, the first video frame could be real-time footage of various hardware devices on-site, captured by a monitoring camera on the virtual filming set, showing on-site personnel manually operating these devices.

[0102] Optionally, the multiple hardware devices include a control unit, which is used to receive software operations during the virtual production shooting process. The control unit then displays a second video screen corresponding to the running video data of the hardware devices. The second video screen is the screen display of the control unit at the virtual production shooting site, and the screen display is used to show the software operation process during the virtual production shooting process.

[0103] In illustrative terms, the screen display corresponding to the second video frame is used to display the operation process of the control software corresponding to each hardware device. For example, the rendering machine corresponds to a rendering engine, and the screen display of the operator will display the operation process of the rendering task in the rendering engine.

[0104] Step 204: Perform statistical analysis on the operating performance data of the hardware device and display the results of the statistical analysis on the operating performance data in the form of charts.

[0105] Optionally, hardware performance data is a standard used to measure the quality of hardware devices, and performance data includes: graphics and image processing performance data, storage performance data, and video processing performance data, etc. This application does not limit this aspect.

[0106] Optionally, the number of operating performance data can be one or more, in which case statistical analysis can be performed on multiple operating performance data of the hardware device, and the statistical analysis results of multiple operating performance data can be displayed in the form of a statistical chart / statistical table.

[0107] This is an illustrative example of collecting frame rate data from the rendering machine over a period of time and displaying a line graph showing the changes in the rendering machine's frame rate during that period.

[0108] In summary, the equipment monitoring method based on virtual production provided in this application, on the one hand, matches the operating status data of various hardware devices with the monitoring indicators of the corresponding equipment types, displays the matching results between the operating status data and monitoring indicators on the monitoring terminal device, and displays the working scene of multiple hardware devices on the virtual production set. This allows staff to manage multiple hardware devices in a unified manner and monitor them remotely, automating the data detection of hardware aspects of various devices on the virtual production shooting set, improving the efficiency of hardware device monitoring on the virtual production shooting set, and reducing the workload of on-site staff. On the other hand, by performing statistical analysis on the operating performance data of the hardware devices and displaying the statistical analysis results in chart form, the performance of the hardware devices during operation is more intuitively displayed, facilitating staff to understand the current operating status of the hardware devices in real time and promptly resolve any potential hardware operation problems.

[0109] The method provided in this application embodiment displays the operating status data of multiple hardware devices in real time on the monitoring terminal device, which makes it convenient for staff to check the status of the hardware devices in real time, eliminating the need to manually check each parameter of the hardware devices one by one, and improving the monitoring efficiency of the hardware devices.

[0110] The method provided in this application embodiment displays the footage of a virtual film production shooting scene captured by a camera on a monitoring terminal device. On the one hand, by acquiring the shooting scene footage from the hardware device, it can help staff troubleshoot problems during the shooting process; on the other hand, by acquiring the shooting footage of the film production, it can help staff determine whether the current shooting footage meets the shooting requirements, and it can also be used for problem backtracking to confirm the effect of the shooting footage.

[0111] Figure 4 shows a flowchart of a device monitoring method based on virtual fabrication provided in an exemplary embodiment of this application. Taking the application of this method in the monitoring terminal device 110 shown in Figure 1 as an example, the method includes:

[0112] Step 401: Obtain the operating status data and operating video data corresponding to multiple hardware devices respectively.

[0113] Among them, multiple hardware devices are devices located on the virtual film production shooting site. Multiple hardware devices are used to build and collect shooting scenes of the virtual film production shooting site. Multiple hardware devices correspond to at least two types of devices. The running status data includes running performance data, which is used to represent the hardware performance of the hardware devices during operation.

[0114] Among them, the operating status data corresponding to the hardware device is used to indicate the operation status of each hardware component in the hardware device, and each hardware device has its own corresponding operating status data.

[0115] Optionally, multiple hardware devices include a rendering machine, an LED processor, a control unit, a camera for capturing images, and a monitoring camera. For illustration, please refer to Figure 5, which shows an interactive flowchart of a device monitoring method based on virtual production. At regular intervals, status data is obtained from the rendering machine 501, control unit 502, camera 503, and LED processor 504, and sent to a data server for storage. The monitoring device 505 can request to log in to the web server; after successful login, it can request the status data stored in the data server through the web server.

[0116] The aforementioned video data is used to indicate the working scene of the hardware equipment on the virtual film production shooting site.

[0117] As illustrated in Figure 5, video data is obtained from the operator 502 and camera 503 at regular intervals and sent to the video server for storage. The monitoring terminal device 505 can request to log in to the web server. After successful login, it can request to retrieve the video data stored in the video server through the web server.

[0118] Step 402: Display the video screen corresponding to the operating video data of the hardware device.

[0119] Optionally, the video footage includes the following three types:

[0120] 1. The film shot on the virtual production shooting set is a video shot with a camera, which is a fusion of the LED display screen and the scene in front of the screen, shot by the camera.

[0121] 2. The monitoring cameras on the virtual filming set capture real-time working scenes of various hardware devices on set, as well as the manual operation scenes of on-site staff on various hardware devices.

[0122] 3. The screen display of the operator's console on the virtual production shooting set displays the control software operation process of each hardware device. The operator's console receives software operations during the virtual production shooting process. For example, a rendering machine corresponds to a rendering engine, and the operator's screen displays the operation flow of the rendering tasks in that rendering engine.

[0123] For illustration, please refer to Figure 5. After obtaining the video data of camera 501 and operator 502 from the video server through the WEB server, the video data is returned to the monitoring terminal device 503 for display.

[0124] Step 403: Real-time display of the operating status data of multiple hardware devices.

[0125] For illustration, please refer to Figure 5. After obtaining the status data from the rendering machine 504, camera 501, operator 502 and LED processor 505 through the video server via the WEB server, the status data is returned to the monitoring terminal device 503 for real-time display.

[0126] Step 404: In response to the fact that the operating status data of the hardware device does not meet the monitoring indicators of the corresponding device type, the operating status data of the hardware device is highlighted as the matching result.

[0127] Among them, the monitoring indicators are used to indicate the operating conditions that the hardware devices corresponding to the equipment type need to meet during operation.

[0128] Optionally, if the monitoring indicators are stored in a designated memory in the monitoring terminal device, then the monitoring indicators corresponding to at least two device types can be retrieved directly from the designated memory; or, if the monitoring indicators are stored in a data server, then the monitoring terminal device can obtain the monitoring indicators corresponding to at least two device types from the data server.

[0129] Optionally, the operating conditions may be reference operating parameters corresponding to hardware components during the operation of the hardware device; or, the range of reference operating parameters corresponding to hardware components during the operation of the hardware device; or, the reference operating state corresponding to hardware components during the operation of the hardware device (e.g., the reference operating state of the camera is standby state and shooting state).

[0130] For illustration, please refer to Figure 3. The synchronization signal indicator of the rendering machine is "synchronizing". The synchronization signal status of rendering machine 5 is "synchronization lost". Since the synchronization signal indicator has not been reached, it is an abnormal situation. Therefore, the "synchronization lost" status is highlighted as the matching result. At the same time, the frame rate indicator of the rendering machine is 30 frames per second. The frame rate of rendering machine 3 is 20 frames per second. Since the frame rate indicator has not been reached, "20" is highlighted as the matching result.

[0131] Step 405: In response to the difference between the operating status data of the hardware device and the monitoring index of the corresponding device type being less than or equal to a preset difference, a warning message for the operating status data is displayed as the matching result.

[0132] Optionally, the above monitoring indicators can be implemented as the minimum conditions that the hardware device needs to meet during operation. Taking a camera as an example, if the difference between the remaining power of the camera and the minimum power required for the camera to operate is less than or equal to a preset difference, a power warning message is displayed as the matching result.

[0133] For illustrative purposes, the minimum power required for the camera to operate is 20%, the preset difference is 5%, and the remaining power of camera 1 is 24%. Then, the warning message "Camera 1's power is about to run out" can be displayed as the power status matching result of camera 1.

[0134] In some optional embodiments, the operating status data of the matched hardware devices can also be classified into faults according to the matching situation. Optionally, the classification categories include at least one of the following categories:

[0135] 1. Error Category.

[0136] If the operating status data of a hardware device fails to meet the monitoring indicators for the corresponding device type, the operating status data of that hardware device will be classified as an error category.

[0137] 2. Warning categories.

[0138] If the difference between the operating status data of a hardware device and the monitoring index of the corresponding device type is less than or equal to a preset difference, the operating status data of the hardware device will be classified as a warning.

[0139] 3. Normal category.

[0140] When the operating status data of a hardware device reaches the monitoring indicators corresponding to the device type, the operating status data of that hardware device is classified as normal. This means the hardware device is not malfunctioning and can operate normally.

[0141] Optionally, based on the classification categories corresponding to the matching hardware device operating status data, data alarm information corresponding to multiple hardware devices is displayed. The data alarm information is used to display the operating status data of the fault and the category corresponding to the fault.

[0142] In some optional embodiments, the monitoring terminal device also stores log data corresponding to multiple hardware devices, wherein the log data is used to record the operating status data of multiple hardware devices in a historical time period and the matching results corresponding to the operating status data.

[0143] In some optional embodiments, when the operating status data of the hardware device does not meet the monitoring indicators of the corresponding device type, the faulty operating status data can be analyzed through the video data corresponding to the hardware device to obtain the fault analysis results of the operating status data.

[0144] Optionally, in response to the hardware device's target operating status data failing to meet the monitoring indicators for the corresponding device type within a specified time period, target operating video data of the hardware device is acquired within the specified time period; based on the target operating video data within the specified time period, fault detection is performed on the target operating status data to determine the cause of the fault; and according to the cause of the fault, fault repair prompts are displayed, which provide a reference for fault repair of the target operating status data.

[0145] For illustrative purposes, if a hardware device fails to meet its corresponding monitoring indicators for a certain period of time, it indicates that the hardware component corresponding to the target operating status data has malfunctioned. Monitoring video data of the hardware device during that period of time, as well as software operation video data of the operator controlling the hardware device during that period of time, can be obtained.

[0146] Optionally, the target running video data and target running status data within a specified time period are input into the fault detection model. The fault detection model is used to predict the fault cause corresponding to the target running status data. The fault detection model is a neural network model that has been trained in advance using sample data. The output is the fault cause prediction result of the target running status data as the fault cause.

[0147] The target operation video data includes monitoring videos of hardware devices within a specified time period and videos of software operation processes.

[0148] Optionally, there is a one-to-one correspondence between the fault detection model and the hardware device. In other words, the fault detection model corresponding to the rendering machine is used to predict the operational faults generated by the rendering machine.

[0149] The fault prediction model is a model trained using sample operating status data (operating status data corresponding to faulty hardware devices), sample monitoring videos, sample software operation process videos, and reference fault causes. Schematic, the sample monitoring videos, sample software operation process videos, and sample operating status data are input into the sample fault prediction model, and the predicted fault cause corresponding to the sample operating status data is output. The sample fault prediction model is trained based on the difference between the predicted fault cause and the reference fault cause to obtain the fault prediction model.

[0150] Optionally, after obtaining the cause of the fault in the target operating status data, it is also necessary to determine the corresponding fault repair prompt information. In this case, the fault type library corresponding to the target operating status data is obtained. The fault type library includes fault types and corresponding fault repair reference information. The fault cause is matched with the fault type library, and the target fault type that matches the fault cause is determined in the fault type library. The fault repair reference information corresponding to the target fault type is used as the fault repair prompt information.

[0151] Among them, the method of matching fault causes with fault type library can be illustrated by the semantic similarity method, which calculates the semantic similarity between fault causes and each fault type in fault type library, and determines the fault type with the highest semantic similarity as the target fault type.

[0152] Step 406: Perform statistical analysis on the rendering data of the rendering machine and display the statistical analysis results of the rendering data in the form of a line chart.

[0153] The aforementioned hardware devices include a rendering machine, where the rendering machine's performance data includes its rendering data. Statistical analysis of the rendering data corresponding to the rendering machine is then performed, and the results are displayed in the form of a line graph.

[0154] Optionally, the rendering data of the rendering machine includes the rendering frame rate, rendering duration, GPU processing time, etc. during rendering.

[0155] This is an illustrative example of collecting the rendering frame rate, rendering duration, and GPU processing time of the rendering machine over a period of time, and displaying a line graph showing the changes in the rendering frame rate, rendering duration, and GPU processing time during this period.

[0156] In summary, the equipment monitoring method based on virtual production provided in this application, on the one hand, matches the operating status data of various hardware devices with the monitoring indicators of the corresponding equipment types, displays the matching results between the operating status data and monitoring indicators on the monitoring terminal device, and displays the working scene of multiple hardware devices on the virtual production set. This allows staff to manage multiple hardware devices in a unified manner and monitor them remotely, automating the data detection of hardware aspects of various devices on the virtual production shooting set, improving the efficiency of hardware device monitoring on the virtual production shooting set, and reducing the workload of on-site staff. On the other hand, by performing statistical analysis on the operating performance data of the hardware devices and displaying the statistical analysis results in chart form, the performance of the hardware devices during operation is more intuitively displayed, facilitating staff to understand the current operating status of the hardware devices in real time and promptly resolve any potential hardware operation problems.

[0157] The method provided in this application embodiment highlights the hardware device's operating status data when the device's operating status data fails to meet the corresponding monitoring indicators. This allows staff to efficiently grasp the hardware device's fault status, determine the cause of the fault, and further improve the monitoring efficiency of hardware devices on the virtual filming set.

[0158] The method provided in this application embodiment will display a warning message for the operating status data when the difference between the operating status data of the hardware device and the monitoring index corresponding to the hardware device is less than or equal to a preset difference. The warning message will indicate the status data that may be faulty, such as: the camera battery is about to run out of power or the hard disk space is about to run out of space. This will allow staff to prepare solutions in advance before the fault occurs, thus improving the efficiency of troubleshooting status data.

[0159] The method provided in this application embodiment performs statistical analysis on the performance (rendering) data of the rendering machine and displays the statistical analysis results of the performance data in the form of a line graph. By analyzing the performance data of the rendering machine in the form of a line graph, it is easier for staff to understand more intuitively whether the frame rate of the rendering machine is stable when a specified rendering task is performed. This increases the amount of information expressed by the running status data and improves the work efficiency of the staff.

[0160] The method provided in this application acquires target running video data within a specified time period, performs fault detection on the target running status data, thereby determining the cause of the fault in the target running status data, and finally displays fault repair prompts for the cause of the fault, providing a reference for fault repair of the target running status data and improving the efficiency of staff in fault detection and repair of hardware devices.

[0161] The method provided in this application improves the accuracy of fault detection by predicting the causes of faults in target operating status data using a neural network model (fault prediction model). Furthermore, by matching the fault causes with a fault type library, the method identifies the target fault type matching the fault cause in the library, thereby obtaining fault repair prompts for the fault causes and improving the accuracy of fault repair.

[0162] Figure 6 shows a flowchart of a device monitoring method based on virtual fabrication provided in an exemplary embodiment of this application. Taking the application of this method in the monitoring terminal device 110 shown in Figure 1 as an example, the method includes:

[0163] Step 601: Receive device monitoring operations for multiple hardware devices.

[0164] Among them, multiple hardware devices are devices located on the virtual film production shooting site. Multiple hardware devices are used to build and capture shooting scenes of the virtual film production shooting site. Multiple hardware devices correspond to at least two types of devices.

[0165] The aforementioned device monitoring operations are used to detect the operating status data corresponding to multiple hardware devices. Optionally, the aforementioned device monitoring operations are also used to display the operating status data corresponding to the multiple hardware devices.

[0166] Step 602: In response to the device monitoring operation, display the matching results between the operating status data of the hardware device and the monitoring indicators of the corresponding device type.

[0167] Among them, the monitoring indicators are used to indicate the operating conditions that the hardware devices corresponding to the equipment type need to meet during operation, the matching results are used to indicate the operating faults of the hardware devices in the virtual film production shooting site, and the operating status data includes operating performance data, which is used to represent the hardware performance of the hardware devices during operation.

[0168] Optionally, the above-mentioned device monitoring operation can be implemented by clicking controls on the monitoring device. For illustration, please refer to Figure 7, which shows a schematic diagram of the interface of the hardware status check module in the device monitoring software provided in an exemplary embodiment of this application. The device monitoring software is a target application installed on the monitoring device, and it is mainly divided into six functional modules, as shown in Figure 7. These six functional modules are:

[0169] The hardware status check module is used to check the operating status of various hardware devices on the shooting site before the start of virtual filming.

[0170] The hardware real-time status module is used to check the real-time operating status of various hardware devices on the shooting site during the virtual production shooting process.

[0171] The data alarm module is used to display the fault status in the operating status data of various hardware devices;

[0172] The historical log module is used to record log information in the hardware environment of various hardware devices;

[0173] The on-site video monitoring module includes three types of video footage: the screen display of the operator device, which records the software operation process during shooting; the footage captured by the camera; and the working scene footage of various hardware devices captured by the camera.

[0174] The performance data acquisition module is used to collect the performance data of the hardware in the hardware device. The hardware performance data is a standard used to measure the quality of the hardware device. The performance data includes: graphics and image processing performance data, storage performance data, and video processing performance data, etc. This application embodiment does not limit this.

[0175] For illustration, please refer to Figure 7. Clicking the "Hardware Status Check" control 701 will take you to the hardware status check module interface 700, where a "Start Detection" control 702 is displayed. Clicking the "Start Detection" control 702 will begin detecting each hardware device of the currently connected device monitoring software. The detection type, detection items, detection results, and detection status will be displayed in real time through a statistical table. As shown in Figure 7, the detection result is highlighted, indicating that the record failed the detection, and the current status of the hardware device will be displayed as the detection result (i.e., the matching result).

[0176] In Figure 7, when detecting the process data of the rendering machine, if multiple window processes are detected, these are considered useless processes, and the corresponding row of data is highlighted. Optionally, in Figure 7, each row of data does not represent the detection result of a single rendering machine, but rather the detection results of all rendering machines connected to the device monitoring system. Optionally, the highlighted detection results may also display "Renderer x has multiple window processes," indicating that the process data of rendering machine x is abnormal.

[0177] Optionally, the above-mentioned equipment monitoring operation can also realize the operation of automatically triggering the monitoring terminal device to display and detect the operating status data corresponding to multiple hardware devices according to a preset time interval. Optionally, based on the preset time interval, the operating status data corresponding to multiple hardware devices is displayed in real time, and the matching result between the operating status data of the hardware devices and the monitoring indicators of the corresponding equipment type of the hardware devices is displayed.

[0178] For illustration, please refer to Figure 8. Clicking the "Hardware Real-Time Status" control 801 will take you to the hardware status check module interface 800. Interface 800 displays real-time operating status data for multiple hardware devices, and updates the data in the table periodically based on the latest acquired operating status data. As shown in Figure 8, the operating status data is highlighted, indicating that the record has not met its corresponding monitoring indicator. The highlighted operating status data is used as the matching result for that data.

[0179] In some optional embodiments, the device monitoring software installed in the monitoring terminal device is also used to display data alarm information of the operating status data that has a fault. The data alarm information is used to indicate the fault level and fault type of the operating status data.

[0180] Optionally, when the operating status data fails (i.e., when the monitoring indicators are not met), the monitoring device will send the data alarm information of the failed operating status data to the server, which will then store and statistically analyze the data alarm information. In response to receiving the data alarm information export operation in the monitoring device, the data alarm information will be displayed in the monitoring device.

[0181] For illustration, please refer to Figure 9. Clicking the "Data Alarm" control 901 will take you to the data alarm module interface 900. Interface 900 displays an "Export" control 902. Clicking the "Export" control 902 will export the data alarm information, which includes: fault source, fault type, fault level, fault code, fault message, and the time of fault occurrence. Optionally, staff can click the fault type column in interface 900 to filter the data alarm information according to the fault type.

[0182] Optionally, a "Clear" control 903 is displayed in the interface 900. Clicking the "Clear" control 903 will clear the data alarm information displayed on the current interface.

[0183] In some optional embodiments, the device monitoring software installed in the monitoring terminal device is also used to display log information in the hardware environment corresponding to each hardware device. The log information includes not only data alarm information but also normal log content.

[0184] For illustration, please refer to Figure 10. Clicking the "Historical Logs" control 1001 will take you to the interface 1000 of the historical logs module. The interface 1000 displays a log list corresponding to each hardware device, as shown in Figure 10. Clicking log3 will allow you to view the detailed information of log3.

[0185] Step 603: Display the video screen corresponding to the operating video data of the hardware device.

[0186] Among them, video footage refers to the working scene of hardware equipment on a virtual film production and shooting set.

[0187] Optionally, the aforementioned video footage includes a first video footage and a second video footage.

[0188] Optionally, the first video frame can be footage shot by a camera on a virtual filming set, specifically a fused image of the LED display screen and the foreground scenery captured by the camera. Alternatively, the first video frame can be real-time footage of various hardware devices on-site, captured by a monitoring camera on the virtual filming set, as well as footage of on-site personnel manually operating these hardware devices.

[0189] Optionally, the second video screen is the screen display of the operator device, which is used to display the operation process of the control software corresponding to each hardware device. For example, the rendering machine corresponds to a rendering engine, and the operation process of the rendering task in the rendering engine will be displayed on the screen display of the operator device.

[0190] Optionally, in response to receiving a control click operation in the monitoring terminal device, the operating video data corresponding to each of the multiple hardware devices is displayed; or, based on a preset time interval, the operating video data corresponding to each of the multiple hardware devices is displayed.

[0191] For illustration, please refer to Figure 11. Clicking the "On-site Video Monitoring" control 1101 will take you to the on-site video monitoring module interface 1100. The interface 1100 displays in real time the images of multiple hardware devices, including the camera footage 1103 for shooting, the operator screen 1104, and the monitoring camera footage 1105. The interface updates the images every so often based on the latest acquired video data.

[0192] As shown in Figure 11, the right side of the interface 1100 also displays historical time records. Clicking on a specific time record in the historical time records will jump to the video screen corresponding to the specified time record in the camera screen 1103, the operator screen 1104, and the monitoring camera screen 1105.

[0193] Step 604: In response to receiving the analysis operation of the hardware device's operating performance data, display the statistical analysis results of the operating performance data in the form of a chart.

[0194] The analysis operation is used to perform statistical analysis on specified running status data.

[0195] The aforementioned hardware performance data is used to represent the hardware performance of the hardware device during operation, such as frame rate data, GPU processing time, and rendering time during rendering.

[0196] For illustration, please refer to Figure 12. Clicking the "Performance Data Acquisition" control 1201 will take you to the performance data module interface 1200. Interface 1200 displays a "Start Acquisition" control 1202. Clicking the "Start Acquisition" control 1202 will begin collecting the performance data corresponding to the rendering machine at the current moment, including: rendering frame rate, GPU processing time, rendering time, total rendering duration, etc., and displaying the performance data change curve in the form of a statistical chart. Optionally, the collected performance data will also be displayed as a statistical table on the right side of interface 1200.

[0197] Optionally, the interface 1200 displays a "End Acquisition" control 1203. Clicking the "End Acquisition" control 1202 will end the acquisition of the performance data corresponding to the rendering machine.

[0198] Optionally, the right side of the interface 1200 also displays historical collection records. Clicking on a specific collection time in the historical collection record will take you to the performance data analysis interface corresponding to that collection time.

[0199] In summary, the equipment monitoring method based on virtual production provided in this application, on the one hand, matches the operating status data of various hardware devices with the monitoring indicators of the corresponding equipment types, displays the matching results between the operating status data and monitoring indicators on the monitoring terminal device, and displays the working scene of multiple hardware devices on the virtual production set. This allows staff to manage multiple hardware devices in a unified manner and monitor them remotely, automating the data detection of hardware aspects of various devices on the virtual production shooting set, improving the efficiency of hardware device monitoring on the virtual production shooting set, and reducing the workload of on-site staff. On the other hand, by performing statistical analysis on the operating performance data of the hardware devices and displaying the statistical analysis results in chart form, the performance of the hardware devices during operation is more intuitively displayed, facilitating staff to understand the current operating status of the hardware devices in real time and promptly resolve any potential hardware operation problems.

[0200] Please refer to Figure 13, which shows a structural block diagram of a device monitoring apparatus based on virtual fabrication according to an exemplary embodiment of this application. The apparatus includes:

[0201] The acquisition module 1300 is used to acquire the running status data and running video data corresponding to multiple hardware devices. The multiple hardware devices are devices located in the virtual film production shooting scene. The multiple hardware devices are used to build and collect the shooting scene of the virtual film production shooting scene. The multiple hardware devices correspond to at least two types of devices. The running status data includes running performance data, which is used to represent the hardware performance of the hardware devices during operation.

[0202] The first display module 1310 is used to match the operating status data of the hardware device with the monitoring indicators of the device type corresponding to the hardware device, and display the matching result between the operating status data of the hardware device and the monitoring indicators. The monitoring indicators are used to indicate the operating conditions that the hardware device corresponding to the device type needs to meet during operation, and the matching result is used to indicate the operating fault status of the hardware device in the virtual film production shooting site.

[0203] The first display module 1310 is also used to display video frames corresponding to the operating video data of the hardware device, wherein the video frames refer to the working scene frames of the hardware device on the virtual film production shooting site;

[0204] The first display module 1310 is also used to perform statistical analysis on the operating performance data of the hardware device and display the statistical analysis results of the operating performance data in the form of charts.

[0205] In some optional embodiments, the first display module 1310 is also used to display the operating status data corresponding to multiple hardware devices in real time.

[0206] In some optional embodiments, the first display module 1310 is further configured to, in response to the hardware device's operating status data not meeting the monitoring indicators for the device type corresponding to the hardware device, highlight the hardware device's operating status data as the matching result.

[0207] In some optional embodiments, the first display module 1310 is further configured to display a warning message for the operating status data as the matching result in response to the difference between the operating status data of the hardware device and the monitoring index of the device type corresponding to the hardware device being less than or equal to a preset difference.

[0208] In some optional embodiments, the plurality of hardware devices include a rendering machine, and the performance data of the rendering machine includes the rendering data of the rendering machine; the first display module 1310 is further configured to perform rendering data statistical analysis on the rendering data of the rendering machine, and display the statistical analysis results of the rendering data in the form of a line graph.

[0209] In some optional embodiments, the first display module 1310 is further configured to display a first video frame corresponding to the operating video data of the hardware device, wherein the first video frame is a shot taken by a camera at the virtual production shooting location of the virtual production shooting location.

[0210] In some optional embodiments, the plurality of hardware devices include an operator unit, which is used to receive software operations during the virtual production shooting process. The first display module 1310 is also used to display a second video screen corresponding to the running video data of the hardware devices. The second video screen is the screen display screen of the operator unit at the virtual production shooting site, and the screen display screen is used to display the software operation process during the virtual production shooting process.

[0211] The acquisition module 1300 is further configured to acquire target operating video data of the hardware device within the specified time period in response to the hardware device's target operating status data failing to meet the monitoring indicators for the corresponding device type within the specified time period; the acquisition module 1300 is further configured to perform fault detection on the target operating status data based on the target operating video data within the specified time period to determine the cause of the fault in the target operating status data; the first display module 1310 is further configured to display fault repair prompt information according to the cause of the fault, the fault repair prompt information being used to provide a reference for fault repair of the target operating status data.

[0212] In some optional embodiments, the acquisition module 1300 is further configured to input the target running video data and the target running status data within the specified time period into a fault detection model, wherein the fault detection model is used to predict the fault cause corresponding to the target running status data, and the fault detection model is a neural network model pre-trained with sample data; the acquisition module 1300 is further configured to output the fault cause prediction result of the target running status data as the fault cause.

[0213] In some optional embodiments, the acquisition module 1300 is further configured to acquire a fault type library corresponding to the target operating status data, the fault type library including fault types and fault repair reference information corresponding to the fault types; the acquisition module 1300 is further configured to match the fault cause with the fault type library, determine the target fault type that matches the fault cause in the fault type library, and use the fault repair reference information corresponding to the target fault type as the fault repair prompt information.

[0214] In summary, the equipment monitoring device based on virtual production provided in this application, on the one hand, matches the operating status data of various hardware devices with the monitoring indicators of the corresponding equipment types, displays the matching results between the operating status data and monitoring indicators on the monitoring terminal device, and displays the working scene of multiple hardware devices on the virtual production set. This allows staff to manage multiple hardware devices in a unified manner and monitor them remotely, automating the data detection of hardware aspects of various devices on the virtual production shooting set, improving the efficiency of hardware device monitoring on the virtual production shooting set, and reducing the workload of on-site staff. On the other hand, by performing statistical analysis on the operating performance data of the hardware devices and displaying the statistical analysis results in chart form, the performance of the hardware devices during operation is more intuitively displayed, facilitating staff to understand the current operating status of the hardware devices in real time and promptly resolve any potential hardware operation problems.

[0215] Please refer to Figure 14, which shows a structural block diagram of a device monitoring apparatus based on virtual fabrication according to another exemplary embodiment of this application. The apparatus includes:

[0216] The receiving module 1400 is used to receive device monitoring operations on multiple hardware devices, which are devices located in a virtual film production shooting scene. The multiple hardware devices are used to build and collect the shooting scene of the virtual film production shooting scene, and the multiple hardware devices correspond to at least two device types.

[0217] The second display module 1410 is used to respond to the device monitoring operation and display the matching result between the operating status data of the hardware device and the monitoring indicators of the device type corresponding to the hardware device. The monitoring indicators are used to indicate the operating conditions that the hardware device corresponding to the device type needs to meet during operation. The matching result is used to indicate the operating fault status of the hardware device in the virtual film production shooting site. The operating status data includes operating performance data, which is used to represent the hardware performance of the hardware device during operation.

[0218] The second display module 1410 is also used to display video footage corresponding to the operating video data of the hardware device, wherein the video footage refers to the working scene of the hardware device on the virtual film production shooting site;

[0219] The second display module 1410 is further configured to, in response to receiving an analysis operation on the operating performance data of the hardware device, display the statistical analysis results of the operating performance data in the form of a chart, wherein the analysis operation is used to perform statistical analysis on the specified operating status data.

[0220] In summary, the equipment monitoring device based on virtual production provided in this application, on the one hand, matches the operating status data of various hardware devices with the monitoring indicators of the corresponding equipment types, displays the matching results between the operating status data and monitoring indicators on the monitoring terminal device, and displays the working scene of multiple hardware devices on the virtual production set. This allows staff to manage multiple hardware devices in a unified manner and monitor them remotely, automating the data detection of hardware aspects of various devices on the virtual production shooting set, improving the efficiency of hardware device monitoring on the virtual production shooting set, and reducing the workload of on-site staff. On the other hand, by performing statistical analysis on the operating performance data of the hardware devices and displaying the statistical analysis results in chart form, the performance of the hardware devices during operation is more intuitively displayed, facilitating staff to understand the current operating status of the hardware devices in real time and promptly resolve any potential hardware operation problems.

[0221] It should be noted that the virtual fabrication-based device monitoring device provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the virtual fabrication-based device monitoring device and the virtual fabrication-based device monitoring method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0222] Figure 15 shows a structural block diagram of a computer device 1500 provided in an exemplary embodiment of this application. The computer device 1500 may be a smartphone, tablet computer, Moving Picture Experts Group Audio Layer III (MP3) player, Moving Picture Experts Group Audio Layer IV (MP4) player, laptop computer, or desktop computer. The computer device 1500 may also be referred to as a user device, portable computer device, laptop computer device, desktop computer device, or other names.

[0223] Typically, computer device 1500 includes a processor 1501 and a memory 1502.

[0224] Processor 1501 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1501 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). Processor 1501 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1501 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 1501 may also include an Artificial Intelligence (AI) processor, which is used to handle computational operations related to machine learning.

[0225] Memory 1502 may include one or more computer-readable storage media, which may be non-transitory. Memory 1502 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in memory 1502 is used to store at least one instruction, which is executed by processor 1501 to implement the device monitoring method based on virtual fabrication provided in the method embodiments of this application.

[0226] In illustrative purposes, computer device 1500 also includes other components. Those skilled in the art will understand that the structure shown in FIG15 does not constitute a limitation on computer device 1500 and may include more or fewer components than shown, or combine certain components, or employ different component arrangements.

[0227] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. This program can be stored in a computer-readable storage medium, which may be a computer-readable storage medium included in the memory described in the above embodiments; or it may be a standalone computer-readable storage medium not assembled into a computer device. The computer-readable storage medium stores at least one instruction, at least one program segment, a code set, or an instruction set. The at least one instruction, the at least one program segment, the code set, or the instruction set is loaded and executed by the processor to implement any of the virtual chip-based device monitoring methods described in the above embodiments.

[0228] Optionally, the computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), solid-state drives (SSDs), or optical discs, etc. The random access memory may include resistive random access memory (ReRAM) and dynamic random access memory (DRAM). The sequence numbers of the embodiments in this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0229] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0230] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A device monitoring method based on virtual film production, characterized in that, The method includes: acquiring operational status data and operational video data corresponding to multiple hardware devices; the multiple hardware devices are devices located on a virtual film production shooting site, used to build and collect shooting scenes of the virtual film production shooting site; the operational status data includes operational performance data used to indicate the hardware performance of the hardware devices during operation; matching the operational status data of the hardware devices with monitoring indicators corresponding to the device type of the hardware devices, displaying the matching result between the operational status data and the monitoring indicators, the matching result being used to indicate the operational failure status of the hardware devices on the virtual film production shooting site; displaying the video screen corresponding to the operational video data, the video screen referring to the working scene screen of the hardware devices on the virtual film production shooting site; the video screen includes the monitoring screen of the camera on the virtual film production shooting site and the screen display screen of the operator, the operator being used to receive software operations during the virtual film production shooting process, and the screen display screen being used to display the control software operation process corresponding to the hardware devices. The system performs statistical analysis on the operating performance data of the hardware device and displays the results in chart form. In response to the hardware device failing to meet the monitoring indicators for a specified time period in its target operating status data, it acquires target operating video data for that time period, including monitoring videos of the hardware device and software operation videos of the hardware device's operating mechanism. The system inputs the target operating video data and the target operating status data into a fault detection model and outputs the cause of the fault in the target operating status data. The fault detection model is a neural network model pre-trained using sample data. The sample data includes sample operating status data corresponding to faulty hardware devices, sample monitoring videos, sample software operation videos, and reference fault causes corresponding to the sample operating status data. Based on the fault cause, the system displays fault repair prompts, which provide a reference for fault repair of the target operating status data.

2. The method according to claim 1, characterized in that, The method further includes: displaying the operating status data corresponding to the multiple hardware devices in real time.

3. The method according to claim 2, characterized in that, The step of displaying the matching result between the operating status data of the hardware device and the monitoring indicator includes: in response to the fact that the operating status data of the hardware device does not meet the monitoring indicator of the device type corresponding to the hardware device, highlighting the operating status data of the hardware device as the matching result.

4. The method according to claim 2, characterized in that, The step of displaying the matching result between the operating status data of the hardware device and the monitoring indicator includes: in response to the difference between the operating status data of the hardware device and the monitoring indicator of the device type corresponding to the hardware device being less than or equal to a preset difference, displaying a warning message for the operating status data as the matching result.

5. The method according to any one of claims 1 to 4, characterized in that, The plurality of hardware devices include a rendering machine, and the rendering machine's performance data includes the rendering data of the rendering machine; The step of performing statistical analysis on the operating performance data of the hardware device and displaying the statistical analysis results of the operating performance data in the form of charts includes: performing statistical analysis on the rendering data of the rendering machine and displaying the statistical analysis results of the rendering data in the form of line charts.

6. The method according to any one of claims 1 to 4, characterized in that, The step of displaying fault repair prompt information based on the fault cause includes: obtaining a fault type library corresponding to the target operating status data, the fault type library including fault types and fault repair reference information corresponding to the fault types; matching the fault cause with the fault type library, determining a target fault type that matches the fault cause in the fault type library, and using the fault repair reference information corresponding to the target fault type as the fault repair prompt information.

7. A device monitoring method based on virtual film production, characterized in that, The method includes: receiving device monitoring operations on multiple hardware devices, the multiple hardware devices being devices located at a virtual film production shooting location, the multiple hardware devices being used to build and collect shooting scenes at the virtual film production shooting location; responding to the device monitoring operations, displaying a matching result between the operating status data of the hardware devices and monitoring indicators corresponding to the device type of the hardware devices, the matching result being used to indicate the operating fault status of the hardware devices at the virtual film production shooting location, the operating status data including operating performance data indicating the hardware performance of the hardware devices during operation; displaying a video frame corresponding to the operating video data of the hardware devices, the video frame referring to the working scene of the hardware devices at the virtual film production shooting location; the video frame including a monitoring frame of the virtual film production shooting location from a camera and a screen display of an operator device, the operator device being used to receive software operations during the virtual film production shooting process, the screen display being used to display the control software operation process corresponding to the hardware devices; responding to receiving the operating status data of the hardware devices... The system performs data analysis operations, displaying the statistical analysis results of the operational performance data in chart form. These operations are used to perform statistical analysis on specified operational status data. In response to the hardware device failing to meet the monitoring indicators for a specified time period's target operational status data, the system acquires target operational video data for that time period. This target operational video data includes monitoring videos of the hardware device and software operation videos of the hardware device's operating mechanism. The system inputs the target operational video data and the target operational status data into a fault detection model, outputting the cause of the fault in the target operational status data. The fault detection model is a neural network model pre-trained using sample data. The sample data includes sample operational status data corresponding to the faulty hardware device, as well as sample monitoring videos, sample software operation videos, and reference fault causes corresponding to the sample operational status data. Based on the fault cause, the system displays fault repair prompts, which provide a reference for fault repair of the target operational status data.

8. A device monitoring system based on virtual film production, characterized in that, The device includes: an acquisition module for acquiring operational status data and operational video data corresponding to multiple hardware devices; the multiple hardware devices are devices located on a virtual film production shooting site and used to build and collect shooting scenes of the virtual film production shooting site; the operational status data includes operational performance data indicating the hardware performance of the hardware devices during operation; a first display module for matching the operational status data of the hardware devices with monitoring indicators corresponding to the device type of the hardware devices, displaying the matching result between the operational status data and the monitoring indicators, the matching result indicating the operational failure status of the hardware devices on the virtual film production shooting site; and displaying video footage corresponding to the operational video data, the video footage referring to the working scene footage of the hardware devices on the virtual film production shooting site; the video footage includes the monitoring footage of the virtual film production shooting site from a camera and the screen display of an operator device, the operator device being used to receive software operations during the virtual film production shooting process, and the screen display being used to display the control software corresponding to the hardware devices. The operation process includes: performing statistical analysis on the operating performance data of the hardware device and displaying the statistical analysis results in chart form; the acquisition module, in response to the hardware device's target operating status data failing to meet the monitoring indicators within a specified time period, acquiring target operating video data within that time period, including monitoring videos of the hardware device and software operation process videos of the hardware device's operating mechanism; inputting the target operating video data and the target operating status data into a fault detection model and outputting the fault cause of the target operating status data; the fault detection model is a neural network model pre-trained using sample data; the sample data includes sample operating status data corresponding to faulty hardware devices, sample monitoring videos, sample software operation process videos, and reference fault causes corresponding to the sample operating status data; and displaying fault repair prompts based on the fault causes, which provide a reference for fault repair of the target operating status data.

9. A device monitoring system based on virtual film production, characterized in that, The device includes: a receiving module for receiving device monitoring operations on multiple hardware devices, the multiple hardware devices being devices located at a virtual film production shooting location, the multiple hardware devices being used to build and collect shooting scenes at the virtual film production shooting location; a second display module for responding to the device monitoring operations by displaying a matching result between the operating status data of the hardware devices and the monitoring indicators corresponding to the device type of the hardware devices, the matching result being used to indicate the operating fault status of the hardware devices at the virtual film production shooting location, the operating status data including operating performance data indicating the hardware performance of the hardware devices during operation; displaying video footage corresponding to the operating video data of the hardware devices, the video footage referring to the working scene of the hardware devices at the virtual film production shooting location; the video footage including the monitoring footage of the virtual film production shooting location from a camera and the screen display of an operator device, the operator device being used to receive software operations during the virtual film production shooting process, the screen display being used to display the control software operation process corresponding to the hardware devices; and responding to receiving the monitoring operations on the multiple hardware devices by displaying a matching result between the operating status data of the hardware devices and the monitoring indicators corresponding to the device type of the hardware devices. The analysis of hardware device operating performance data displays the statistical analysis results of the operating performance data in chart form. This analysis is used to perform statistical analysis on specified operating status data. In response to the hardware device failing to meet the monitoring indicators for a target operating status within a specified time period, target operating video data for that time period is acquired. This target operating video data includes monitoring videos of the hardware device and software operation process videos of the hardware device's operating mechanism. The target operating video data and the target operating status data are input into a fault detection model, which outputs the cause of the fault in the target operating status data. The fault detection model is a neural network model pre-trained using sample data. The sample data includes sample operating status data corresponding to faulty hardware devices, sample monitoring videos, sample software operation process videos, and reference fault causes corresponding to the sample operating status data. Based on the fault cause, fault repair prompts are displayed, providing a reference for fault repair of the target operating status data.

10. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one computer program, which is loaded and executed by the processor to implement the device monitoring method based on virtual filmmaking as described in any one of claims 1 to 7.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to implement the device monitoring method based on virtual filmmaking as described in any one of claims 1 to 7.

12. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the device monitoring method based on virtual fabrication as described in any one of claims 1 to 7.

Images