Screen model creation method, device and mobile terminal for virtual shooting system

Abstract

The present disclosure relates to a method, apparatus, and mobile terminal for creating a screen model for a virtual photography system. The method comprises: in response to configuring image acquisition parameters in an interactive page of a mobile terminal and triggering an image acquisition function, displaying multiple screen images, wherein the multiple screen images include images of at least one physical screen captured by an image acquisition device configured in the image acquisition parameters, the image acquisition device including a mobile terminal; in response to triggering a model creation function in an interactive interface of the mobile terminal, sending the multiple screen images to a computing device connected to the mobile terminal, causing the computing device to display a screen model corresponding to the physical screen based on the multiple screen images. According to embodiments of the present disclosure, screen models can be generated efficiently and conveniently, accelerating model construction efficiency and reducing screen model construction costs.

Description

Technical Field

[0001] The present disclosure relates to the technical field of virtual shooting, and in particular to a method, device, and mobile terminal for creating a screen model for a virtual shooting system. Background Art

[0002] Traditional film and television production requires significant time and manpower to select filming locations, create props, and build sets to meet filming requirements. Virtual filming (or virtual production) technology, however, replaces real-world sets with virtual scenes rendered on constructed LED screens, reducing film and television production's reliance on locations and sets, thereby lowering costs and production cycles. Virtual filming typically involves placing a virtual camera within the virtual scene rendered by a rendering engine to simulate the displacement and rotation of a physical camera. A 1:1 replica of the real-world LED screen used to display the virtual scene is also required. The virtual image within the virtual camera's field of view is then projected onto the screen model through a three-dimensional transformation, and then mapped onto the real-world LED screen.

[0003] The traditional screen model construction process requires staff to spend a lot of time and energy to measure the screen size with a ruler and then model it on the computer, which will lead to problems such as long time, inaccurate precision, and low efficiency. Alternatively, staff can also use the crew's camera to capture screen images and use the captured screen images to reconstruct the three-dimensional model of the screen. Although this method can improve modeling accuracy and efficiency to a certain extent, it requires reliance on the hardware equipment of the crew's camera, which increases the cost of model construction. In addition, if the crew's camera is not in place, the screen model cannot be built, affecting the progress of subsequent virtual shooting. Summary of the Invention

[0004] In view of this, the present disclosure proposes a screen model creation method, device and mobile terminal for a virtual shooting system, which can generate screen models efficiently and conveniently without relying on a camera to capture screen images, speed up model construction efficiency and reduce screen model construction costs.

[0005] According to one aspect of the present disclosure, a method for creating a screen model for a virtual shooting system is provided, wherein the virtual shooting system includes at least one physical screen. The method is applied to a mobile terminal, and includes: in response to configuring image acquisition parameters in an interactive page of the mobile terminal and triggering an image acquisition function, displaying multiple screen images, wherein the multiple screen images include images of the at least one physical screen captured by an image acquisition device configured in the image acquisition parameters, and the image acquisition device includes the mobile terminal; in response to triggering a model creation function in an interactive interface of the mobile terminal, sending the multiple screen images to a computing device connected to the mobile terminal, so that the computing device displays a screen model corresponding to the physical screen based on the multiple screen images.

[0006] In one possible implementation, the method further includes: displaying image quality scores corresponding to the multiple screen images, the image quality scores representing the image quality of each screen image; and / or, displaying the image coverage corresponding to the physical screen, the image coverage representing the proportion of the screen area covered by the multiple screen images relative to the total screen area of the physical screen; and / or, displaying screen error values corresponding to the multiple screen images, the screen error value representing the deviation between the actual screen size of the physical screen and the predicted screen size estimated using the multiple screen images; wherein, at least one of the image quality score, the image coverage and the screen error value is used to prompt the user whether the multiple screen images meet the screen model construction requirements.

[0007] In one possible implementation, the multiple screen images include images captured by an image acquisition device when the physical screen displays an identification graphic array, and the image acquisition parameters also include: the array density of the identification graphics in the identification graphic array to be displayed; wherein, after triggering the image acquisition function, the method further includes: controlling the physical screen to display the identification graphic array corresponding to the array density configured in the image acquisition parameters, and controlling the image acquisition device configured in the image acquisition parameters to perform image acquisition on the physical screen to obtain the multiple screen images.

[0008] In a possible implementation, the image acquisition parameters also include an image acquisition mode, and the image acquisition mode includes video acquisition or picture acquisition, wherein the controlling the image acquisition device configured in the image acquisition parameters to perform image acquisition on the physical screen to obtain the multiple screen images includes: controlling the image acquisition device configured in the image acquisition parameters to perform image acquisition on the physical screen according to the image acquisition mode configured in the image acquisition parameters to obtain multiple screen images; wherein, when the image acquisition mode configured in the image acquisition parameters is video acquisition, the multiple screen images include multiple image frames extracted from the video acquired by the image acquisition device from the physical screen; when the image acquisition mode configured in the image acquisition parameters is picture acquisition, the multiple screen images include multiple images taken by the image acquisition device from the physical screen.

[0009] In one possible implementation, the virtual shooting system also includes a camera. When the image acquisition device configured in the image acquisition parameters is the camera, controlling the image acquisition device configured in the image acquisition parameters to perform image acquisition on the physical screen to obtain the multiple screen images includes: controlling the camera to perform image acquisition on the physical screen to obtain the multiple screen images.

[0010] In one possible implementation, the interactive interface of the terminal device is also used to configure model building information of the screen model, and the model building information includes at least one of the following: the correspondence between the network addresses of each physical screen in the virtual shooting system and their corresponding screen names, whether to merge the screen models of all physical screens, and if the screen models of all physical screens are not merged, the network addresses or screen names of some physical screens to be merged.

[0011] In one possible implementation, the image acquisition parameters also include: a physical screen for displaying prompt information, wherein the prompt information represents the image acquisition progress of the physical screen by the image acquisition device. The method also includes: during the process of the image acquisition device performing image acquisition on the physical screen, controlling the physical screen configured in the image acquisition parameters to display the prompt information; and / or controlling the physical screen for which image acquisition has been completed to display a reminder mark, wherein the reminder mark is used to guide the image acquisition device to perform image acquisition on the physical screen for which acquisition has not been performed.

[0012] According to another aspect of the present disclosure, a screen model creation device for a virtual shooting system is provided, wherein the virtual shooting system includes at least one physical screen, and is characterized in that the device is applied to a mobile terminal, comprising: an image acquisition module, configured to display multiple screen images in response to configuring image acquisition parameters in an interactive page of the mobile terminal and triggering an image acquisition function, wherein the multiple screen images include images of the at least one physical screen captured by the image acquisition device configured in the image acquisition parameters, and the image acquisition device includes the mobile terminal; a model creation module, configured to send the multiple screen images to a computing device connected to the mobile terminal in response to triggering the model creation function in the interactive interface of the mobile terminal, so that the computing device displays the screen model corresponding to the physical screen based on the multiple screen images.

[0013] According to another aspect of the present disclosure, a mobile terminal is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0014] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.

[0015] According to another aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0016] According to the embodiments of the present disclosure, by configuring the mobile terminal to capture images of the physical screen, and then sending the captured screen images to the computing device, the computing device generates and displays a screen model based on the screen images. This can achieve the goal of not relying on the camera in the virtual shooting system to capture the screen image, and can efficiently and conveniently generate a screen model, which is beneficial to improving the efficiency of screen model construction and reducing hardware costs.

[0017] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0019] Figure 1 A schematic diagram illustrating an application scenario according to an embodiment of the present disclosure is shown.

[0020] Figure 2 A flowchart of a method for creating a screen model for a virtual shooting system according to an embodiment of the present disclosure is shown.

[0021] Figure 3 A schematic diagram showing a display method of prompt information and reminder signs according to an embodiment of the present disclosure.

[0022] Figure 4 A schematic diagram of an interactive interface for triggering an image acquisition function according to an embodiment of the present disclosure is shown.

[0023] Figure 5 A schematic diagram illustrating an interactive interface during an image acquisition process according to an embodiment of the present disclosure.

[0024] Figure 6 A schematic diagram showing an interactive interface after image acquisition is completed according to an embodiment of the present disclosure.

[0025] Figure 7a and Figure 7b A schematic diagram illustrating an interactive interface for configuring model building information according to an embodiment of the present disclosure is shown.

[0026] Figure 8 A schematic diagram of an interactive interface for configuring model building information according to an embodiment of the present disclosure is shown.

[0027] Figure 9 A block diagram illustrating a screen model creation apparatus for a virtual shooting system according to an embodiment of the present disclosure is shown.

[0028] Figure 10 A block diagram of an electronic device 800 according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0029] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0030] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0031] In addition, numerous specific details are provided in the following detailed description to better illustrate the present disclosure. Those skilled in the art will appreciate that the present disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art are not described in detail in order to highlight the main points of the present disclosure.

[0032] Figure 1 A schematic diagram showing an application scenario according to an embodiment of the present disclosure is shown as follows: Figure 1 As shown, the virtual shooting system includes three physical screens (021, 022, and 023). The mobile terminal 01 can execute the screen model creation method of the embodiment of the present disclosure. The mobile terminal 01 can establish communication connections with the three physical screens (021, 022, and 023) respectively. The mobile terminal 01 can establish a communication connection with the computing device 03. The mobile terminal 01 can control each physical screen to display an identification graphic array and send multiple screen images obtained by image capture of each physical screen to the computing device 03. The computing device 03 can generate and display a screen model based on the multiple screen images sent by the mobile terminal 01.

[0033] The physical screen used in the virtual shooting system can be an LED display, an LCD display, or other types, and can be a curved screen or a flat screen. It should be understood that those skilled in the art can customize the type, quantity, size, resolution, etc. of the physical screen in the virtual shooting system according to actual needs, and the embodiments of the present disclosure are not limited to this. It should be understood that the embodiments of the present disclosure do not limit the communication connection method between devices.

[0034] The screen model creation method of the embodiment of the present disclosure can be deployed on various mobile terminals through software or hardware modification. The mobile terminal involved in the embodiment of the present disclosure can be a mobile terminal that can capture screen images in a portably manner. For example, the mobile terminal can include but is not limited to a handheld device, a tablet computer, a PDA, etc.; and the computing device involved in the embodiment of the present disclosure can be an electronic device with strong computing and processing capabilities. For example, the computing device can include but is not limited to a desktop computer, a laptop computer, etc.

[0035] The mobile terminal or computing device involved in the embodiments of the present disclosure may refer to a device with a wireless connection function and / or a wired connection function. The wireless connection function refers to the ability to connect to other devices through wireless connection methods such as Wi-Fi and Bluetooth. The mobile terminal or computing device involved in the embodiments of the present disclosure may also be connected to other devices through a wired connection function. The mobile terminal or computing device involved in the embodiments of the present disclosure may be a touch screen or a non-touch screen. A touch screen mobile terminal or computing device may be controlled by clicking or sliding on the display with a finger or stylus. A non-touch screen mobile terminal may be connected to an input device such as a mouse, keyboard, or touch panel to control the mobile terminal or computing device through the input device. The embodiments of the present disclosure do not limit this.

[0036] Figure 2 FIG. 1 is a flow chart showing a method for creating a screen model for a virtual shooting system according to an embodiment of the present disclosure. Figure 2 As shown, the method includes: step S21 to step S22.

[0037] In step S21, in response to configuring image acquisition parameters in the interactive page of the mobile terminal and triggering the image acquisition function, multiple screen images are displayed, wherein the multiple screen images include images captured by the image acquisition device configured in the image acquisition parameters on at least one physical screen, and the image acquisition device includes a mobile terminal.

[0038] As described above, the screen model creation method of the embodiment of the present disclosure can be deployed in a mobile terminal through software, for example, it can be deployed in a mobile terminal in the form of an application APP. When the user wants to create a screen model, he can open the application and enter the above-mentioned interactive interface for triggering the image acquisition function. A button control for triggering the image acquisition function can be displayed in the interactive interface. The user can click the button control in the interactive interface to control the image acquisition device to start acquiring screen images.

[0039] The interactive interface for triggering the image acquisition function may also include configurations of image acquisition parameters. The image acquisition parameters may include the type of image acquisition device. A mobile terminal may be selected as the image acquisition device by default. Thus, when a user triggers the image acquisition function in the interactive interface, the mobile terminal may call upon its own camera to begin image acquisition. It should be understood that when using a mobile terminal to acquire images of a physical screen, the user may capture images of the physical screen in different postures by holding the mobile terminal, thereby obtaining multiple screen images acquired in different postures and displaying the multiple acquired screen images in the interactive interface.

[0040] In actual applications, in order to generate a screen model using a screen image, it is usually necessary to display an identification graphic in the physical screen and make the displayed identification graphic cover the entire physical screen. In this way, the captured screen image also contains the identification graphic, that is, multiple screen images include images captured when the image acquisition device displays the identification graphic array on the physical screen. In this way, when the screen image is subsequently used to generate a screen model, the identification graphic can be identified from the screen image and the two-dimensional coordinates of the feature points on the identification graphic in the screen image can be determined. Then, the two-dimensional coordinates of the feature points on the identification graphic in the screen image and the three-dimensional coordinates displayed in the physical screen can be used to construct a three-dimensional screen model of the physical screen, that is, to realize the construction of the screen model of the physical screen.

[0041] The identification graphic may, for example, be at least one of a two-dimensional identification code ArUco code, a circular identifier, and a rectangular identifier, wherein the feature points of the ArUco code may, for example, be the corner points of the ArUco code, the feature points of the circular identifier may, for example, be the center point of the circular identifier, and the feature points of the rectangular identifier may, for example, be the vertices of the rectangular identifier. In practical applications, when using the ArUco code as the identification graphic, the code value of the ArUco code may be used as the number of the feature point, and different ArUco code values may indicate different feature point numbers. The numbers of different feature points may be associated with the three-dimensional coordinates of the feature points. In this way, different feature points can be easily distinguished by identifying the code value of the ArUco code in the screen image and determining the two-dimensional coordinates and three-dimensional coordinates corresponding to the different feature points.

[0042] Considering that the physical screens in different virtual capture systems vary in size, or in other words, the display units comprising the physical screens vary in size, in order to ensure that the array of identification patterns fills the entire physical screen, the image acquisition parameters may further include: an array density of identification patterns in the array of identification patterns to be displayed; wherein the array density may include the density of identification patterns displayed in each display unit comprising the physical screen. The user may configure the array density based on actual needs. For example, if the display units are larger, the array density may be configured to be higher, so that more identification patterns are displayed in each display unit. Conversely, if the display units are smaller, the array density may be configured to be lower, so that fewer identification patterns are displayed in each display unit. The identification patterns displayed by all display units in the entire physical screen constitute the identification pattern array, and configuring the array density of identification patterns in the display units of the physical screen is equivalent to configuring the array density of the identification pattern array displayed on the physical screen.

[0043] As mentioned above, the physical screen can be an LED screen. The LED screen is a screen composed of multiple LED boxes, so the LED box can be the display unit of the LED screen. If the LED box is large in size, a larger array density can be configured so that each LED box can display more identification graphics. For example, if the configured array density is 3, it can represent that each LED box displays 3×3 identification graphics. If the LED screen is composed of 3×3 LED boxes, the number of rows and columns of the identification graphic array displayed by the LED screen are both 27. If the LED screen is composed of 4×3 LED boxes, the number of rows and columns of the identification graphic array displayed by the LED screen is 36, and the number of columns is 27.

[0044] Based on the array density configured in the aforementioned image acquisition parameters, in one possible implementation, after triggering the image acquisition function, the method further includes: controlling the physical screen to display an array of identification graphics corresponding to the array density configured in the image acquisition parameters, and controlling the image acquisition device configured in the image acquisition parameters to capture images of the physical screen to obtain multiple screen images. Controlling the physical screen to display the array of identification graphics corresponding to the array density configured in the image acquisition parameters may include: first generating the array of identification graphics based on the array density configured in the image acquisition parameters, and then transmitting the array of identification graphics to the physical screen to display the array of identification graphics on the physical screen.

[0045] As mentioned above, the image acquisition device configured in the image acquisition parameters can be a mobile terminal. Then, the image acquisition device configured in the above-mentioned control image acquisition parameters performs image acquisition on the physical screen to obtain multiple screen images. It can be understood that the mobile terminal calls the camera set by itself to perform image acquisition on the physical screen to obtain multiple screen images.

[0046] It is known that the virtual shooting system also includes a camera. If the camera is in place before the screen model is constructed, the user can also choose to use the camera to capture the screen image, that is, the user can select the image acquisition device as the camera in the above-mentioned interactive interface, and can control the mobile terminal to establish a communication connection with the camera. Based on this, in a possible implementation method, when the image acquisition device configured in the image acquisition parameters is a camera, the image acquisition device configured in the image acquisition parameters is controlled to capture images of the physical screen to obtain multiple screen images. It can also include: controlling the camera to capture images of the physical screen to obtain multiple screen images. It should be understood that the user can adjust the camera to different postures to capture images of the physical screen, and the camera can send the captured multiple screen images to the mobile terminal.

[0047] Taking into account that, in actual situations, the screen image may not only be image data directly captured by the image capture device, but also be an image frame extracted from video data captured by the image capture device. Based on this, in one possible implementation, the above-mentioned image capture parameters may further include an image capture mode, where the image capture mode includes video capture or picture capture. Controlling the image capture device configured in the image capture parameters to capture images of the physical screen to obtain multiple screen images may include: controlling the image capture device configured in the image capture parameters to capture images of the physical screen according to the image capture mode configured in the image capture parameters to obtain multiple screen images;

[0048] Specifically, when the image acquisition method configured in the image acquisition parameters is video acquisition, the multiple screen images include multiple image frames extracted from the video acquired by the image acquisition device from the physical screen; when the image acquisition method configured in the image acquisition parameters is picture acquisition, the multiple screen images include multiple images captured by the image acquisition device from the physical screen. In actual applications, after the image acquisition device begins acquiring screen images, the number of screen images acquired by the image acquisition device can also be monitored. When the number of acquired screen images reaches a specified number, the image acquisition device can be controlled to stop image acquisition, for example, by controlling a camera set on a mobile device to stop image acquisition, or by controlling a video camera to stop image acquisition.

[0049] Optionally, considering that the aforementioned image acquisition process takes a long time, a prompt message can be displayed to the user in real time to inform them of the image acquisition progress. Since the virtual capture system may include multiple physical screens, the image acquisition parameters may further include: a physical screen for displaying prompt information, wherein the prompt information indicates the image acquisition device's image acquisition progress on the physical screen. Based on this, in one possible implementation, the method further includes: during the image acquisition device's image acquisition of the physical screen, controlling the physical screen configured in the image acquisition parameters to display prompt information; and / or controlling the physical screen for which image acquisition has completed to display a reminder indicator, wherein the reminder indicator is used to guide the image acquisition device to acquire images of physical screens that have not yet been acquired. The prompt information may be information describing the image acquisition progress, for example, "Acquisition in progress, please do not block the screen," "Acquisition completed, screen model being generated," etc. It should be understood that the embodiments of this disclosure do not limit the specific content of the prompt information.

[0050] It should be understood that when the virtual shooting system includes multiple physical screens and the size of each physical screen is also large, the user may easily forget which physical screen has been shot. Therefore, the physical screen for which image capture has been completed can be determined based on the identification graphic in the captured screen image. The physical screen for which image capture has been completed can be understood as the entire area of the physical screen included in the captured screen image. Then, the physical screen for which image capture has been completed can be controlled to display a reminder logo in real time. For example, Figure 3 Shows a display method of prompt information and reminder signs, such as Figure 3 As shown, each physical screen displays an identification graphic array composed of ArUco codes, and the physical screen on the left displays a prompt message "Acquisition in progress, please do not block the screen". The green square (i.e., rectangular square) displayed in each physical screen can be a reminder mark indicating that the physical screen has completed image acquisition, which can guide the user to control the image acquisition device to acquire images of the physical screens that have not been acquired.

[0051] As described above, the identification pattern may include an ArUco code, the code value of which may indicate the number of the feature points of the identification pattern. When generating the identification pattern array, the code value of each ArUco code in the identification pattern array may be controlled to be unique, i.e., the number of the feature points in the identification pattern displayed on each physical screen is unique and known. Thus, by identifying the code values of the ArUco codes in the captured screen image, it is possible to determine whether all the identification patterns displayed on each physical screen are within the captured screen image, thereby determining whether the image capture device has completed image capture for each physical screen, i.e., identifying the physical screens for which image capture has completed. For example, if a physical screen displays 27 different ArUco codes, and only 10 different ArUco codes are identified from the currently captured screen image, then acquisition of the physical screen has not been completed. If 27 different ArUco codes are identified, then image capture of the physical screen has been completed, and the physical screens for which image capture has completed can be controlled to display the aforementioned reminder mark.

[0052] For example, Figure 4 An interactive interface for triggering an image acquisition function provided by an embodiment of the present disclosure is shown. Figure 4As shown, you can choose to use an iPad (tablet computer) or a camera as the image acquisition device for capturing screen images at "Device Selection", where the iPad can be selected by default, and the selected image acquisition device can be indicated by highlighting; you can choose whether to use video acquisition or photo acquisition (i.e., picture acquisition) at "Acquisition Mode"; you can set the array density at "Dot Density Value", for example, the dot density value set in the figure is 10, which means the array density is 10; you can set the physical screen for displaying prompt information at "Display Result Screen" by selecting the screen name of the physical screen, for example, the screen name set in the figure is a physical screen with the screen name "Screen One".

[0053] Alternatively, as Figure 4 The interactive interface shown may include a screen image display area under "Capture Picture" for displaying the screen image to be captured. When no screen image is captured, a prompt "No captured image yet" may be displayed; and the interactive interface may also include a real-scene display area for displaying the real-scene within the current field of view of the camera, so that the user can adjust the posture of the image capture device based on the displayed real-scene. When the user clicks the "Capture Screen" button control, that is, when the image capture function is triggered, the image capture device can be controlled to officially start capturing the screen image. It should be noted that Figure 4 The real-life scene displayed in the interactive interface is only an example and does not represent the actual scene captured by the camera.

[0054] When the user clicks Figure 4 After triggering the image acquisition function by clicking the button control of the "Acquisition Screen" in the Figure 5 The interactive interface during the image acquisition process is shown in Figure 5 As shown, the screen images captured in real time can be displayed in sequence in the display area below the "Capturing Picture", and the "Capturing Screen" can be transformed into a prompt of "Capturing..."; wherein, during the image capture process, the various function controls in the interactive interface can be grayed out to prevent the user from accidentally touching and affecting the image capture; if it is detected that the image capture device is moving at a fast speed, a prompt message of "The shooting speed is too fast, please move the device slowly" can also be displayed to guide the user to capture the screen image.

[0055] Considering that the screen image captured by the user-controlled image capture device may be blurred or not cover the entire physical screen, which may affect the accuracy of subsequent screen model construction, in order to facilitate the user to know whether the currently captured screen image meets the screen model construction requirements, in one possible implementation, the method further includes:

[0056] Displaying image quality scores corresponding to multiple screen images, where the image quality scores represent the image quality of each screen image; and / or,

[0057] Display the image coverage corresponding to the physical screen, where the image coverage represents the ratio of the screen area covered by multiple screen images to the total screen area of the physical screen; and / or

[0058] Displaying screen error values corresponding to the multiple screen images, where the screen error value represents a deviation between an actual screen size of the physical screen and a predicted screen size estimated using the multiple screen images;

[0059] Among them, at least one of the image quality score, image coverage and screen error value is used to prompt the user whether multiple screen images meet the screen model construction requirements.

[0060] It should be understood that those skilled in the art can use image quality evaluation indicators known in the art, such as PSNR (Peak Signal-to-Noise Ratio) peak signal-to-noise ratio, SSIM (Structural SIMilarity) structural similarity, etc., to evaluate the image quality of each screen image and obtain the image quality score of each screen image. This embodiment of the present disclosure does not limit this. This embodiment of the present disclosure does not limit the display method of the image quality score. For example, Figure 6 In an interactive interface shown, the upper right corner of the screen image displayed below "Captured Picture" can display the image quality score of the screen, for example, "Average 0.7", "Extremely High 0.9", "Extremely Poor 0.2".

[0061] As described above, the virtual shooting system may include multiple physical screens. Based on this, the image coverage may include: the global coverage corresponding to all physical screens and / or the local coverage corresponding to a single physical screen, wherein the global coverage may represent the proportion of the screen area covering all physical screens in the currently captured screen image relative to the total screen area of all physical screens, and the local coverage may represent the proportion of the screen area covering a single physical screen in the currently captured screen image relative to the total screen area of the single physical screen.

[0062] As described above, the identification graphic may include an ArUco code, and the image coverage may be determined by identifying the code value of the ArUco code in the captured screen image. For example, if there are three physical screens (021, 022, 023) in the virtual shooting system, each physical screen displays 27 different ArUco codes, and 27 ArUco codes displayed on the physical screen 021, 10 ArUco codes displayed on the physical screen 022, and 20 ArUco codes displayed on the physical screen 023 are identified from the currently captured screen image, then the global coverage may be (27+10+20) / (27+27+27)≈70%, the local coverage corresponding to the physical screen 021 may be 27 / 27=100%, the local coverage corresponding to the physical screen 022 may be 10 / 27=37%, and the local coverage corresponding to the physical screen 023 may be 20 / 27=74%.

[0063] For example, image coverage can be displayed in Figure 4 In the area below "Collection Results", "Camera Coverage" is used to display the global coverage, "Screen 1" is used to display the local coverage of the screen named "Screen 1", "Floor Screen Left 1" is used to display the local coverage of the screen named "Floor Screen Left 1", and "Screen 2" is used to display the local coverage of the screen named "Screen 2". During the image collection process, the image coverage can be determined and displayed in real time based on the real-time screen image collected. For example, Figure 5 The "Acquisition Results" section may display a form of image coverage, wherein the image coverage may be graphically displayed by the ratio of the number of highlighted squares (i.e., blue squares) relative to the number of gray squares. The more blue squares, the higher the image coverage. The area covered by the blue squares may also graphically represent the approximate screen area of the covered physical screen. For example, the approximate screen area covered by the real-life image may be determined by identifying the ArUco code in the real-life image. Of course, the number of highlighted squares may also be determined based on the screen coverage, and the gray squares may be transformed into highlighted squares in a manner that diffuses from left to right or from the center to the surrounding areas. The "Acquisition Results" section may synchronize the acquisition progress of each physical screen in real time, and the "Acquisition Picture" section may automatically display the acquired screen image.

[0064] Optionally, after completing an image acquisition, or after the number of captured screen images reaches a specified number and image acquisition is stopped, the global coverage can be displayed in the form of graphics, numbers, etc., such as Figure 6 In the interactive interface shown, "89%" is displayed at "Camera Coverage" under "Collection Results", which means that the global coverage is 89%.

[0065] In actual applications, the mobile terminal may also provide an interactive interface for configuring screen parameters, through which the user can set the screen parameters of the physical screen, wherein the screen parameters may include the screen name, actual screen size, resolution and other information of each physical screen; the actual screen size may include the length and width of the physical screen; as mentioned above, the physical screen may be an LED screen, and each LED screen may be composed of multiple LED boxes, and thus, the actual screen size of each LED screen may be set by setting the box size (length and width) of a single LED box and the number of rows and columns of LED boxes in each LED screen; the screen name of the physical screen may be used to distinguish different physical screens; and the resolution of the entire LED screen may be obtained by setting the resolution of a single LED box. If the virtual shooting system includes multiple physical screens, the screen parameters set by the user may also include the angle between each two physical screens, for example Figure 1 In the three physical screens (021, 022, and 023) shown in FIG, the angle between physical screens 021 and 022 can be 120 degrees, the angle between physical screens 021 and 023 is 90 degrees, and the angle between physical screens 022 and 023 is 90 degrees. By setting the angle between each two physical screens, the adjacent relationship between the physical screens can be known, which can facilitate the subsequent construction of the screen model.

[0066] Among them, the two-dimensional vertex coordinates of the vertices of the screen area where the physical screen is located in the screen image can be identified, and combined with the projection transformation relationship pre-calibrated by the image acquisition device (the projection transformation relationship represents the coordinate transformation relationship between the pixel coordinate system and the world coordinate system), the two-dimensional vertex coordinates of the vertices of the screen area can be projected into three-dimensional vertex coordinates, and then the three-dimensional vertex coordinates obtained by projection can be used to estimate the predicted screen size of the physical screen; then, the screen error value can be determined based on the deviation between the actual screen size of the physical screen and the predicted screen size estimated using the screen image, for example, it can be the difference between the actual screen size and the predicted screen size, or it can be the ratio between the difference between the actual screen size and the predicted screen size and the actual screen size, etc., and the embodiments of the present disclosure are not limited to this. For example, if Figure 6 The calibration error value 0.6 displayed in the interactive interface may be a representation of an image error value.

[0067] In the embodiment of the present disclosure, by displaying at least one of the image quality score, image coverage and screen error value, it is convenient for the user to judge whether the multiple screen images currently collected meet the screen model construction requirements, and to decide whether to use the currently collected screen image to build the screen model, wherein the screen construction requirements may include, for example, an image quality score greater than a preset score threshold, an image coverage greater than a preset coverage threshold, a screen error value less than a preset error threshold, etc. If the user believes that the currently collected screen image does not meet the screen model construction requirements, the mobile terminal can be used to re-collect the screen image until the collected screen image meets the screen model construction requirements, for example, by clicking Figure 6 The "Recapture" button control is shown in the figure to re-trigger the image capture function; if the user believes that the currently captured screen image meets the screen model building requirements, for example, by clicking Figure 6 The "Create Model" button control shown in the figure is used to trigger the model creation function.

[0068] As mentioned above, there is a communication connection between the mobile terminal and the physical screen. In actual applications, the network address (such as IP address) of each physical screen can usually be obtained from the local control machine used to manage all physical screens, but the correspondence between the network address and the physical screen is unknown, and thus the display content of different physical screens cannot be controlled; and the user may also have the need to merge the screen models of all or part of the physical screens in the virtual shooting system into one screen model. Based on this, the interactive interface of the terminal device can also be used to configure the model construction information of the screen model. The model construction information includes at least one of the following: the correspondence between the network addresses of each physical screen in the virtual shooting system and their corresponding screen names, whether to merge the screen models of all physical screens, and if the screen models of all physical screens are not merged, the network addresses or screen names of some physical screens to be merged.

[0069] As described above, when setting screen parameters, the user can set the screen name of the physical screen. The screen name of the physical screen can be the same as the model name of the screen model of the physical screen. Thus, the network address of each physical screen can be associated with its corresponding screen name, that is, the physical screen can be associated with the screen model. This facilitates the subsequent use of the correspondence between the network address of each physical screen and its corresponding screen name to control the display of logo graphics, prompt information, reminder icons, and other content on different physical screens.

[0070] For example, Figure 7a An interactive interface for configuring model building information is shown, such as Figure 7aAs shown, in the interactive interface, a name can be set at the "total model name" to identify the model construction information of the current interface configuration. The user can set the IP address of the "physical screen (i.e. physical screen)" (such as 30.78.248.181 to 30.78.248.181) and the model name of the "virtual screen (i.e. screen model)" (i.e. screen name) to achieve the corresponding relationship between the network address of the physical screen and the corresponding screen name, that is, to set the corresponding relationship between the physical screen and the screen model; wherein, the grayed-out IP address indicates that the physical screen has been disconnected, or that the mobile terminal cannot communicate normally with the physical screen of the grayed-out IP address; wherein, the user can choose to turn on "Merge all screens" at "Merge screens", that is, to merge the screen models of all physical screens into a whole screen model; as shown Figure 7b As shown, when the user chooses to turn off "Merge all screens", the IP addresses of each physical screen that can be merged can be displayed, and the user can select some physical screens to be merged by clicking on the IP address. For example, the highlighted IP address in the figure represents the physical screen selected by the user, that is, the user can manually select some physical screens to merge; of course, it is also possible to display the screen names of each physical screen that can be merged and select some physical screens to be merged by clicking on the screen name, and this is not limited to the embodiments of the present disclosure.

[0071] Alternatively, as Figure 7a or Figure 7b The interactive interface shown can also display information about whether the physical screen is a curved screen, so that users can know whether each physical screen is a curved screen or a flat screen. After the user sets the model building information, the user can click Figure 7a or Figure 7b The "match screen" button control shown in the figure confirms the currently set model building information, or confirms the correspondence between the network address of the currently set physical screen and the screen name; then, the user can also trigger Figure 7a or Figure 7b Click the button control indicated by "Click to enter formal modeling" to enter the subsequent image acquisition process, for example, you can enter the above Figure 4 An interactive interface for triggering the image acquisition function is shown. It should be understood that the interactive interface for configuring the model building information and the interactive interface for triggering the image acquisition function can also be the same interactive interface, and this embodiment of the present disclosure does not limit this.

[0072] Considering that the above model building information can be configured in addition to the interactive interface, it can also be imported from the local storage in the mobile terminal. Figure 7a or Figure 7b Before showing the interactive interface, for example, you can also first show Figure 8In the interactive interface shown, if the user clicks "Import local UE model" in the interactive interface, the process of importing model building information from local storage can be entered. After importing the model building information from local storage, the above Figure 4 The interactive interface shown; if the user clicks "New Model", they can enter the above Figure 7a or Figure 7b The interactive interface shown is used to configure model building information; Figure 8 The model name corresponding to the configured or imported model building information can be displayed under "Select Model" in the interactive interface. In this way, the user can also select any model name in the interactive interface to set the model building information. For example, if the user selects "Lighthouse Curved Screen", it means that the user has selected the model building information with the model name "Lighthouse Curved Screen" to execute the subsequent screen model building process, that is, the user can directly enter the above Figure 4 The interactive interface shown.

[0073] It should be noted that the above Figures 4 to 8 The interactive interfaces shown are some possible implementation methods provided by the embodiments of the present disclosure. In fact, those skilled in the art can design the page layout, included functional controls, etc. of each interactive interface according to actual needs, and the embodiments of the present disclosure are not limited to this.

[0074] In step S22, in response to triggering the model creation function in the interactive interface of the mobile terminal, multiple screen images are sent to a computing device connected to the mobile terminal, so that the computing device displays a screen model corresponding to the physical screen based on the multiple screen images.

[0075] It should be understood that screen model construction usually requires a large amount of computing power, and the computing power of the mobile terminal may not be able to efficiently use the screen image to construct the screen model. Therefore, after the user triggers the model creation function through the interactive interface, the multiple screen images currently collected can be sent to the computing device, and the computing device can be deployed with a model construction algorithm known in the art to realize the construction of a screen model of the physical screen based on multiple screen images.

[0076] In one possible implementation, a three-dimensional reconstruction algorithm can be used to construct a screen model of a physical screen. Specifically, the two-dimensional coordinates of feature points on the identification graphics in the screen image can be extracted, and then the two-dimensional coordinates of each feature point in the screen image and the three-dimensional coordinates when displayed on the physical screen can be used, combined with the projection transformation relationship corresponding to the image acquisition device, to reconstruct the three-dimensional screen model of the physical screen.

[0077] As described above, the user can also set the screen parameters of the physical screen. In one possible implementation, the screen parameters and the screen image can be combined to construct a screen model of the physical screen. Specifically, an initial screen model of the physical screen can be first constructed based on the screen parameters, and the feature points of the identification graphic can be identified from the screen image. Then, based on the two-dimensional coordinates of the feature points in the screen image, the initial screen model can be optimized to obtain a screen model of the physical screen. It should be understood that when the identification graphic is displayed on the physical screen, the three-dimensional coordinates of the feature points on the identification graphic on the physical screen can be determined. There is a correspondence between the two-dimensional coordinates of the feature points on the screen image and the three-dimensional coordinates of the feature points on the initial screen model, and there is also a correspondence between the three-dimensional coordinates of the feature points on the initial screen model and the three-dimensional coordinates of the feature points on the physical screen. The initial screen model can then be optimized based on the above two correspondences so that the positions of the feature points on the physical screen are the same as those on the optimized screen model, thereby obtaining a screen model corresponding to the physical screen. The screen model of the physical screen can be a screen model that is a 1:1 restoration of the physical screen.

[0078] As described above, the user may also choose to merge all or part of the screen models of the physical screen when configuring the model building information. Therefore, the computing device may also merge the generated screen models of all or part of the physical screen into one screen model based on the model building information.

[0079] According to an embodiment of the present invention, by configuring a mobile terminal to capture images of a physical screen, and then sending the captured screen images to a computing device, the computing device generates and displays a screen model based on the screen images. This allows the screen model to be generated efficiently and conveniently without relying on the camera in the virtual shooting system to capture the screen images, thereby improving the efficiency of screen model construction and reducing hardware costs.

[0080] The screen model creation method of the disclosed embodiment can be applied to a virtual filming system, aiming to quickly create a screen model by capturing screen images using a portable mobile terminal to prepare for virtual filming. In the screen model creation method of the disclosed embodiment, the screen image can be captured by video capture to speed up the pre-filming preparation time. Since the traditional modeling process requires a lot of time and effort, workers need to use a ruler to measure the screen size in front of the screen and then model it on a computer. This has problems such as long time, inaccurate accuracy, and low efficiency. The screen model creation method of the disclosed embodiment can capture the screen image of the ArUco code displayed on the physical screen by video capture, thereby quickly generating a highly accurate screen model, avoiding errors caused by manual measurement, and more quickly obtaining the screen model, thereby speeding up the pre-filming preparation time. The screen model creation method of the disclosed embodiment can also provide a more realistic and accurate screen model. Using video capture to capture the screen image can quickly capture the details and features of the screen and convert them into an accurate three-dimensional screen model. The relationship between the screen models can also be quickly obtained and merged, thus obtaining a more realistic and accurate screen model, which is helpful for subsequent physical screen calibration. In addition, the traditional image acquisition method is to capture images through a camera, but the cameras on the market basically follow the crew, so screen modeling can only start after the camera is in place, which wastes the early preparation time. The screen model creation method of the embodiment of the present invention can be an application deployed on a mobile terminal, so that it can be prepared for modeling in advance without relying on the hardware device of the camera, saving preparation time for virtual shooting. Moreover, using a mobile terminal in combination with a video acquisition method to create a screen model can also reduce modeling costs. The traditional modeling process may require hiring professional modelers, etc., while shooting videos can be handed over to any non-professional for practical operation, greatly reducing modeling costs, which is especially important for shooting projects with limited budgets. In summary, the screen model creation method of the embodiment of the present invention, the screen model creation method using a mobile terminal in combination with a video acquisition method, has significant value, that is, a method for creating a screen model by shooting videos that does not rely on the hardware of a camera is designed, which can speed up the preparation process before shooting, provide a more realistic and accurate model, and reduce costs.

[0081] Figure 9 A block diagram of a screen model creation device for a virtual shooting system according to an embodiment of the present disclosure is shown. The virtual shooting system includes at least one physical screen. The device is applied to a mobile terminal and includes:

[0082] An image acquisition module 901 is configured to display a plurality of screen images in response to configuring image acquisition parameters in an interactive page of the mobile terminal and triggering an image acquisition function, wherein the plurality of screen images include images acquired by an image acquisition device configured in the image acquisition parameters of the at least one physical screen, the image acquisition device including the mobile terminal;

[0083] The model creation module 902 is used to send the multiple screen images to a computing device connected to the mobile terminal in response to triggering the model creation function in the interactive interface of the mobile terminal, so that the computing device displays the screen model corresponding to the physical screen based on the multiple screen images.

[0084] In one possible implementation, the device also includes: displaying image quality scores corresponding to the multiple screen images, the image quality scores representing the image quality of each screen image; and / or, displaying the image coverage corresponding to the physical screen, the image coverage representing the proportion of the screen area covered by the multiple screen images relative to the total screen area of the physical screen; and / or, displaying screen error values corresponding to the multiple screen images, the screen error value representing the deviation between the actual screen size of the physical screen and the predicted screen size estimated using the multiple screen images; wherein, at least one of the image quality score, the image coverage and the screen error value is used to prompt the user whether the multiple screen images meet the screen model construction requirements.

[0085] In one possible implementation, the multiple screen images include images captured by an image capture device when the physical screen displays an identification graphic array, and the image capture parameters also include: an array density of identification graphics in the identification graphic array to be displayed; wherein, after triggering the image capture function, the device further includes: a first control module, configured to control the physical screen to display the identification graphic array corresponding to the array density configured in the image capture parameters, and control the image capture device configured in the image capture parameters to perform image capture on the physical screen to obtain the multiple screen images.

[0086] In a possible implementation, the image acquisition parameters also include an image acquisition mode, and the image acquisition mode includes video acquisition or picture acquisition, wherein the controlling the image acquisition device configured in the image acquisition parameters to perform image acquisition on the physical screen to obtain the multiple screen images includes: controlling the image acquisition device configured in the image acquisition parameters to perform image acquisition on the physical screen according to the image acquisition mode configured in the image acquisition parameters to obtain multiple screen images; wherein, when the image acquisition mode configured in the image acquisition parameters is video acquisition, the multiple screen images include multiple image frames extracted from the video acquired by the image acquisition device from the physical screen; when the image acquisition mode configured in the image acquisition parameters is picture acquisition, the multiple screen images include multiple images taken by the image acquisition device from the physical screen.

[0087] In one possible implementation, the virtual shooting system also includes a camera. When the image acquisition device configured in the image acquisition parameters is the camera, controlling the image acquisition device configured in the image acquisition parameters to perform image acquisition on the physical screen to obtain the multiple screen images includes: controlling the camera to perform image acquisition on the physical screen to obtain the multiple screen images.

[0088] In one possible implementation, the interactive interface of the terminal device is also used to configure model building information of the screen model, and the model building information includes at least one of the following: the correspondence between the network addresses of each physical screen in the virtual shooting system and their corresponding screen names, whether to merge the screen models of all physical screens, and if the screen models of all physical screens are not merged, the network addresses or screen names of some physical screens to be merged.

[0089] In one possible implementation, the image acquisition parameters also include: a physical screen for displaying prompt information, wherein the prompt information represents the image acquisition progress of the physical screen by the image acquisition device, and the device also includes: a second control module for controlling the physical screen configured in the image acquisition parameters to display the prompt information during the process of the image acquisition device performing image acquisition on the physical screen; and / or controlling the physical screen for which image acquisition has been completed to display a reminder mark, wherein the reminder mark is used to guide the image acquisition device to perform image acquisition on the physical screen for which acquisition has not been performed.

[0090] According to the embodiments of the present disclosure, by configuring the mobile terminal to capture images of the physical screen, and then sending the captured screen images to the computing device, the computing device generates and displays a screen model based on the screen images. This allows the screen model to be generated efficiently and conveniently without relying on the camera in the virtual shooting system to capture the screen images, thereby improving the efficiency of screen model construction and reducing hardware costs.

[0091] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0092] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0093] An embodiment of the present disclosure further proposes a mobile terminal, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0094] An embodiment of the present disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0095] Figure 10 FIG. 8 is a block diagram of an electronic device 800 according to an embodiment of the present disclosure. The electronic device 800 may be provided as a mobile terminal or a computing device. Figure 10 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output interface 812 (I / O interface), a sensor component 814 , and a communication component 816 .

[0096] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 802 may include one or more modules to facilitate interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

[0097] The memory 804 is configured to store various types of data to support operations on the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0098] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0099] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0100] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), which is configured to receive external audio signals when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0101] The input / output interface 812 provides an interface between the processing component 802 and peripheral interface modules, such as a keyboard, a click wheel, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0102] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the electronic device 800, the relative positioning of components, such as the display and keypad of the electronic device 800. The sensor assembly 814 can also detect changes in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and temperature changes of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0103] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0104] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0105] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 804 including computer program instructions. The computer program instructions can be executed by the processor 820 of the electronic device 800 to perform the above method.

[0106] The present disclosure may be a system, method and / or computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0107] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination thereof. As used herein, a computer-readable storage medium is not to be construed as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through an electrical wire.

[0108] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0109] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, C++, and conventional procedural programming languages such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions. The electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0110] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0111] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0112] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0113] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the prescribed logical function. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0114] While various embodiments of the present disclosure have been described above, the foregoing description is intended to be illustrative, non-exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or technological improvements in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for creating a screen model for a virtual shooting system, wherein the virtual shooting system includes at least one physical screen, characterized in that: The method is applied to a mobile terminal and includes: In response to configuring image acquisition parameters in an interactive page of the mobile terminal and triggering an image acquisition function, a plurality of screen images are displayed, wherein the plurality of screen images include images of the at least one physical screen acquired by an image acquisition device configured in the image acquisition parameters, the image acquisition device including the mobile terminal; wherein a logo graphic is displayed on the physical screen so that the displayed logo graphic covers the entire physical screen, so that the plurality of acquired screen images also include the logo graphic; In response to triggering a model creation function in the interactive interface of the mobile terminal, the multiple screen images are sent to a computing device connected to the mobile terminal, so that the computing device displays a screen model corresponding to the physical screen based on the multiple screen images; wherein, the computing device identifies an identification graphic from the screen image, and determines the two-dimensional coordinates of the feature points on the identification graphic in the screen image, and uses the two-dimensional coordinates of the feature points on the identification graphic in the screen image and the three-dimensional coordinates displayed in the physical screen to construct a three-dimensional screen model of the physical screen.

2. The method according to claim 1, characterized in that The method further comprises: displaying image quality scores corresponding to the plurality of screen images, wherein the image quality score represents the image quality of each screen image; and / or, Displaying the image coverage rate corresponding to the physical screen, where the image coverage rate represents the proportion of the screen area covered by the multiple screen images relative to the total screen area of the physical screen; and / or, displaying screen error values corresponding to the plurality of screen images, where the screen error value represents a deviation between an actual screen size of the physical screen and a predicted screen size estimated using the plurality of screen images; Among them, at least one of the image quality score, the image coverage and the screen error value is used to prompt the user whether the multiple screen images meet the screen model construction requirements.

3. The method according to claim 1, characterized in that The plurality of screen images include images captured by an image capture device when the physical screen displays an array of identification graphics. The image capture parameters further include: an array density of identification graphics in the array of identification graphics to be displayed. After triggering the image capture function, the method further includes: The physical screen is controlled to display an identification graphic array corresponding to the array density configured in the image acquisition parameters, and the image acquisition device configured in the image acquisition parameters is controlled to perform image acquisition on the physical screen to obtain the multiple screen images.

4. The method according to claim 3, characterized in that The image acquisition parameters also include an image acquisition mode, which includes video acquisition or picture acquisition. The controlling the image acquisition device configured in the image acquisition parameters to acquire images of the physical screen to obtain the multiple screen images includes: Controlling the image acquisition device configured in the image acquisition parameters to acquire images of the physical screen according to the image acquisition method configured in the image acquisition parameters to obtain multiple screen images; Among them, when the image acquisition method configured in the image acquisition parameters is video acquisition, the multiple screen images include multiple image frames extracted from the video acquired by the image acquisition device on the physical screen; when the image acquisition method configured in the image acquisition parameters is picture acquisition, the multiple screen images include multiple images taken by the image acquisition device on the physical screen.

5. The method according to claim 3, characterized in that The virtual shooting system also includes a camera. When the image acquisition device configured in the image acquisition parameters is the camera, controlling the image acquisition device configured in the image acquisition parameters to acquire images of the physical screen to obtain the multiple screen images includes: controlling the camera to acquire images of the physical screen to obtain the multiple screen images.

6. The method according to claim 1, characterized in that The interactive interface of the mobile terminal is also used to configure model construction information of the screen model, and the model construction information includes at least one of the following: the correspondence between the network address of each physical screen in the virtual shooting system and the corresponding screen name, whether to merge the screen models of all physical screens, and if the screen models of all physical screens are not merged, the network addresses or screen names of some physical screens to be merged.

7. The method according to claim 1, characterized in that The image acquisition parameters further include: a physical screen for displaying prompt information, wherein the prompt information represents the image acquisition progress of the physical screen by the image acquisition device. The method further includes: During the process of the image acquisition device acquiring an image of the physical screen, controlling the physical screen configured in the image acquisition parameters to display the prompt information; and / or, The entity screen for which image acquisition has been completed is controlled to display a reminder mark, wherein the reminder mark is used to guide the image acquisition device to acquire images from the entity screen for which acquisition has not been performed.

8. A screen model creation device for a virtual shooting system, wherein the virtual shooting system includes at least one physical screen, characterized in that: The device is applied to a mobile terminal and includes: an image acquisition module, configured to, in response to configuring image acquisition parameters in an interactive page of the mobile terminal and triggering an image acquisition function, display a plurality of screen images, wherein the plurality of screen images include images acquired by an image acquisition device configured in the image acquisition parameters of the at least one physical screen, the image acquisition device including the mobile terminal; wherein an identification graphic is displayed on the physical screen so that the displayed identification graphic covers the entire physical screen, so that the plurality of acquired screen images also include the identification graphic; A model creation module is used to send the multiple screen images to a computing device connected to the mobile terminal in response to triggering a model creation function in the interactive interface of the mobile terminal, so that the computing device displays a screen model corresponding to the physical screen based on the multiple screen images; wherein the computing device recognizes an identification graphic from the screen image and determines the two-dimensional coordinates of the feature points on the identification graphic in the screen image, and uses the two-dimensional coordinates of the feature points on the identification graphic in the screen image and the three-dimensional coordinates displayed on the physical screen to construct a three-dimensional screen model of the physical screen.

9. A mobile terminal, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method according to any one of claims 1 to 7 when executing the instructions stored in the memory.

10. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images