Twin system video mapping method, device, electronic device and storage medium

Abstract

The embodiments of the present application disclose a twin system video mapping method, device, electronic device and storage medium. The video parameters of the device to be mapped are obtained; based on the video parameters, the twin mapping parameters corresponding to the device to be mapped in the three-dimensional simulation scene of the twin system are determined; based on the video to be mapped collected by the device to be mapped, the corresponding light source texture and the twin mapping parameters, the video to be mapped is mapped into the twin system. The embodiments of the present application improve the realism of video mapping into the twin system and enrich the applicable scenarios.

Description

Technical Field

[0001] Embodiments of the present application relate to image technology, and in particular, to a twin system video mapping method, device, electronic device and storage medium. Background Art

[0002] The twin system is a technology that creates a digital model corresponding to the physical entity in a virtual space to achieve comprehensive monitoring, prediction and analysis of the physical entity. It is of great significance in the field of monitoring.

[0003] In the prior art, the video stream is processed by opencv (a professional term, a graphics processing software) in the twin system, which limits the applicable scenarios, the fusion visual effect is poor, and the video mapping to the twin system lacks realism. Summary of the invention

[0004] The present application provides a twin system video mapping method, device, electronic device and storage medium to improve the realism of video mapping into the twin system and enrich the applicable scenarios.

[0005] In a first aspect, an embodiment of the present application provides a twin system video mapping method, the twin system video mapping method comprising:

[0006] Get the video parameters of the device to be mapped;

[0007] According to the video parameters, determine the twin mapping parameters corresponding to the device to be mapped in the three-dimensional simulation scene of the twin system;

[0008] According to the video to be mapped collected by the device to be mapped, the corresponding light source texture and twin mapping parameters, the video to be mapped is mapped into the twin system.

[0009] In a second aspect, an embodiment of the present application further provides a twin system video mapping device, the twin system video mapping device comprising:

[0010] A video parameter acquisition module is used to obtain video parameters of the device to be mapped;

[0011] A twin mapping parameter determination module is used to determine the twin mapping parameters corresponding to the device to be mapped in the three-dimensional simulation scene of the twin system according to the video parameters;

[0012] The video mapping module is used to map the video to be mapped into the twin system according to the video to be mapped collected by the device to be mapped, the corresponding light source texture and the twin mapping parameters.

[0013] In a third aspect, an embodiment of the present application further provides an electronic device, the electronic device comprising:

[0014] one or more processors;

[0015] A storage device for storing one or more programs;

[0016] When one or more programs are executed by one or more processors, one or more processors implement any twin system video mapping method provided in the embodiments of the present application.

[0017] In a fourth aspect, an embodiment of the present application also provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute any twin system video mapping method provided in an embodiment of the present application.

[0018] This application obtains the video parameters of the device to be mapped; according to the video parameters, determines the twin mapping parameters corresponding to the device to be mapped in the three-dimensional simulation scene of the twin system; maps the video to be mapped to the twin system according to the video to be mapped collected by the device to be mapped, the corresponding light source texture and the twin mapping parameters, and can imitate the effects of various light sources in the real scene through the light source texture, and project the video to be mapped received by the monitoring from the monitoring position and angle to the twin system through the light source, allowing accurate adjustment of the projection visual effect according to the shape and material properties of the solid surface in the real scene, whether it is a flat surface or a complex curved surface, it can fit well, enrich the applicable scenes, and improve the realism of the video mapping to the twin system. Therefore, the technical solution of this application solves the problem that the video stream is processed by opencv, the usable scenes are limited, and the video mapping to the twin system has no sense of reality, and achieves the effect of enriching the applicable scenes and improving the realism of the video mapping to the twin system. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flowchart of a twin system video mapping method in Embodiment 1 of the present application;

[0020] Figure 2 It is a flowchart of a twin system video mapping method in Embodiment 2 of the present application;

[0021] Figure 3 It is a structural schematic diagram of a twin system video mapping device in Embodiment 3 of the present application;

[0022] Figure 4 It is a structural diagram of an electronic device in Embodiment 4 of the present application. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0024] It should be noted that the terms "first" and "second" etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] Embodiment 1

[0026] Figure 1 A flowchart of a twin system video mapping method provided in Example 1 of the present application. This embodiment is applicable to the situation of mapping a video to a three-dimensional space in a twin system. The method can be executed by a twin system video mapping device, which can be implemented in software and / or hardware and specifically configured in the twin system.

[0027] See also Figure 1 The twin system video mapping method shown specifically includes the following steps:

[0028] S110: Obtain video parameters of the device to be mapped.

[0029] The device to be mapped may be an electronic device that obtains the video to be mapped. For example, the device to be mapped may be a surveillance camera. The video parameters may be parameters related to the acquisition of the video by the device to be mapped, and are used to determine the twin mapping parameters of the twin video corresponding to the device to be mapped in the three-dimensional simulation scene of the twin system. Exemplarily, the video parameters may include: device position, video stream address, and focal length, aperture, field of view, close-up distance, and depth of field of the camera, etc., which are not specifically limited in this application.

[0030] S120. Determine, based on the video parameters, the twin mapping parameters corresponding to the device to be mapped in the three-dimensional simulation scene of the twin system.

[0031] The twin mapping parameters can be used to convert the video parameters of the device to be mapped into the corresponding parameters in the twin system, so as to accurately map the video monitored by the device to be mapped into the twin system. Exemplarily, the twin mapping parameters may include projection range and projection brightness, etc. For example, a projection control entity in a twin system can be generated according to the video parameters, the position and angle of the projection control can be bound according to the position and orientation of the device to be mapped in the real space, the projection range of the projection control can be automatically set according to the focal length and field of view of the device to be mapped, and the projection brightness of the projection control can be automatically set according to the aperture parameters, etc., so as to seamlessly integrate the video stream in the real scene into the three-dimensional simulation scene of the corresponding twin system.

[0032] Optionally, a three-dimensional simulation scene of the twin system can be constructed through the unity3d engine (a professional term, an image processing engine). Collect data such as photos and cad (Computer Aided Design) drawings, and perform high-precision modeling according to the actual environment through 3dMax (a professional term, a modeling software) software to obtain a target three-dimensional model that truly reflects the real environment one to one. Import the target three-dimensional model into the unity3d engine, configure the sky box and lighting, modify and adjust the model map, save the scene, and initially construct a twin scene entity. For example, in a hospital scene, twin scene entities corresponding to each department and key area can be constructed based on the unity3d engine.

[0033] Optionally, you can identify the video monitoring areas that need to be focused on in the twin system and collect the video parameters of the devices to be mapped in the key areas. The operation and maintenance personnel select the devices to be mapped in the twin system, drag the model corresponding to the devices to be mapped to the position in the twin system, calibrate the video parameters to be bound, including the camera position and orientation, as well as the video stream address and camera hardware parameters, and store the data in the background server.

[0034] S130. Map the video to be mapped into the twin system according to the video to be mapped collected by the device to be mapped, the corresponding light source texture and the twin mapping parameters.

[0035] The video to be mapped can be a video of a real scene captured by the device to be mapped, which is used to map it to the twin system. The light source texture can be the texture of different light sources, which is used to simulate different light source projections to form different light and shadow effects, thereby enhancing the realism and immersion of the video twin scene. After obtaining the video to be mapped, the mapping area corresponding to the video to be mapped in the twin system can be determined according to the video parameters of the device to be mapped that obtains the video to be mapped, and the video to be mapped is mapped to the corresponding area in the twin system by simulating the on-site light source according to the light source texture. Optionally, after mapping the video to be mapped to the twin system, manual intervention can be performed to fine-tune the projection control offset brightness, throw ratio, and screen aspect ratio, etc., to ensure the perfect integration of the video to be mapped and the three-dimensional simulated scene in the twin system, thereby improving the realism of the picture in the twin system.

[0036] Digital twin technology plays a significant role in improving the effectiveness of security monitoring and training exercises in specific areas. However, the video mapping technology in the existing twin system has requirements for the camera position and camera hardware. The video stream is processed by opencv, and the applicable scenarios are limited. The existing video twin needs to pre-process and edit the video in advance. It is difficult to merge two-dimensional videos into three-dimensional ones, especially multiple videos cannot be directly synthesized into a three-dimensional video model, and it cannot be well presented on complex surfaces. The video is played on multiple surfaces by tilting the camera to form a pseudo-three-dimensional video. When viewed from a close distance, it can be seen that the fusion of the video plane and the surrounding environment is unreal.

[0037] The technical solution of this embodiment is to obtain the video parameters of the device to be mapped; determine the twin mapping parameters corresponding to the device to be mapped in the three-dimensional simulation scene of the twin system according to the video parameters; map the video to be mapped into the twin system according to the video to be mapped collected by the device to be mapped, the corresponding light source texture and the twin mapping parameters, and imitate the effects produced by various light sources in the real scene through the light source texture, and project the video to be mapped received by the monitoring from the monitoring position and angle to the twin system through the light source, allowing accurate adjustment of the projection visual effect according to the shape and material properties of the solid surface in the real scene, so that both flat surfaces and complex curved surfaces can be well fitted, enriching the applicable scenarios and improving the realism of the video mapping into the twin system. Therefore, the technical solution of this application solves the problem that the available scenarios are limited and the video mapping into the twin system lacks realism when processing the video stream through OpenCV, and achieves the effect of enriching the applicable scenarios and improving the realism of the video mapping into the twin system.

[0038] Embodiment 2

[0039] Figure 2 This is a flowchart of a twin system video mapping method provided in Example 2 of the present application. The technical solution of this embodiment is further refined on the basis of the above technical solution.

[0040] Furthermore, "map the video to be mapped into the twin system according to the video to be mapped collected by the device to be mapped, the corresponding light source texture and twin mapping parameters" is refined as: "For each frame image in the video to be mapped, obtain the target pixel value in the frame image; determine the light source parameter of each target pixel point according to the target pixel value and the corresponding light source texture; determine the corresponding light source parameters according to the texture coordinates of each light-receiving surface in the twin system to map the frame image into the twin system" to map each frame image into the twin system.

[0041] See also Figure 2 A twin system video mapping method is shown, comprising:

[0042] S210: Obtain video parameters of the device to be mapped.

[0043] S220. Determine, based on the video parameters, the twin mapping parameters corresponding to the device to be mapped in the three-dimensional simulation scene of the twin system.

[0044] S230: For each frame of image in the video to be mapped, obtain each target pixel value in the frame of image.

[0045] The target pixel value can be the pixel value of each pixel point that constitutes each frame of the image. The video to be mapped is composed of multiple frames of images. For each frame of the video to be mapped, the target pixel values ​​in the image are read to facilitate subsequent pixel-by-pixel processing.

[0046] S240: Determine the light source parameter of each target pixel point according to each target pixel value and the corresponding light source texture.

[0047] The light source texture needs to be considered in the mapping process of the twin system. According to the light source texture of the target pixel value, the pixel value mapped to the twin system is determined. The light source parameters can include transparency texture and color texture. According to the target pixel value, the transparency texture and color texture corresponding to each pixel point can be determined as its light source parameter to determine the light source texture feature corresponding to each target pixel value mapped to the twin system.

[0048] In an optional embodiment, the light source parameters of each target pixel value are determined according to each target pixel value and the corresponding light source texture, including: determining a light transmission channel map and a three-primary color light source according to the light source texture corresponding to each target pixel value; determining the light source parameters of each target pixel point according to the light transmission channel map and the three-primary color light source.

[0049] Transparency can be the transmittance of a light source, used to simulate the light and shadow effect of a light source shining on an object. A light transmission channel map can be a texture map that simulates the transmittance effect of a light source, used to determine the transparency of each pixel. The three primary colors can be the three primary colors that form the color of a pixel, specifically including red, green, and blue. The three primary color light source is used to determine the color characteristics of each pixel.

[0050] In the Unity engine, the light source parameters of each target pixel can be determined through Cookie (a professional term, a type of light source). Cookie refers specifically to the texture used on the light source (such as a point light source, spotlight or directional light), usually an alpha channel (a professional term, a grayscale channel) texture map, which is used to simulate the shape of the light source or the masking effect. The alpha value determines the transparency of the light. When the alpha value is 1 (white), it means that the light in the area is completely passed; when the alpha value is 0 (black), it means that the light in the area is completely blocked. By marking the alpha value, the transparent channel map can be determined to determine the shape of the projection. The transparent channel map, also known as the Cookie map, can be a square 2D alpha texture or a cube map, which is used on the light source. The Cookie map is placed in front of the light source and moves with the light source, thereby affecting the shape and intensity distribution of the light emitted by the light source. Therefore, by designing alpha patterns of different shapes, a variety of lighting effects can be created. Cookies are dynamically generated, and then the video frame is projected through the three primary color light source and the transparent channel map, that is, a picture is filled pixel by pixel. There is an editor in the twin system that can expose the projection control parameters, which can then be fine-tuned by the operation and maintenance personnel. According to the on-site environment, the light source is simulated, and the corresponding monitoring hardware is selected through the video twin editing tool. After simple adjustments, the effect can be intuitively seen.

[0051] By determining the transmittance channel map based on the transparency corresponding to each pixel point, and determining the light source parameters of each target pixel value based on each target pixel value, the three primary color light source and the transmittance channel map, through lighting simulation and pixel-by-pixel projection of the three primary colors, it is possible to reproduce light and shadow effects and color details that are very close to the real world on the physical surface, thereby enhancing the realism and immersion of the video twin scene.

[0052] S250. Determine corresponding light source parameters according to the texture coordinates of each light-receiving surface in the twin system to map the frame image into the twin system.

[0053] Determine the corresponding light source parameters according to the texture coordinates of each light-receiving surface in the twin system, including: determine the texture coordinates corresponding to each light-receiving surface according to the direction vector of each light-receiving surface in the twin system relative to the light source; determine the light source parameters corresponding to each texture coordinate.

[0054] The light-receiving surface can be any surface in the scene that can receive light. For example, when using shader technology to render a 3D model, it is necessary to determine the light-receiving surface. Texture coordinates are used to determine the light source parameters corresponding to each pixel in the light-receiving surface. After determining the texture coordinates of each light-receiving surface, the corresponding light source parameters can be obtained according to the texture coordinates, and then the frame image can be mapped to the twin system according to the light source parameters, and the monitoring receiving screen is projected from the monitoring position and angle to the scene through the light source. It allows precise adjustment of the projection visual effect according to the shape and material properties of the solid surface, and it can fit well whether it is a flat surface or a complex curved surface.

[0055] In an optional embodiment, the corresponding light source parameters are determined according to the texture coordinates of each light-receiving surface in the twin system, including: determining the texture coordinates corresponding to each light-receiving surface according to the direction vector of each light-receiving surface in the twin system relative to the light source; and determining the light source parameters corresponding to each texture coordinate.

[0056] Calculate the direction vector of each illuminated surface relative to the light source. Based on the relationship between the direction vector and the cone angle range of the light source, it can be determined whether the vector is within the effective area of ​​the Cookie projection. If it is within the effective area, the direction vector is normalized and converted into Cookie texture coordinates. The corresponding light source parameters can be determined based on the texture coordinates. According to the sampled Alpha value, adjust the light intensity or color contribution of the light to the illuminated surface. Then form a video twin by playing more than 30 frames of images per second.

[0057] Since texture coordinates need to be between 0 and 1, the direction vector needs to be normalized. A normalized direction vector has a length of 1, making it easier to convert to a valid texture coordinate. Normalizing a vector is the process of dividing the vector by its own length to ensure that its length is 1. The scaling factor and offset are applied to the normalized direction vector to convert it to texture coordinates (often called UV coordinates). The scaling factor and offset are calculated to ensure that the converted coordinate values ​​fall within the valid range of the texture, which is between 0 and 1. The scaling factor is used to adjust the size of the direction vector so that it fits the dimensions of the texture. For example, if the texture is 512x512 pixels, the scaling factor on the X and Y axes might be 512. However, because texture coordinates need to be between 0 and 1, the scaling factor actually used is the inverse of the texture size, which is 1 / 512. Therefore, even if the original vector has a large range, it will be scaled to give a value between 0 and 1. The offset is used to adjust the coordinate origin so that the texture can be correctly positioned within the entire texture space. In some cases, the texture may need to start at a non-origin location in space, in which case an offset will need to be applied. The offset is usually added to the scaled coordinate value to ensure that the coordinate falls within the visible area of ​​the texture. After applying the scale and offset, the normalized direction vector can be converted to texture coordinates. Specifically, the scale and offset can be multiplied by the scale factor multiplied by the normalized direction vector and then added to the offset.

[0058] By determining the texture coordinates corresponding to each illuminated surface according to the direction vector of each illuminated surface in the twin system relative to the light source, and determining the light source parameters corresponding to each texture coordinate, it is possible to accurately adjust the projection visual effect according to the shape and material properties of the solid surface, so that both flat surfaces and complex curved surfaces can be well fitted, enriching the applicable scenarios.

[0059] In an optional embodiment, the twin system video mapping method further includes: obtaining monitoring data in the twin entity in the video to be mapped, and mapping the monitoring data to the corresponding twin entity in the twin system.

[0060] The target object can be a pre-set target device to be detected. For example, in a hospital, the target object can be an elevator, hospital medical gas (oxygen tanks and busbars, etc.), uninterruptible power supply (UPS), refrigeration room, fire pump room, water supply and drainage, smoke sensor, temperature sensor and other equipment detection hardware. The detection data will be fed back to the twin entity in the twin system in real time through the IoT middle platform. The twin entity can be the entity corresponding to the target object in the twin system. The monitoring data is mapped to the corresponding twin entity of the twin system through the preset data interface. The user can obtain the monitoring data of the target object in the video to be mapped, which is convenient for viewing the monitoring data of each target object in real time through the twin system.

[0061] Optionally, object detection and tracking algorithms (such as FairMOT) can be used and third-party data interfaces can be accessed to perform real-time identification and continuous tracking of people, vehicles, and fires, and feed back to the corresponding twin entities of the twin system.

[0062] For example, the positioning and inspection trajectory of security personnel can be presented in the twin system. Bluetooth locators are deployed on site, and the operation and maintenance personnel bind the locators to the twin entities in the twin system. Security personnel wear signal locators and obtain the location of security personnel through a three-point positioning algorithm. The location data is transmitted to the background, and the location and historical trajectory of security personnel can be viewed in real time. Specifically, a beacon (a professional term, a locator) locator is deployed on site, and signals are transmitted through Bluetooth or a gateway. The distance between the positioning card and the locator is calculated based on the signal transmission strength. The coordinates of the current positioning card can be determined through three locators.

[0063] In an optional embodiment, a twin system video mapping method includes: responding to a user's rotation instruction, and determining, according to a preset coordinate conversion formula, to map the video to be mapped to an image after rotation of the target mapping area.

[0064] The rotation instruction may be a rotation instruction controlled by a user through a mouse. The preset coordinate conversion formula may be a predefined coordinate conversion formula for rotating the screen in the twin system according to the rotation instruction. According to the preset coordinate conversion formula, it may be determined to map the video to be mapped to the image after rotation of the target mapping area.

[0065] The user uses the mouse to convert the mouse movement on the two-dimensional screen into the rotation and translation of the camera perspective in the three-dimensional scene. With the help of predefined coordinate conversion formulas, a flexible and intuitive global perspective browsing experience is achieved.

[0066] The technical solution of this embodiment is to obtain the target pixel value in each frame image of the video to be mapped; determine the light source parameters of each target pixel point according to the target pixel value and the corresponding light source texture, and project the monitored image to the scene through the light source from the monitoring position and angle considering the light source characteristics. Through illumination simulation and pixel-by-pixel projection of the three primary colors, it is possible to reproduce light and shadow effects and color details that are very close to the real world on the physical surface, thereby enhancing the realism and immersion of the video twin scene; determine the corresponding light source parameters according to the texture coordinates of each illuminated surface in the twin system to map the frame image to the twin system, and make precise projection visual effect adjustments according to the shape and material properties of the physical surface, so that both flat surfaces and complex curved surfaces can be well fitted, enriching the applicable scenarios.

[0067] Embodiment 3

[0068] Figure 3 The figure shows a schematic diagram of the structure of a twin system video mapping device provided in the third embodiment of the present application. This embodiment is applicable to the case of mapping a video to a three-dimensional space in a twin system. The specific structure of the twin system video mapping device is as follows:

[0069] The video parameter acquisition module 310 is used to acquire the video parameters of the device to be mapped;

[0070] A twin mapping parameter determination module 320 is used to determine the twin mapping parameters corresponding to the device to be mapped in the three-dimensional simulation scene of the twin system according to the video parameters;

[0071] The video mapping module 330 is used to map the video to be mapped into the twin system according to the video to be mapped collected by the device to be mapped, the corresponding light source texture and the twin mapping parameters.

[0072] The technical solution of this embodiment is to obtain the video parameters of the device to be mapped; determine the twin mapping parameters corresponding to the device to be mapped in the three-dimensional simulation scene of the twin system according to the video parameters; map the video to be mapped into the twin system according to the video to be mapped collected by the device to be mapped, the corresponding light source texture and the twin mapping parameters, and imitate the effects produced by various light sources in the real scene through the light source texture, and project the video to be mapped received by the monitoring from the monitoring position and angle to the twin system through the light source, allowing accurate adjustment of the projection visual effect according to the shape and material properties of the solid surface in the real scene, so that both flat surfaces and complex curved surfaces can be well fitted, enriching the applicable scenarios and improving the realism of the video mapping into the twin system. Therefore, the technical solution of this application solves the problem that the available scenarios are limited and the video mapping into the twin system lacks realism when processing the video stream through OpenCV, and achieves the effect of enriching the applicable scenarios and improving the realism of the video mapping into the twin system.

[0073] Optionally, the twin mapping parameter determination module 320 includes:

[0074] A unit is used for obtaining a unit, which is used for obtaining each unit in each frame image in the video to be mapped, for;

[0075] A light source parameter determination unit, used to determine the light source parameter of each target pixel point according to each target pixel value and the corresponding light source texture;

[0076] The image mapping unit is used to determine the corresponding light source parameters according to the texture coordinates of each light-receiving surface in the twin system to map the frame image into the twin system.

[0077] Optionally, the light source parameter determination unit includes:

[0078] The light source texture analysis subunit is used to determine the light transmission channel map and the three primary color light source according to the light source texture corresponding to each target pixel value;

[0079] The light source parameter determination subunit is used to determine the light source parameters of each target pixel point according to the light transmission channel map and the three primary color light source.

[0080] Optional, image mapping unit, including:

[0081] A texture coordinate determination subunit, used to determine the texture coordinates corresponding to each light-receiving surface piece according to the direction vector of each light-receiving surface piece relative to the light source in the twin system;

[0082] The light source parameter sampling subunit is used to determine the light source parameter corresponding to each texture coordinate.

[0083] Optionally, the twin system video mapping device further includes:

[0084] The monitoring data mapping module is used to obtain the monitoring data of the target object in the video to be mapped, and map the monitoring data to the corresponding twin entity of the twin system.

[0085] Optionally, the twin system video mapping device further includes:

[0086] The image rotation module is used to respond to the user's rotation instruction and determine to map the video to be mapped to the image after rotation in the target mapping area according to a preset coordinate conversion formula.

[0087] The twin system video mapping device provided in the embodiments of the present application can execute the twin system video mapping method provided in any embodiment of the present application, and has the corresponding functional modules and beneficial effects for executing the twin system video mapping method.

[0088] According to an embodiment of the present invention, the present invention also provides an electronic device, a readable storage medium and a computer program product.

[0089] Embodiment 4

[0090] Figure 4 A schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present application is shown in FIG. Figure 4 As shown, the electronic device includes a processor 410, a memory 420, an input device 430, and an output device 440; the number of the processor 410 in the electronic device can be one or more. Figure 4 A processor 410 is taken as an example; the processor 410, the memory 420, the input device 430 and the output device 440 in the electronic device can be connected via a bus or other means. Figure 4 The example of connecting through bus is taken in the following.

[0091] The memory 420, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the twin system video mapping method in the embodiment of the present application (for example, video parameter acquisition module 310, twin mapping parameter determination module 320 video mapping module 330). The processor 410 executes various functional applications and data processing of the electronic device by running the software programs, instructions and modules stored in the memory 420, that is, realizing the above-mentioned twin system video mapping method.

[0092] The memory 420 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 420 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 420 may further include a memory remotely arranged relative to the processor 410, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0093] The input device 430 may be used to receive input character information and generate key signal input related to user settings and function control of the electronic device. The output device 440 may include a display device such as a display screen.

[0094] Embodiment 5

[0095] Embodiment 5 of the present application also provides a storage medium containing computer executable instructions, which, when executed by a computer processor, are used to execute a twin system video mapping method, the method comprising: obtaining video parameters of a device to be mapped; determining, based on the video parameters, the twin mapping parameters corresponding to the device to be mapped in the three-dimensional simulation scene of the twin system; and mapping the video to be mapped into the twin system based on the video to be mapped collected by the device to be mapped, the corresponding light source texture and the twin mapping parameters.

[0096] Of course, the storage medium containing computer executable instructions provided in an embodiment of the present application, whose computer executable instructions are not limited to the method operations described above, can also execute related operations in the twin system video mapping method provided in any embodiment of the present application.

[0097] Through the above description of the implementation method, the technicians in the relevant field can clearly understand that the present application can be implemented with the help of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application can be essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for an electronic device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0098] It is worth noting that in the embodiment of the above-mentioned twin system video mapping device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application.

[0099] Note that the above are only preferred embodiments of the present application and the technical principles used. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A twin system video mapping method, characterized in that: include: Get the video parameters of the device to be mapped; Determine, according to the video parameters, the twin mapping parameters corresponding to the device to be mapped in the three-dimensional simulation scene of the twin system; Mapping the video to be mapped into the twin system according to the video to be mapped collected by the device to be mapped, the corresponding light source texture and the twin mapping parameters; The step of mapping the video to be mapped into the twin system according to the video to be mapped collected by the device to be mapped, the corresponding light source texture and the twin mapping parameter comprises: For each frame image in the video to be mapped, obtaining each target pixel value in the frame image; Determine the light source parameters of each target pixel point according to each target pixel value and the corresponding light source texture; wherein the light source parameters include transparency texture and color texture; Determine corresponding light source parameters according to the texture coordinates of each light-receiving surface in the twin system to map the frame image into the twin system; The step of determining corresponding light source parameters according to the texture coordinates of each light-receiving surface in the twin system includes: According to the direction vector of each light-receiving surface piece relative to the light source in the twin system, the direction vector is normalized to determine the texture coordinates corresponding to each light-receiving surface piece; Determine the light source parameters corresponding to each of the texture coordinates.

2. The method according to claim 1, characterized in that Determining the light source parameter of each target pixel point according to each target pixel value and the corresponding light source texture includes: Determine a light transmission channel map and a three-primary color light source according to the light source texture corresponding to each target pixel value; According to the light transmission channel map and the three primary color light source, the light source parameters of each target pixel point are determined.

3. The method according to claim 1, characterized in that The method further comprises: Acquire monitoring data of the target object in the video to be mapped, and map the monitoring data to the corresponding twin entity of the twin system.

4. The method according to claim 1, characterized in that: The method comprises: In response to a rotation instruction from a user, a determination is made to map the video to be mapped to an image after rotation of the target mapping area according to a preset coordinate conversion formula.

5. A twin system video mapping device, characterized in that: include: A video parameter acquisition module is used to obtain video parameters of the device to be mapped; A twin mapping parameter determination module, used to determine the twin mapping parameters corresponding to the device to be mapped in the three-dimensional simulation scene of the twin system according to the video parameters; A video mapping module, used to map the video to be mapped into the twin system according to the video to be mapped collected by the device to be mapped, the corresponding light source texture and the twin mapping parameters; Wherein, the twin mapping parameter determination module includes: An acquisition unit, used for acquiring each target pixel value in each frame image in the video to be mapped; A light source parameter determination unit, used to determine the light source parameters of each target pixel point according to each target pixel value and the corresponding light source texture; wherein the light source parameters include transparency texture and color texture; An image mapping unit, used to determine corresponding light source parameters according to the texture coordinates of each light-receiving surface in the twin system, so as to map the frame image into the twin system; The image mapping unit comprises: A texture coordinate determination subunit, used for normalizing the direction vector of each light-receiving surface piece relative to the light source in the twin system, and determining the texture coordinates corresponding to each light-receiving surface piece; The light source parameter sampling subunit is used to determine the light source parameter corresponding to each of the texture coordinates.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it implements the twin system video mapping method as described in any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the twin system video mapping method as described in any one of claims 1-4 is implemented.

8. A computer program product, characterized in that The computer program product comprises a computer program which, when executed by a processor, implements the twin system video mapping method according to any one of claims 1 to 4.

Images