Interaction method suitable for digital twin networking

Abstract

The invention relates to an interaction method suitable for digital twin networking. The interaction method comprises the following steps of projecting a picture acquired by a camera to a screen; carrying out digital twinning matching through a digital twinning management system, generating a UI (User Interface), and segmenting regions; performing function endowing; a traditional management system is connected, and point position projection is carried out; function triggering in the virtual scene is fed back to the screen. Therefore, region segmentation can be performed on the screen through the UI interface, a virtual scene can be segmented into a plurality of regions in a traditional management system and a digital twinning project, and grouping correspondence is realized. Data of each area after scene segmentation can be managed independently, and matching of effective data in the areas is met. According to the invention, the trigger position of the emergency can be quickly identified, and a user can quickly associate the area where the emergency occurs. Therefore, the characteristics of the eagle-eye camera can be utilized to the maximum extent, and the aerial view panoramic state can be obtained.

Description

Technical Field

[0001] The present invention relates to an interaction method related to a digital twin project and a traditional management system, and in particular to an interaction method suitable for digital twin networking. Background Art

[0002] As for the interaction between existing digital twin projects and traditional management systems, it is mainly achieved by the intercommunication of backend data. For the virtual scenes in digital twins, the effects within them can only be expressed through backend data. During the implementation period, the interaction between traditional management systems and real scenes needs to rely on cameras. To this end, in order to meet the necessary collection, more cameras need to be distributed and set up, and the processing of scene connection edges needs to be involved, which increases the data processing burden. In addition, traditional digital twin scenes (virtual scenes) can only use modeling scenes as the base, and cameras in most cases only have the function of video pop-up boxes, and the degree of interaction is very low.

[0003] In view of the above-mentioned defects, the designers actively carried out research and innovation in order to create an interactive method suitable for digital twin networking, making it more valuable for industrial use. Summary of the invention

[0004] In order to solve the above technical problems, the purpose of the present invention is to provide an interactive method suitable for digital twin networking.

[0005] The interactive method applicable to digital twin networking of the present invention comprises the following steps: Step 1: Project the image captured by the camera onto the screen; Step 2: Use the digital twin management system to match the digital twins, generate the UI interface, and divide the area; Step three, assign functions; Step 4: Connect to the traditional management system and perform point projection; Step 5: The function in the virtual scene is triggered and fed back to the screen.

[0006] Furthermore, in the above-mentioned interactive method applicable to digital twin networking, in the step one, an Eagle Eye camera is selected to project the Eagle Eye camera image in a panoramic bird's-eye view state onto the screen through pixel stream.

[0007] Furthermore, in the above-mentioned interactive method applicable to digital twin networking, in the step 2, the real scene is aligned with the position and angle of the virtual scene on the screen to form a UI interface.

[0008] Furthermore, in the above-mentioned interactive method suitable for digital twin networking, in the step 2, in the UI interface, the screen is divided into several areas of the same size, each area represents a real scene, and the center point of each area is provided with longitude and latitude; the virtual scene is accompanied by the coordinates of the virtual scene constructed by the Unreal Engine, and the coordinates of the virtual scene are grouped and corresponded with the longitude and latitude of the center point of the area representing the real scene.

[0009] Furthermore, in the above-mentioned interactive method applicable to digital twin networking, in the step three, a click interaction function is given to the segmented areas in the UI interface.

[0010] Furthermore, in the above-mentioned interactive method suitable for digital twin networking, in the step four, after the function in the traditional management system is triggered, the corresponding background transmits the longitude and latitude, and locates the function trigger point in the virtual scene through the coordinates of the virtual scene corresponding to the longitude and latitude.

[0011] Furthermore, in the above-mentioned interactive method applicable to digital twin networking, in the step five, a function is triggered in the virtual scene, and its triggered position can be located in the real scene through the longitude and latitude corresponding to the screen coordinates.

[0012] By means of the above scheme, the present invention has at least the following advantages: 1. The screen can be divided into regions through the UI interface, and the virtual scene can be divided into several regions in traditional management systems and digital twin projects, and group correspondence can be achieved.

[0013] 2. After the scene is segmented, the data of each area can be managed separately to meet the matching of valid data in the area.

[0014] 3. It can quickly identify the triggering location of an emergency, allowing users to quickly associate the area where the emergency occurred.

[0015] 4. It can maximize the use of the characteristics of the Eagle Eye camera to achieve a bird's-eye view of the panoramic state.

[0016] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present invention in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of the implementation process of the interactive method suitable for digital twin networking. DETAILED DESCRIPTION

[0018] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0019] like Figure 1 The interactive method for digital twin networking is unique in that it includes the following steps: Step one, project the picture obtained by the camera onto the screen. During implementation, an Eagle Eye camera is used to transmit the Eagle Eye camera picture in a panoramic overlooking state to the digital twin project through the network through the pixel stream, and then the digital twin project puts the picture data into the UI interface to complete the process of transferring the Eagle Eye camera picture to the screen. In this way, a more comprehensive field of view can be obtained, allowing the screen to obtain the most comprehensive image data. During implementation, the edge distortion of the Eagle Eye picture does not need to be processed, because during subsequent operations, after the grid is divided, it is only necessary to process the virtual scene coordinates of the center point of the grid (in this application, the virtual scene is constructed by the UE engine, which can also be referred to as the UE scene) and the actual longitude and latitude records in reality. It can reduce the processing steps and increase the subsequent matching rate.

[0020] Step 2: Through the digital twin management system, digital twin matching is performed, a UI interface is generated, and the area is divided. Specifically, the real scene is aligned with the position and angle of the virtual scene on the screen to form a UI interface. The alignment method is to manually align the virtual scene divided on the screen through the digital twin management system, first observe the virtual scene divided on the screen, and then find the corresponding position in the real scene and record the longitude, latitude and angle. The digital twin management system adopted by the present invention is a digital twin derivative system under the current conventional construction, which will not be repeated here.

[0021] At the same time, in the UI interface adopted, the screen is divided into several areas of the same size according to the needs, each area represents a real scene, and the center point of each area is provided with longitude and latitude. It is recommended to store all longitude and latitude and angles in the back end. Each area divided by the screen is 16:9, because most horizontal screens are 16:9, so that adaptation can be guaranteed in horizontal screen mode. If it is necessary to apply it in a vertical screen port, it is necessary to re-divide the screen area and then realign the screen area and the modeling scene position. In addition, the virtual scene during the implementation of the present invention is accompanied by the coordinates of the virtual scene constructed by the Unreal Engine, and the coordinates include the XYZ axis data of the Unreal Engine. In this way, the coordinates of the virtual scene can be grouped and corresponded with the longitude and latitude of the center point of the area representing the real scene, completing the data intercommunication.

[0022] Step three, assign functions. A UI button can be added to the divided area in the UI interface, and any function you want to trigger can be bound to this UI button through dynamic notification. At the same time, data interoperability can be achieved, that is, the functions of the digital twin project and the functions of the traditional management system can achieve basic interoperability. During the implementation period, in order to improve the convenience of operation, interactive functions are adopted, including hiding the eagle eye screen, displaying the virtual scene, and jumping to the corresponding position of the clicked screen after clicking the UI. In this way, the daily operation and application needs of the digital twin after networking can be realized. At the same time, on the UI button, you can bind event notifications to trigger other functions that need to be triggered, which can be adjusted according to the actual implementation needs, and I will not go into details here.

[0023] Step 4: Connect to the traditional management system and perform point projection. During the implementation period, when a function in the traditional management system is triggered, the corresponding background can transmit the longitude and latitude. After that, the function trigger point is located in the virtual scene through the coordinates of the virtual scene corresponding to the longitude and latitude. For example, when a camera in the traditional management system detects a fire warning, it will generate warning data in the back end and send it to the digital twin management system through the network. Then the longitude and latitude in the data are converted into UE scene coordinates and UE screen coordinates through the algorithm. At this time, a flame special effect will be generated in the corresponding position of the virtual scene, and a UI icon will be generated in the corresponding position of the corresponding screen. These all belong to the presentation layer. After the presentation layer is executed, any set function can be triggered.

[0024] Step 5: The function in the virtual scene is triggered and fed back to the screen. Specifically, when a function is triggered in the virtual scene, the triggering position can be located in the real scene through the longitude and latitude corresponding to the screen coordinates. In this way, a deep connection in space can be achieved.

[0025] During the implementation, the scenes corresponding to each area in the virtual scene are divided into blocks for management, and a management-class Actor is placed in each divided scene. This Actor stores the data for interaction with the traditional management system in each area, and can realize functions such as camera aggregation and data statistics in each area.

[0026] The working principle of the present invention is as follows: The traditional management system discovers events such as warnings, generates data at the back end, and sends it to the digital twin management system through the network. The digital twin management system converts the longitude and latitude in the data into UE screen coordinates and UE scene coordinates, and then triggers functions at the corresponding positions of the screen and scene, such as adding icons to the screen and adding special effects to the scene. If the digital twin management system actively triggers an event, the event and the scene coordinates where the event occurs are converted into longitude and latitude, written into Json and sent to the back end of the traditional management system through the network.

[0027] For example, at a certain place in the real scene, the corresponding camera discovered a car accident. After the traditional management system learns of this, it generates data on the car accident event in the background. After the traditional management system applies this method, the data on the car accident event generated by it includes at least the camera number that obtained the car accident information and the longitude and latitude of the place where the car accident occurred. Afterwards, the traditional management system transmits the data on the car accident event to the digital twin management system through the network. Afterwards, the digital twin management system matches the corresponding camera in the virtual scene through the camera number, and replaces the camera icon with a car accident warning icon. Subsequently, the longitude and latitude of the place where the car accident occurred are converted into coordinates in the virtual scene (UE scene), and a car accident special effect is generated in the virtual scene.

[0028] It can be seen from the above textual description and the accompanying drawings that the present invention has the following advantages: 1. The screen can be divided into regions through the UI interface, and the virtual scene can be divided into several regions in traditional management systems and digital twin projects, and group correspondence can be achieved.

[0029] 2. After the scene is segmented, the data of each area can be managed separately to meet the matching of valid data in the area.

[0030] 3. It can quickly identify the triggering location of an emergency, allowing users to quickly associate the area where the emergency occurred.

[0031] 4. It can maximize the use of the characteristics of the Eagle Eye camera to achieve a bird's-eye view of the panoramic state.

[0032] In addition, the indicated orientations or positional relationships described in the present invention are all based on the orientations or positional relationships shown in the accompanying drawings and are only for the convenience of describing the present invention and simplifying the description. They do not indicate or imply that the device or structure referred to must have a specific orientation or be operated with a specific orientation structure. Therefore, they cannot be understood as limitations on the present invention.

[0033] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the technical principles of the present invention, and these improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. An interactive method suitable for digital twin networking, characterized by The following steps are involved: Step 1: Project the image captured by the camera onto the screen; Step 2: Use the digital twin management system to match the digital twins, generate the UI interface, and divide the area; Step three, assign functions; Step 4: Connect to the traditional management system and perform point projection; Step 5: The function in the virtual scene is triggered and fed back to the screen.

2. The interactive method suitable for digital twin networking according to claim 1, characterized in that: In the step 1, an Eagle Eye camera is selected, and the Eagle Eye camera image in a panoramic bird's-eye view state is projected onto the screen through a pixel stream.

3. The interactive method suitable for digital twin networking according to claim 1, characterized in that: In the step 2, the real scene is aligned with the position and angle of the virtual scene on the screen to form a UI interface. The alignment process is to convert the longitude and latitude of the real scene and the virtual scene, and map them to the coordinates of the UE scene.

4. The interactive method suitable for digital twin networking according to claim 3 is characterized in that: In the step 2, in the UI interface, the screen is divided into several areas of equal size, each area represents a real scene, and the center point of each area is provided with longitude and latitude; the virtual scene is accompanied by the coordinates of the virtual scene constructed by the Unreal Engine, and the coordinates of the virtual scene are grouped and corresponded with the longitude and latitude of the center point of the area representing the real scene.

5. The interactive method suitable for digital twin networking according to claim 1, characterized in that: In the step three, the segmented areas in the UI interface are given a click-interactive function, wherein after clicking the UI, the interactive function includes hiding the Hawkeye screen, displaying the virtual scene, and jumping to the corresponding position of the clicked screen.

6. The interactive method suitable for digital twin networking according to claim 1, characterized in that: In the step 4, after the function in the traditional management system is triggered, the corresponding background transmits the longitude and latitude, and locates the function trigger point in the virtual scene through the coordinates of the virtual scene corresponding to the longitude and latitude.

7. The interactive method suitable for digital twin networking according to claim 1, characterized in that: In step five, a function is triggered in the real scene, and the triggering position can be located in the virtual scene through the scene coordinates corresponding to the longitude and latitude.

Images