A data transmission method and system for virtual scene generation

Abstract

The application provides a data transmission method and system for virtual scene generation, comprising the following steps: two data modes are used to realize data request and real-time push of a client and a back end in parallel; scene data information is requested, and the scene data information is stored into a program instance; a virtual entity corresponding to an actual business device is generated, and the virtual device is updated in real time; the state of each virtual entity is updated according to the scene data information; a transparent browser container is created to display a web page; a UI control module completes UI interaction and display through event driving according to user operation. The application has the beneficial effects that the data transmission speed is accelerated, the response speed and real-time performance of the system are improved, different systems and data formats are compatible, data interaction and integration are facilitated, received data is quickly parsed and processed, and the data processing capacity and efficiency of the digital twin system are improved.

Description

Technical Field

[0001] This invention belongs to the field of data transmission, and in particular relates to a data transmission method and system for virtual scene generation. Background Technology

[0002] In mining operations, PLC controllers are often used to control operational equipment, supplemented by video equipment for monitoring. This allows staff to monitor, control, and manage the status of operational equipment and schedule work. However, video equipment often cannot fully display the status of operational equipment during operation and is affected by the shooting angle, failing to fully represent the current working environment. Therefore, virtual scene generation technology is needed to assist in this work.

[0003] The process of generating virtual scenes requires acquiring various data from the real world, including sensor data, image data, audio data, and so on. The collection and transmission of this data is one of the key technologies for generating virtual scenes. During the process of generating virtual scenes, it is necessary to interact with various sensors, devices, and applications to acquire the data they generate. These data are then transmitted to the cloud or local server for processing via Modbus TCP protocol, OPC protocol, and MQTT forwarding service. The system needs to efficiently store and manage large amounts of data, including real-time data and historical data.

[0004] In existing technologies, the speed and quality of data transmission during the virtual scene generation process often cannot be guaranteed, resulting in serious lag and errors in the generated scene. At the same time, the systems of different business devices are not completely the same, which makes it impossible for existing technologies to respond quickly when receiving data from different devices, resulting in a significant reduction in the ability to parse and process data. Summary of the Invention

[0005] In view of this, the present invention aims to provide a data transmission method and system for virtual scene generation, in order to solve at least one of the above-mentioned technical problems.

[0006] To achieve the above objectives, the technical solution of the present invention is implemented as follows:

[0007] In a first aspect, the present invention provides a data transmission method for virtual scene generation, comprising the following steps:

[0008] S1. Two data modes are used in parallel to realize data requests and real-time push between the client and the backend;

[0009] S2. Request scene data information and store the scene data information in the program instance;

[0010] S3. Generate virtual entities corresponding to actual business devices and update the virtual devices in real time;

[0011] S4. Update the status of each virtual entity based on scene data information;

[0012] S5. Create a transparent browser container to display the web page;

[0013] S6. UI interaction and display are completed through event-driven processes based on user actions.

[0014] Furthermore, the two data modes in step S1 are the request data mode and the real-time data mode, wherein:

[0015] Data request mode: The client actively sends a request to the backend to obtain the required data, using the HTTP protocol, and provides it to the client through an API interface;

[0016] Real-time data mode: Establish a two-way communication channel through the WebSocket protocol to continuously push lightweight real-time data.

[0017] Furthermore, in the real-time data mode, the continuous push of lightweight real-time data is divided into two channels:

[0018] Attitude-driven data: The attitude information of the business devices is transmitted to Unreal Engine in real time using JSON packets.

[0019] Web page control instructions: A single-pass stack data structure is used to describe the instruction code and execution data value. When the web page needs to send control instructions to the backend, the data is packaged into JSON format and sent to the client via the WebSocket protocol.

[0020] Furthermore, based on the scene data in the program instance in step S2, a corresponding virtual entity is generated, and the posture attributes and display effects of the virtual entity are initialized. At the same time, corresponding action programs and timing functions are added to the virtual entity according to the actual motion state of the business device.

[0021] In the timing function of the virtual entity, the position and ID number of the virtual entity's motion control point are pushed to the custom event and structure array of the player controller;

[0022] In the timing function of the player controller, the system reads and writes data repeatedly to determine whether the structure array corresponding to each virtual entity has changed, and updates the virtual entities according to the corresponding changes.

[0023] Furthermore, after the custom event starts running, it obtains the player controller instance and creates a data connection based on the WebSocket protocol;

[0024] The data connection established via the WebSocket protocol awaits the backend data push module to push scene data, and the scene is parsed from string format and converted into JSON structure format;

[0025] Scene data in JSON structure format is passed to the player controller, which then controls and adjusts the posture and actions of the virtual entities in real time based on the scene data.

[0026] Furthermore, the process of acquiring service device data and sending it to each terminal in steps S1-S6 is as follows:

[0027] A1. Use the Modbus protocol plugin to receive raw data from external business devices;

[0028] A2. By utilizing the characteristics of reflection and structures, the original data is parsed and converted into data in a fixed format;

[0029] A3. Use the MQTT server to send data in a fixed format to the backend of the data transmission system.

[0030] Furthermore, the specific execution process of step A1 is as follows:

[0031] A101. Obtain the service device parameters and Modbus protocol configuration information, and obtain the corresponding Modbus protocol address and quantity mapping based on the service device parameters.

[0032] A102. Loop through each address in the Modbus protocol;

[0033] A103. Concatenate the original values ​​of the service device parameters read from each address into byte slices, and parse the byte slices to obtain the slice parameter values ​​according to the Modbus protocol configuration by calling the parsing function.

[0034] A104. Based on the slice parameter value type, select the method for appending the slice parameter value to obtain the complete parameter value corresponding to the original value;

[0035] A105. After looping through all addresses, return all resolved complete parameter values.

[0036] Furthermore, the specific execution process of step A2 is as follows:

[0037] A201. Determine if a field of the business device parameter is greater than 1. If it is, parse the byte slice in a loop; otherwise, exit directly.

[0038] A202. Each time, take out a sub-slice of length 2 bytes, parse the byte slice into an integer of uint16 form and add it to the uint16 integer array;

[0039] A203. Define string variables to record digital and analog quantity types;

[0040] A204. Loop through the nodes of each business device. If the Aplit field of a node is empty, it does not need to be split again. Encapsulate the return value and add the encapsulated return value to the result array.

[0041] A205. If the Aplit field is not empty, it needs to be split again. Convert the integer array to be split into a bit array, and then split the bit array and process each bit in a loop.

[0042] Furthermore, the specific process of step A3 is as follows:

[0043] A301. Check if the connection was established successfully;

[0044] A302. If the connection is successfully established, then enter a loop.

[0045] A303, Monitor the stop signal channel and message receiving channel within the loop body;

[0046] A304. If a stop signal is detected, exit the loop directly; if a message is detected, determine whether the message needs to be sent.

[0047] A305. If sending is required, serialize the message into a JAON string and check for serialization errors; if sending is not required, return to step A303.

[0048] A306. If an error occurs, return to step A303; if no error occurs, publish the JAON string and return to step A303.

[0049] In a second aspect, the present invention provides a data transmission system for virtual scene generation, comprising:

[0050] The backend data push module uses two data modes to implement data requests and real-time push between the client and the backend in parallel.

[0051] The scene data request module requests scene data information from the backend data push module and stores the scene data information in the program instance;

[0052] The scene generation / update module generates virtual entities corresponding to actual business devices and updates the virtual devices in real time.

[0053] The data-driven module updates the status of each virtual entity based on scene data information;

[0054] For the front-end page, create a transparent browser container to display the web page;

[0055] The UI control module uses event-driven methods to complete UI interactions and displays based on user actions.

[0056] Compared with existing technologies, the data transmission method and system for virtual scene generation described in this invention have the following advantages:

[0057] The present invention provides a data transmission method and system for virtual scene generation, which accelerates data transmission speed, improves system response speed and real-time performance, is compatible with different systems and data formats, facilitates data interaction and integration, and enables rapid parsing and processing of received data, thereby improving the data processing capabilities and efficiency of the digital twin system. Attached Figure Description

[0058] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0059] Figure 1 This is a flowchart illustrating a data transmission method for virtual scene generation according to an embodiment of the present invention. Detailed Implementation

[0060] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0061] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0062] Example 1:

[0063] A data pass-through method for virtual scene generation includes the following steps:

[0064] S1. Two data modes are used in parallel to realize data requests and real-time push between the client and the backend;

[0065] S2. Request scene data information and store the scene data information in the program instance;

[0066] S3. Generate virtual entities corresponding to actual business devices and update the virtual devices in real time;

[0067] S4. Update the status of each virtual entity based on scene data information;

[0068] S5. Create a transparent browser container to display the web page;

[0069] S6. UI interaction and display are completed through event-driven processes based on user actions.

[0070] The two data modes in step S1 are the request data mode and the real-time data mode, wherein:

[0071] Data request mode: The client actively sends a request to the backend to obtain the required data, using the HTTP protocol, and provides it to the client through an API interface;

[0072] Real-time data mode: Establish a two-way communication channel through the WebSocket protocol to continuously push lightweight real-time data.

[0073] In real-time data mode, the continuous push of lightweight real-time data is carried out through two channels:

[0074] Attitude-driven data: The attitude information of the business devices is transmitted to Unreal Engine in real time using JSON packets.

[0075] Web page control instructions: A single-pass stack data structure is used to describe the instruction code and execution data value. When the web page needs to send control instructions to the backend, the data is packaged into JSON format and sent to the client via the WebSocket protocol.

[0076] Based on the scene data in the program instance in step S2, generate the corresponding virtual entity, initialize the posture attributes and display effects of the virtual entity, and add corresponding action programs and timing functions to the virtual entity according to the actual motion state of the business device.

[0077] In the timing function of the virtual entity, the position and ID number of the virtual entity's motion control point are pushed to the custom event and structure array of the player controller;

[0078] In the timing function of the player controller, the system reads and writes data repeatedly to determine whether the structure array corresponding to each virtual entity has changed, and updates the virtual entities according to the corresponding changes.

[0079] After the custom event starts running, it retrieves the player controller instance and creates a data connection based on the WebSocket protocol;

[0080] The data connection established via the WebSocket protocol awaits the backend data push module to push scene data, and the scene is parsed from string format and converted into JSON structure format;

[0081] Scene data in JSON structure format is passed to the player controller, which then controls and adjusts the posture and actions of the virtual entities in real time based on the scene data.

[0082] Create custom events for different UI operations and execute each UI operation sequentially from top to bottom using a sequence logic approach;

[0083] Use pseudo-class selectors to distinguish different UI operation types and perform corresponding functional logic processing based on the operation type.

[0084] The process of acquiring service device data and sending it to each terminal in steps S1-S6 is as follows:

[0085] A1. Use the Modbus protocol plugin in the Go language to receive raw data from external business devices;

[0086] A2. By utilizing the characteristics of reflection and structures, the original data is parsed and converted into data in a fixed format;

[0087] A3. Use the MQTT server to send data in a fixed format to the backend of the data transmission system.

[0088] The specific execution process of step A1 is as follows:

[0089] A101. Obtain the service device parameters and Modbus protocol configuration information, and based on the service device parameters, obtain the corresponding Modbus protocol address and quantity mapping.

[0090] A102. Loop through each address in the Modbus protocol, specifically:

[0091] A10201. Loop through the address parameters in the configuration parameters of each service device and process each address parameter as follows:

[0092] Obtain the Modbus address and quantity mapping information of the address parameters, calculate the actual Modbus address value to be read, and subtract the offset value from the mapped address based on the configured address offset to calculate the actual address value in the device.

[0093] A10202: Call the GetReAultA function to read the calculated actual address value from the device. The GetReAultA function implements the specific Modbus communication read operation. After reading the value, it checks whether there is a connection error. If so, it returns an error. It also checks whether there is a read error. If so, it logs the error and returns an error.

[0094] A10203. The original value results read from each address are concatenated and stored using byte slices.

[0095] A103. Concatenate the original values ​​of the service device parameters read from each address into byte slices, and parse the byte slices to obtain the slice parameter values ​​according to the Modbus protocol configuration by calling the parsing function.

[0096] A104. Based on the slice parameter value type, select the append method for the slice parameter value to obtain the complete parameter value corresponding to the original value, specifically:

[0097] After parsing the data read by Modbus, it may return a single value or an array, so it is necessary to use type assertion to determine the type of the parsed value.

[0098] If value is an array type ([]interface{}), then use nValue,ok := value.([]interface{}) to perform type conversion, and then directly append the nValue array to the result valueA array;

[0099] If value is not an array type, the elAe block is used to append the individual value to the end of the resulting valueA array.

[0100] A105. After looping through all addresses, return all resolved complete parameter values.

[0101] The specific execution process of step A2 is as follows:

[0102] A201. Determine if a field of the business device parameter is greater than 1. If it is, parse the byte slice in a loop; otherwise, exit directly.

[0103] A202. Each time a byte subslice r of length 2 is taken out, BytesToUInt16 is called to parse the byte slice into an integer i2 in uint16 form and add it to the uint16 integer array is.

[0104] A203. Define a string variable to record the switch and analog quantity types. Specifically, declare a global variable of type string, SwitchAnalogTypeVal.

[0105] A204. Loop through the nodes of each service device. If the Aplit field of a node is empty, it does not need to be split again. Encapsulate the return value and add the encapsulated return value to the result array. Specifically:

[0106] A20401. Create a return value map variable value, set the value["name"] field to parameter name + node name, format the corresponding value in the is array to string msg, set the value["val"] field to msg, and if the node rule is "SwitchAnalogType", record the type identifier;

[0107] A20402. If the node rule is not empty, it means that a dictionary query is required.

[0108] A20403. If the rule is "SwitchAnalogValue", get the dictionary from the configuration, get the sub-dictionary according to the type, and if the sub-dictionary is not empty, get the value from the sub-dictionary.

[0109] Otherwise, obtain a dictionary of strings, directly retrieve the value from the dictionary, and set it as the desc field;

[0110] A20404. Add the encapsulated value to the returned array.

[0111] A205. If the Aplit field is not empty, it needs to be split again. Convert the integer array to be split into a bit array, and then split the bit array and process each bit in a loop. Specifically:

[0112] A20501. Split the Split rule string of the parsed node according to "-", and save the result to the nn string slice;

[0113] Convert the first element (string) after splitting to an int type and assign the result to the ns variable;

[0114] Convert the second element (string) after splitting to int type and assign the result to the variable ne;

[0115] The main purpose is to extract the start and end values ​​from the rule string and convert them into numeric types for subsequent calculations; where ns and ne values ​​represent the start number and end number of the split range, respectively.

[0116] A20502. Declare n as an int type variable, determine the relationship between the start value ns and the end value ne. If ns > ne, it means the range is from large to small. Calculate n as the difference between the start value and the end value plus 1.

[0117] Otherwise, the range of identifiers is from smallest to largest, and n is calculated as the difference between the end value and the start value plus 1;

[0118] The goal is to calculate the total number of bits n to be split based on the start and end values ​​of the splitting rules. The value of n will be used later to determine how to split the bit array.

[0119] A20503. Convert the integer values ​​to be split (taken from the is array) into byte slices. The IntToBytes function converts integers into byte slices in a specified base (binary in this case), and then converts the byte slices into a bit array.

[0120] A20504. Based on the previously calculated number of segments n to be split, call the splitBitArray function to split the original bit array into multiple arrays according to n and save them to bitArray.

[0121] A20505, Set a bn as a counter for the loop;

[0122] A20506. Determine the relationship between the start value ns and the end value ne. If ns > ne, it means the range is from large to small; otherwise, it is from small to large.

[0123] Use different for loops depending on the relationship. If ns > ne, use for i := ns; i >= ne; i-- loop from larger to smaller. Otherwise, use for i := ns; i <= ne; i++ loop from smaller to larger.

[0124] Within each loop, the GetBitInfo function is called to process each value i, the return value is added to the values ​​array, and the bn counter is incremented by 1.

[0125] The specific process of step A3 is as follows:

[0126] A301. Check if the connection was established successfully;

[0127] A302. If the connection is successfully established, then enter a loop.

[0128] A303, Monitor the stop signal channel and message receiving channel within the loop body;

[0129] A304. If a stop signal is detected, exit the loop directly; if a message is detected, determine whether the message needs to be sent.

[0130] A305. If sending is required, serialize the message into a JAON string and check for serialization errors; if sending is not required, return to step A303.

[0131] A306. If an error occurs, return to step A303; if no error occurs, publish the JAON string and return to step A303.

[0132] A data transmission system for virtual scene generation, comprising:

[0133] The backend data push module uses two data modes to implement data requests and real-time push between the client and the backend in parallel.

[0134] The scene data request module requests scene data information from the backend data push module and stores the scene data information in the program instance;

[0135] The scene generation / update module generates virtual entities corresponding to actual business devices and updates the virtual devices in real time.

[0136] The data-driven module updates the status of each virtual entity based on scene data information;

[0137] For the front-end page, create a transparent browser container to display the web page;

[0138] The UI control module uses event-driven methods to complete UI interactions and displays based on user actions.

[0139] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0140] In the several embodiments provided in this application, it should be understood that the disclosed methods and systems can be implemented in other ways. For example, the division of units described above is merely a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. The aforementioned units may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention according to actual needs.

[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

[0142] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A data transmission method for virtual scene generation, characterized in that, Includes the following steps: S1. Two data modes are used in parallel to realize data requests and real-time push between the client and the backend; S2. Request scene data information and store the scene data information in the program instance; S3. Generate virtual entities corresponding to actual business devices and update the virtual devices in real time; S4. Update the status of each virtual entity based on scene data information; S5. Create a transparent browser container to display the web page; S6. Complete UI interaction and display through event-driven processes based on user actions; Among them, the two data modes in step S1 are request data mode and real-time data mode; Based on the scene data in the program instance in step S2, generate the corresponding virtual entity, initialize the posture attributes and display effects of the virtual entity, and add corresponding action programs and timing functions to the virtual entity according to the actual motion state of the business device. In the timing function of the virtual entity, the position and ID number of the virtual entity's motion control point are pushed to the custom event and structure array of the player controller; In the timing function of the player controller, the system reads, writes, and checks whether the structure array corresponding to each virtual entity has changed, and updates the virtual entities according to the corresponding changes. After the custom event starts running, it retrieves the player controller instance and creates a data connection based on the WebSocket protocol; The data connection established via the WebSocket protocol awaits the backend data push module to push scene data, and the scene is parsed from string format and converted into JSON structure format; Scene data in JSON structure format is passed to the player controller, which then controls and adjusts the posture and actions of the virtual entities in real time based on the scene data.

2. The data transmission method for virtual scene generation according to claim 1, characterized in that: Data request mode: The client actively sends a request to the backend to obtain the required data, using the HTTP protocol, and provides it to the client through an API interface; Real-time data mode: Establish a two-way communication channel through the WebSocket protocol to continuously push lightweight real-time data.

3. The data transmission method for virtual scene generation according to claim 2, characterized in that: In the real-time data mode, the continuous push of lightweight real-time data is carried out through two channels: attitude-driven data: the attitude information of the business device is transmitted to Unreal Engine in real time using JSON packets. Web page control instructions: A single-pass stack data structure is used to describe the instruction code and execution data value. When the web page needs to send control instructions to the backend, the data is packaged into JSON format and sent to the client via the WebSocket protocol.

4. The data transmission method for virtual scene generation according to claim 1, characterized in that: The process of acquiring service device data and sending it to each terminal in steps S1-S6 is as follows: A1. Use the Modbus protocol plugin to receive raw data from external business devices; A2. By utilizing the characteristics of reflection and structures, the original data is parsed and converted into data in a fixed format; A3. Use the MQTT server to send data in a fixed format to the backend of the data transmission system.

5. The data transmission method for virtual scene generation according to claim 4, characterized in that: The specific execution process of step A1 is as follows: A101. Obtain the service device parameters and Modbus protocol configuration information, and obtain the corresponding Modbus protocol address and quantity mapping based on the service device parameters. A102. Loop through each address in the Modbus protocol; A103. Concatenate the original values ​​of the service device parameters read from each address into byte slices, and parse the byte slices to obtain the slice parameter values ​​according to the Modbus protocol configuration by calling the parsing function. A104. Based on the slice parameter value type, select the method for appending the slice parameter value to obtain the complete parameter value corresponding to the original value; A105. After looping through all addresses, return all resolved complete parameter values.

6. The data transmission method for virtual scene generation according to claim 5, characterized in that: The specific execution process of step A2 is as follows: A201. Determine if a field of the business device parameter is greater than 1. If it is, parse the byte slice in a loop; otherwise, exit directly. A202. Each time, take out a sub-slice of length 2 bytes, parse the byte slice into an integer of uint16 form and add it to the uint16 integer array; A203. Define string variables to record digital and analog quantity types; A204. Loop through the nodes of each business device. If the Aplit field of a node is empty, it does not need to be split again. Encapsulate the return value and add the encapsulated return value to the result array. A205. If the Aplit field is not empty, it needs to be split again. Convert the integer array to be split into a bit array, and then split the bit array and process each bit in a loop.

7. The data transmission method for virtual scene generation according to claim 4, characterized in that: The specific process of step A3 is as follows: A301. Check if the connection was established successfully; A302. If the connection is successfully established, then enter a loop. A303, Monitor the stop signal channel and message receiving channel within the loop body; A304. If a stop signal is detected, exit the loop immediately. If a message is detected, determine whether the message needs to be sent. A305. If sending is required, serialize the message into a JAON string and check for serialization errors; if sending is not required, return to step A303. A306. If an error occurs, return to step A303; if no error occurs, publish the JAON string and return to step A303.

8. A data transmission system for virtual scene generation, based on the method described in any one of claims 1-7, characterized in that, include: The backend data push module uses two data modes to implement data requests and real-time push between the client and the backend in parallel. The scene data request module requests scene data information from the backend data push module and stores the scene data information in the program instance; The scene generation / update module generates virtual entities corresponding to actual business devices and updates the virtual devices in real time. The data-driven module updates the status of each virtual entity based on scene data information; For the front-end page, create a transparent browser container to display the web page; The UI control module uses event-driven methods to complete UI interactions and displays based on user actions.

Images