A method for constructing a metaverse venue with real-time virtual-real inter-scene mapping
By combining static and dynamic asset self-organization status detection with robot-based fixed-point scheduling and inspection, the high cost and complex engineering problems in the real-time virtual-real interaction scenario of the Metaverse venue have been solved, achieving low-cost, high-precision real-time asset status detection and full coverage.
Patent Information
- Application Number
- CN202311257856.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-27
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2043-09-27
AI Technical Summary
Existing technologies are costly, involve large-scale and complex engineering projects when constructing real-time virtual-real interactive scenes in metaverse venues, making it difficult to effectively manage the diverse, large-scale, and widely distributed assets of large venues.
By combining IoT, wireless communication, indoor positioning and robotics technologies, and employing a method of asset self-organizing dynamic and static combined status detection and robot fixed-point scheduling inspection, the asset status is acquired in real time and dynamically rendered in 3D within the metaverse venue.
It achieves low-cost, high-precision real-time asset status detection, covering the entire asset status, with good real-time detection performance, simple construction, and easy promotion.
Smart Images

Figure CN117496049B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method for constructing a real-time virtual-real interactive scene of a metaverse venue. Background Technology
[0002] In the wave of the digital age, metaverse venues, as a new phenomenon in the digitalization of venues, are bringing unprecedented high-quality experiences to people in a brand-new form. Real-time virtual-real inter-visualization is a typical application scenario of metaverse venues, which involves a two-way mapping between physical and virtual venues. This metaverse venue application scenario effectively solves the operational and management difficulties caused by the characteristics of exhibition venues, such as their diverse asset types, large scale, wide distribution, and complex business operations. Through this application scenario, the operational status of each asset in the physical venue can be displayed in real time within the metaverse venue, allowing operators to quickly understand the status of each asset, greatly improving the efficiency of asset use and maintenance, while also enabling the orderly and standardized management and application of assets, ensuring the safe and stable operation of the venue.
[0003] Currently, there is no good way to implement the construction of real-time virtual-real interaction scenes in the metaverse venue. Theoretically, it can be achieved through machine vision. However, due to the large area of the venue, the numerous and widely distributed assets involved, this method requires a large investment in cameras, image recognition computing power equipment, and network equipment, as well as the construction of a large number of recognition model algorithms. The construction cost is extremely high, the workload is large and extremely complex, and it has not been applied in engineering at present. Summary of the Invention
[0004] Purpose of the Invention: To adapt to the digital trend and better apply the metaverse to the venue industry, thereby assisting in the digital transformation and upgrading of the venue operation sector and solving the problem of managing multi-category, large-scale, and widely distributed assets in venues, this method proposes a real-time virtual-real inter-realistic scene construction method for metaverse venues by comprehensively applying technologies such as the Internet of Things, wireless communication, indoor positioning, and robotics. This method combines the asset self-organization dynamic and static combined state detection method with the robot fixed-point scheduling and inspection detection method, which can obtain the state information of densely distributed assets in real time and obtain the state information of isolated and moved assets in near real time. This information is then promptly transmitted to the metaverse venue for dynamic 3D rendering, thereby realizing the real-time virtual-real inter-realistic inter-realistic mapping of all assets in the metaverse venue. This supports venue operators in carrying out refined asset management and other application systems in realizing more advanced functions.
[0005] This invention specifically provides a method for constructing a real-time virtual-real interactive scene of a metaverse venue, including: parallel execution of an asset self-organizing dynamic and static combined state detection process and a robot fixed-point scheduling and inspection detection process;
[0006] The asset self-organizing dynamic and static combined state detection process is to use the asset state detector to self-organize and calculate the attitude and position of each asset based on the motion state of the asset in a cycle of T. At the same time, the robot inspection list is generated for the assets that cannot be detected, which guides the robot to perform the fixed-point scheduling inspection process. T is the asset state detection cycle.
[0007] The robot fixed-point scheduling inspection process is based on the robot inspection list. The fixed-point scheduling inspection robot inspects the assets in the list to complete the status inspection of the assets.
[0008] The asset self-organization dynamic and static combined status detection process specifically includes the following steps:
[0009] Step a1: Construct a digital scene;
[0010] Step a2, asset modeling and initialization;
[0011] Step a3: The asset status detector collects and processes data;
[0012] Step a4, calculate asset pose;
[0013] Step a5: Perform asset self-organization positioning;
[0014] Step a6: Update the robot patrol list;
[0015] Step a7: Update asset status.
[0016] Step a1 includes: performing 3D modeling of the physical scene and establishing a coordinate system c: selecting a point in the scene as the origin o, with the vertical upward direction as the positive z-axis, the east direction as the positive x-axis, and the north direction as the positive y-axis; installing a communication gateway in the scene, with the communication gateway connecting to each asset status detector in the south via LORA communication, and forwarding data to the Metaverse Venue System in the north via Ethernet communication or mobile communication network.
[0017] Step a2 includes:
[0018] Step a2-1, Asset Modeling:
[0019] Suppose there are N assets in the scene. Each asset is numbered, denoted by n, where n∈{1,2,3,……,N}. A model is created for each asset in the scene, and the model of the nth asset is denoted as M(n). A coordinate system w is established for the asset numbered n. zbx (n): The center of the asset is selected as the origin. n The forward direction of the asset is x. n The positive axis direction, perpendicular to the asset surface and outwards, is z. nThe positive direction of the axis, and the right front direction of the asset is y. n The positive axis direction; install asset attitude detectors at the center of each asset, with the center of the attitude detector aligned with the asset center; construct three attributes for the 3D model of each asset: position attribute, position confidence state attribute, and attitude attribute.
[0020] The value of the position attribute indicates the position of the 3D asset model in coordinate system c, including three coordinate values: x, y, and z. When rendering the 3D asset model, the position is rendered according to the value of the position attribute.
[0021] When constructing the 3D model, an indicator model is built. When the value of the location confidence state attribute is 1, the indicator model is highlighted; when it is 0, the indicator model is not highlighted.
[0022] The values of the attitude attributes represent the rotational state of the asset's 3D model in coordinate system c, including the heading angle, pitch angle, and roll angle. The heading angle represents the angle of rotation of the asset's 3D model around the z-axis of coordinate system c, the pitch angle represents the angle of rotation of the asset's 3D model around the x-axis of coordinate system c, and the roll angle represents the angle of rotation of the asset's 3D model around the y-axis of coordinate system c. Counterclockwise rotation is a positive value, and clockwise rotation is a negative value. When rendering the asset's 3D model, the attitude is rendered according to the values of the attitude attributes.
[0023] Step a2-2, Asset Status Initialization:
[0024] Asset status data includes location data, location confidence status data, and attitude data:
[0025] The location data represents the position coordinates of the asset's center point in coordinate system c, denoted by WZ. c (n)(k) represents, Where n is the asset number, k represents the time of the kth asset status detection cycle, and x(n)(k), y(n)(k), and z(n)(k) represent the x-axis, y-axis, and z-axis coordinates of the center point of the asset with number n in the coordinate system c at the time of the kth asset status detection cycle, respectively.
[0026] The location trust status data represents the trust status of asset location data. A value of 1 indicates that the asset location data is trustworthy, and a value of 0 indicates that the asset location data is untrustworthy. Let the location trust status of each asset be represented by STU. wz (n)(k) represents, where n is the asset number and k represents the time of the kth detection period;
[0027] The attitude data represents the asset's attitude in coordinate system c, denoted by ZT. c (n)(k) represents, Wherein, ψ(n)(k) is the heading angle of asset number n at time n of the k-th asset status detection cycle, that is, the angle of rotation of the asset around the z-axis of coordinate system c; θ(n)(k) is the pitch angle of asset number n at time n of the k-th asset status detection cycle, that is, the angle of rotation of the asset around the x-axis of coordinate system c; φ(n)(k) is the roll angle of asset number n at time n of the k-th asset status detection cycle, that is, the angle of rotation of the asset around the y-axis of coordinate system c.
[0028] Once the asset status detectors for each asset are installed, asset status detection begins, and this moment is designated as the 0th asset status detection cycle. The position data WZ of each asset relative to coordinate system c is then acquired through measurement. c (n)(0) and attitude data ZT c (n)(0), and the trusted status data of each asset's location at this moment, STU WZ The value of (n)(0) is set to 1, as shown below:
[0029]
[0030] Among them, WZ c (n)(0) represents the position data of asset number n in coordinate system c during the 0th asset status detection period;
[0031] x(n)(0) represents the x-axis coordinate value of the position data of asset number n in coordinate system c during the 0th asset status detection period, which is obtained by measurement;
[0032] y(n)(0) represents the y-axis coordinate of the position data of asset number n in coordinate system c during the 0th asset status detection period, which is obtained by measurement;
[0033] z(n)(0) represents the z-axis coordinate value of the position data of asset number n in coordinate system c during the 0th asset status detection period, which is obtained by measurement;
[0034] ZT c (n)(0) represents the attitude data of asset numbered n in coordinate system c during the 0th asset status detection cycle;
[0035] ψ(n)(0) represents the heading angle of the attitude data of asset numbered n in coordinate system c during the 0th asset status detection cycle, which is obtained by measurement;
[0036] θ(n)(0) represents the pitch angle of the attitude data of asset numbered n in coordinate system c during the 0th asset status detection cycle, which is obtained by measurement;
[0037] φ(n)(0) represents the roll angle of the attitude data of asset numbered n in coordinate system c during the 0th asset status detection cycle, which is obtained by measurement;
[0038] ZT c (n)(0) represents the location confidence status data of asset number n in the 0th asset status detection period, and its value is set to 1;
[0039] Steps a2-3: Start the equipment operation:
[0040] Enable the Bluetooth signal broadcast function of each asset status detector. The broadcast data includes the MAC address of the Bluetooth positioning module in the asset status detector. The MAC address, short for MediaAccessControlAddress, is the physical address or hardware address and is the unique identifier of the Bluetooth positioning module. Use the MAC address values of the Bluetooth positioning modules of the asset status detectors installed on each asset... n This indicates that n is the asset number;
[0041] Create a variable NUM for each asset. move (n) Records the cumulative number of asset connection movement cycles, NUM move (n) is initially set to 0.
[0042] Step a3 includes: cyclically collecting data detected by the six-axis inertial sensor in the asset status detector with a period of T, and calculating the acceleration and angular velocity values of the asset relative to coordinate system c, respectively. and in Let a be the acceleration vector of the asset numbered n at time point n of the k-th asset status detection period. x (n)(k) represents the acceleration value of asset n on the x-axis in coordinate system c at time point n of the k-th asset status detection cycle. y (n)(k) represents the acceleration value of asset numbered n on the y-axis in coordinate system c at time point n of the k-th asset status detection cycle, a z (n)(k) represents the acceleration value of asset numbered n on the z-axis in coordinate system c at time point n of the k-th asset status detection cycle. Let ω be the rotational angular velocity vector of asset n at time point n of the k-th asset status detection cycle. ψ (n)(k) represents the heading angle and angular velocity value of the asset with the number n at time point n of the k-th asset status detection cycle, ω θ (n)(k) represents the pitch angle and angular velocity value of the asset numbered n at time point n of the k-th asset status detection cycle, ω φ (n)(k) represents the roll angle and angular velocity value of the asset numbered n at the k-th asset status detection cycle.
[0043] Step a4 includes: calculating the attitude data value of each asset according to formula (2):
[0044]
[0045] Step a5 includes:
[0046] Step a5-1: calculating the modulus of the displacement vector of each asset according to formula (3):
[0047]
[0048] Wherein, ΔS(n)(k) represents the value of the modulus of the displacement of the asset numbered n at the k-th asset state detection cycle moment;
[0049] Step a5-2: traverse the value of the modulus ΔS(n)(k) of the displacement vector of each asset, and perform the following operations:
[0050] Step a5-2-1: when ΔS(n)(k)>A, set the position trust state data STU of the n-th asset WZ The value of (n)(k) is set to 0, and the broadcast signal function of the Bluetooth module of the asset state detector installed on the n-th asset is stopped; wherein A and B are constants;
[0051] Step a5-2-2: when A≥ΔS(n)(k)≥B and NUM move (n)>3, set the position trust state data STU of the n-th asset WZ The value of (n)(k) is set to 0, and the broadcast signal function of the Bluetooth module of the asset state detector installed on the n-th asset is stopped;
[0052] Step a5-2-3: when A≥ΔS(n)(k)≥B and its NUM move (n)≤3, increment the NUM of the n-th asset move (n) value by 1, and set the position data of the n-th asset to
[0053] Step a5-2-4: when ΔS(n)(k)<B, assign the position data of the n-th asset at the previous state detection cycle moment to the position data at the current moment, that is, WZ c (n)(k)=WZ c (n)(k-1);
[0054] Step a5-3: Identify assets with a location confidence status value of 0. Activate the Bluetooth signal receiving function of the Bluetooth module of the asset status detector installed on the asset with a location confidence status value of 0. Collect Bluetooth signals broadcast by other asset status detectors at the location. When the number of collected Bluetooth signals is greater than or equal to 3 and the strength value of each Bluetooth signal is greater than -75dB, apply the received Bluetooth signals combined with the location data of the asset installed by the asset status collector based on the broadcast signals. Calculate the location of the asset installed by the asset status checker based on the received Bluetooth signals using the RSSI-based multilateral positioning method. After the calculation is completed, assign the obtained location data to the asset's location data value at the current asset status detection cycle. Then, set the asset's location confidence status data value to 1, set the asset's continuous cumulative movement cycle count to 0, and enable the broadcast signal function of the Bluetooth module of the asset status detector installed on the asset.
[0055] Step a5-4: Repeat step a5-3 until there are no assets with a location trusted state data value of 0, or assets with a location trusted state data value of 0 cannot collect Bluetooth signals broadcast by more than 3 other asset state detectors and the signal strength is greater than -75dB.
[0056] Step a6 includes:
[0057] The asset IDs with a location confidence status data value of 0 are grouped into a set U. ng (k), assuming the location reliability status data of M assets is 0, then U ng The value of (k) is denoted as U ng (k)={ng1,ng2,…ng m ,…,ng M}, where ng m ∈{1,2,3,…,N},m∈{0,1,2,…,M}, M≤N,ng m The set U represents the asset number whose trusted state value is 0 at the m-th location; ng The assets in (k) are the objects that need to be located by the patrol robot, and U ng (k) is defined as the robot's inspection list; the set U is estimated using formula (4). ng Location estimation data for each asset in (k):
[0058]
[0059] Among them, WZ c_gs (ng m (k) represents the time of the k-th asset status detection cycle, numbered ng. m The estimated position of the asset in coordinate system c; g is the gravitational acceleration value.
[0060] Step a7 includes:
[0061] The system transmits the location confidence status data, location data, location estimation data, and attitude data of each asset to the Metaverse system. After receiving the data, the Metaverse venue system sets the attitude attributes of the 3D models of each asset to the attitude data, sets the location attributes of the 3D models of assets with a location confidence status value of 1 to the location data, sets the location confidence status attributes of the 3D models to 1, sets the location attributes of the 3D models of assets with a location confidence status value of 0 to the location estimation data, and sets the location confidence status attributes of the 3D models to 0. Then, the Metaverse venue system renders and displays the data based on the set attribute values of each asset, thus completing the real-time mapping of the position and attitude of each asset in the scene.
[0062] The robot's fixed-point scheduling, inspection, and detection process specifically includes the following steps:
[0063] Step b1, perform the inspection robot scheduling judgment:
[0064] Receive U ng The value of (k) when set U ng If (k) is not empty, jump to step b2; otherwise, wait for time T and then re-execute step b1.
[0065] Step b2, Sorting asset locations:
[0066] Will U ng The estimated location values of each asset in (k) are sorted sequentially by their x-axis, y-axis, and z-axis coordinates. The sorted set is represented by U. ng_yx (k) represents the set U ng_yx The elements in (k) are represented by ng_yx m It means ng_yx m This represents the asset ID at the m-th position after sorting, where the confidence state value is 0. (ng_yx) m ∈{1,2,3,…,N};
[0067] Step b3, Dispatch the patrol robot:
[0068] The metaverse venue system dispatch and inspection robot enters U ng_yx (k) The estimated location of the asset corresponding to the first element in the set is within the sphere with the center of the sphere as the center;
[0069] Step b4, perform asset location calculation:
[0070] Start U ng_yx(k) The first element in the set corresponds to the signal receiving function of the Bluetooth module of the asset status detector installed on the asset. When the Bluetooth module receives Bluetooth signals broadcast from 3 or more patrol robots and the signal strength is greater than -75dB, it obtains the current position data of the patrol robots. Combined with the Bluetooth signal data received by the asset status detector installed on the asset, it calculates the current position data value of the asset using a multi-lateral positioning algorithm based on RSSI. After the calculation is completed, the obtained position data is assigned to the position data of the asset at the current asset status detection cycle. The value of the asset's reliable position status data is set to 1, the number of consecutive cumulative movement cycles of the asset is set to 0, and the signal broadcasting function of the Bluetooth module of the asset status detector installed on the asset is started. Delete set U. ng (k) Midpoint of elements and set U ng_yx (k) For elements with the same value as the first element, repeat step b1.
[0071] This method boasts low implementation costs, simple engineering, and rapid construction. It only requires installing an asset status detector with Bluetooth communication, LoRa communication, and a six-axis inertial sensor on each asset, deploying a basic LoRa communication and Ethernet communication gateway approximately every 500 meters, and deploying three or more ordinary robots equipped with Bluetooth iBeacon devices, LoRa communication, and AGVs. Combined with the data receiving and 3D rendering functions of the Metaverse Venue system, it can achieve real-time virtual-real interaction between all assets in the Metaverse Venue.
[0072] The present invention has the following beneficial effects:
[0073] 1. Low construction cost: This method can comprehensively apply low-cost Bluetooth positioning, inertial sensors, LoRa communication and AGV robot technology, which can provide a low-cost and high-precision real-time detection solution for massive asset status for large venues or large buildings. It can quickly map the status of assets in the physical world to the digital world, laying a solid foundation for the construction of metaverse venues.
[0074] 2. Comprehensive asset coverage: This method achieves comprehensive asset status detection by combining "self-organizing dynamic and static asset status detection" with "robot-based fixed-point scheduling and inspection."
[0075] 3. Good real-time performance of asset status detection: This method can perform real-time status detection on assets that are closely distributed (with 3 or more other assets within a 6-meter range) or whose position movement is less than 2 meters through "asset self-organized dynamic and static combined status detection". For assets that are more isolated (with fewer than 3 other assets within a 6-meter range) and whose position movement exceeds 2 meters, "robot fixed-point scheduling and inspection detection" can quickly achieve near real-time positioning. Since assets are usually closely distributed in engineering practice and most assets do not move frequently, this method can achieve real-time status detection for most assets most of the time, and near real-time status detection for a small number of assets during other periods.
[0076] 4. Simple, quick, and easy to implement: This method only requires the deployment of 3 or more patrol robots, one metaverse venue system, and one communication gateway every 500 meters. At the same time, when the asset is put into use, the asset status detector is attached and the asset status is initialized to complete the construction of the project. There is no need for large-scale construction projects such as the large-scale deployment of Bluetooth base stations or cameras. The construction is simple, quick, and easy to promote. Attached Figure Description
[0077] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.
[0078] Figure 1 This is a system configuration diagram.
[0079] Figure 2 This is a flowchart for the dynamic and static combined status detection of asset self-organization.
[0080] Figure 3 This is a flowchart of the robot's fixed-point scheduling, inspection, and testing process.
[0081] Figure 4 This is a schematic diagram of the scene coordinate system.
[0082] Figure 5 This is a schematic diagram of the asset status detector installation. Detailed Implementation
[0083] This invention provides a method for constructing a real-time virtual-real interactive scene in a metaverse venue. The constructed scene mainly includes a metaverse venue system, a communication gateway, an asset attitude detector, and an inspection robot, such as... Figure 1 As shown.
[0084] The functions of each component are described below:
[0085] Assets: These are tangible physical assets in the physical world.
[0086] Asset Status Detector: Primarily responsible for asset attitude detection, location positioning, and data communication. The asset attitude detector integrates a six-axis inertial sensor, a Bluetooth positioning module, a LoRa communication module, and a replaceable battery module. The six-axis inertial sensor primarily detects changes in asset attitude and position. The Bluetooth positioning module broadcasts Bluetooth signals and receives Bluetooth signals from other Bluetooth devices in its location to achieve its own location positioning. The LoRa communication module transmits data from the asset status detector to the communication gateway. The replaceable battery module provides power to the asset status detector, with a battery life of approximately 1 to 3 years, and is replaceable.
[0087] Communication gateway: Responsible for receiving data from each asset status detector and forwarding it to the metaverse venue system. It has a built-in LORA communication module and an Ethernet communication module. The LORA communication module sends and receives communication data with each asset status detector and patrol robot through LORA communication, while the Ethernet module is responsible for establishing the connection between the LORA communication module and the metaverse venue system.
[0088] Metaverse Venue System: It establishes a connection with each asset status detector and patrol robot through communication gateway. Based on the data detected by the asset status detector, it completes the positioning and attitude detection of each asset. When an asset cannot be positioned, the patrol robot is dispatched to perform single-point patrol positioning. At the same time, it uses 3D visualization technology to complete the 3D model rendering of each asset based on the latest status data of each asset, so as to provide venue operators with an immersive presentation of the position and attitude of each asset.
[0089] The inspection robot consists of an AGV base, a Bluetooth iBeacon device, and a LoRa communication module. The AGV base is responsible for the robot's movement and positioning; the Bluetooth iBeacon device broadcasts Bluetooth signals, including the MAC address of the iBeacon device; and the LoRa communication module enables communication between the inspection robot and the communication gateway via LoRa.
[0090] This invention specifically provides a method for constructing a real-time virtual-real interactive scene of a metaverse venue. This method is implemented by two detection processes: one is the "asset self-organization dynamic and static combined state detection process" and the other is the "robot fixed-point scheduling and inspection detection process".
[0091] The “Asset Self-Organizing Dynamic and Static Combined State Detection Process” uses an asset state detector to self-organize and calculate the attitude and position of each asset based on the motion state of the asset in a cycle of T. At the same time, it generates a “Robot Inspection List” for the assets that cannot be detected, which guides the “Robot Fixed-Point Scheduling Inspection and Detection Process” to perform the detection. T is the asset state detection cycle, and its value is generally set in the range of 500 milliseconds to 5 seconds.
[0092] The other process is the "Robot Fixed-Point Inspection and Detection Process," which supplements the "Asset Self-Organizing Dynamic and Static Combined Status Detection Process." Based on the "Robot Inspection List" generated in the "Asset Self-Organizing Dynamic and Static Combined Status Detection Process," a fixed-point inspection robot is dispatched to inspect the assets in that list to complete the status detection of the assets. The flowcharts for the two processes are as follows: Figure 2 and Figure 3 As shown.
[0093] The specific execution steps of the process are described below:
[0094] Asset self-organization dynamic and static combined status detection process:
[0095] Step a1, Constructing a Digital Scene
[0096] Using 3D modeling software, a 3D model of the physical scene is created. A point is selected in the scene as the origin o of the coordinate system. Vertically upward is the positive z-axis, eastward is the positive x-axis, and northward is the positive y-axis. This coordinate system is denoted as coordinate system c, and its schematic diagram is shown below. Figure 4 As shown, a communication gateway is installed approximately every 500 meters in the scene. The communication gateway connects to each asset status detector in the south via LORA communication, and forwards data to the Metaverse Venue System in the north via Ethernet communication or mobile communication network.
[0097] Step a2, Asset Modeling and Initialization
[0098] Step a2-1, Asset Modeling
[0099] Suppose there are N assets in the scene. Each asset is numbered, denoted by n, where n∈{1,2,3,...,N}. A 3D modeling software is used to model each asset in the scene. The completed asset model is denoted as M(n). The center of each asset is chosen as the origin o. n The forward direction of the asset is x. n The positive axis direction, perpendicular to the asset surface and outwards, is z. n The positive direction of the axis, and the right front direction of the asset is y. n Establish a coordinate system along the positive axis. This coordinate system is the coordinate system for asset numbered n, denoted as coordinate system w. zbx (n). Install asset attitude detectors at each asset center location, with the center of the asset attitude detector aligned with the asset center. An installation diagram is shown below. Figure 5 As shown, three attributes are constructed for the 3D model of each asset: location attribute, location confidence state attribute, and pose attribute. The attributes are described below:
[0100] Position attribute: Its value represents the position of the 3D asset model in coordinate system c, consisting of three coordinate values: x, y, and z. When the 3D asset model is rendered, its position is rendered according to this attribute value.
[0101] Location confidence status attribute: When building a 3D model, an indicator model needs to be built. When this attribute value is 1, the indicator model is highlighted. When this attribute value is 0, the indicator model is not highlighted.
[0102] Attitude attribute: Its value represents the rotation state of the asset's 3D model in coordinate system c. It consists of three angle values: yaw angle, pitch angle, and roll angle. The yaw angle represents the angle of rotation of the asset's 3D model around the z-axis of coordinate system c; the pitch angle represents the angle of rotation of the asset's 3D model around the x-axis of coordinate system c; and the roll angle represents the angle of rotation of the asset's 3D model around the y-axis of coordinate system c. Counterclockwise rotation is positive, and clockwise rotation is negative. When rendering the asset's 3D model, its attitude is rendered according to this attribute value.
[0103] Step a2-2, Asset Status Initialization
[0104] Asset status data includes location data, location confidence status data, and attitude data. Details of each data type are as follows:
[0105] Location data: Represents the position coordinates of the asset's center point in coordinate system c, denoted as WZ. c (n)(k) represents the expression whose value can be expressed as Where n is the asset number, and k represents the time of the kth asset status detection cycle;
[0106] Location Trustworthiness Status: Represents the trustworthiness status of asset location data. A value of 1 indicates that the asset location data is trustworthy, and a value of 0 indicates that the asset location data is untrustworthy. Let STU represent the location trustworthiness status of each asset. wz (n)(k) represents, where n is the asset number and k represents the time of the kth detection period;
[0107] Pose data: Represents the pose of the asset in coordinate system c, expressed as ZT. c (n)(k) represents the expression whose value can be expressed as Where n is the asset number, k is the time of the kth asset status detection cycle, ψ(n)(k) is the heading angle of the asset with number n at the time of the kth asset status detection cycle, that is, the angle of rotation of the asset around the z-axis of coordinate system c; θ(n)(k) is the pitch angle of the asset with number n at the time of the kth asset status detection cycle, that is, the angle of rotation of the asset around the x-axis of coordinate system c; φ(n)(k) is the roll angle of the asset with number n at the time of the kth asset status detection cycle, that is, the angle of rotation of the asset around the y-axis of coordinate system c.
[0108] Once the asset status detectors for each asset are installed, asset status detection begins, and this moment is designated as the 0th asset status detection cycle. Position and attitude data of each asset relative to coordinate system c are acquired via measurement, and the position confidence data value of each asset at this moment is set to 1. These data are represented by WZ... c (n)(0), ZT c (n)(0) and STU WZ (n)(0) indicates that their values can be represented by Equation 1.
[0109]
[0110] in:
[0111] WZ c (n)(0): The position data of asset number n in coordinate system c during the 0th asset status detection period;
[0112] x(n)(0): The x-axis coordinate of the position data of asset number n in coordinate system c during the 0th asset status detection period, which is obtained by measurement;
[0113] y(n)(0): The y-axis coordinate of the position data of asset number n in coordinate system c during the 0th asset status detection period, which is obtained by measurement;
[0114] z(n)(0): The z-axis coordinate of the position data of asset number n in coordinate system c during the 0th asset status detection period, which is obtained by measurement;
[0115] ZT c (n)(0): The attitude data of asset numbered n in coordinate system c during the 0th asset status detection cycle;
[0116] ψ(n)(0): The heading angle of the attitude data of asset numbered n in coordinate system c during the 0th asset status detection cycle, obtained by measurement;
[0117] θ(n)(0): The pitch angle of the attitude data of asset numbered n in coordinate system c during the 0th asset status detection cycle, obtained by measurement;
[0118] φ(n)(0): The roll angle of the attitude data of asset numbered n in coordinate system c during the 0th asset status detection cycle, obtained by measurement;
[0119] ZT c (n)(0): The location confidence status data of asset number n in the 0th asset status detection period, with its value set to 1;
[0120] Steps a2-3: Start the equipment operation
[0121] Enable the Bluetooth signal broadcast function of each asset status detector. The broadcast data includes the MAC address of the Bluetooth module. Bind this MAC address to the asset number of the asset installed on the asset status detector, and record it as MAC. n Where n is the asset number; the broadcast interval can be set from 100 milliseconds to 1 second, and can be set as needed. The smaller the interval, the more power it consumes but the better the real-time detection, and vice versa.
[0122] Create a variable for each asset to record the cumulative number of asset connections and moves, using NUM. move (n) represents the asset number, which is initially set to 0;
[0123] Step a3: Asset Status Detector Data Acquisition and Processing
[0124] Data from the six-axis inertial sensor in the asset status detector is collected cyclically with a period of T. The acceleration and angular velocity of the asset relative to coordinate system c are obtained through quaternion, rotation matrix, or Euler angle calculation methods. Let their values be respectively... and in Let a be the acceleration vector of the asset numbered n at time point n of the k-th asset status detection period. x (n)(k) represents the acceleration value of asset n on the x-axis in coordinate system c at time point n of the k-th asset status detection cycle. y (n)(k) represents the acceleration value of asset numbered n on the y-axis in coordinate system c at time point n of the k-th asset status detection cycle, a z (n)(k) represents the acceleration value of asset numbered n on the z-axis in coordinate system c at time point n of the k-th asset status detection cycle. Let ω be the rotational angular velocity vector of asset n at time point n of the k-th asset status detection cycle. ψ (n)(k) represents the heading angle and angular velocity value of the asset with the number n at time point n of the k-th asset status detection cycle, ω θ (n)(k) represents the pitch angle and angular velocity value of the asset numbered n at time point n of the k-th asset status detection cycle, ω φ (n)(k) represents the roll angle and angular velocity value of the asset numbered n at the k-th asset status detection cycle.
[0125] Step a4, calculate asset pose
[0126] Calculate the attitude data values of each asset according to formula (2).
[0127]
[0128] Step a5, performing asset self-organized positioning
[0129] Step a5-1, calculating the modulus of the displacement vector of each asset according to Formula (3):
[0130]
[0131] Where ΔS(n)(k) represents the value of the modulus of the displacement of the asset numbered n at the moment of the k-th asset state detection cycle;
[0132] Step a5-2, traversing the value of the modulus ΔS(n)(k) of each asset displacement vector, and performing the following operations:
[0133] Step a5-2-1, when ΔS(n)(k)>A, set the position trust state data STU of the n-th asset WZ the value of (n)(k) to 0, and stop the broadcast signal function of the Bluetooth module of the asset state detector installed on the n-th asset. Where A is a constant, its value is generally greater than B and less than or equal to 2, B is also a constant, which is used to eliminate the disturbance error of the asset state detector, and is generally a value greater than 0 and less than 1;
[0134] Step a5-2-2, when A≥ΔS(n)(k)≥B and NUM move (n)>3, set the position trust state data STU of the n-th asset WZ (n)(k) to 0, and stop the broadcast signal function of the Bluetooth module of the asset state detector installed on the n-th asset;
[0135] Step a5-2-3, when A≥ΔS(n)(k)≥B and its NUM move (n)≤3, increment the NUM of the n-th asset move the value of (n) by 1, and set the position data of the n-th asset to
[0136] Step a5-2-4, when ΔS(n)(k)<B, assign the position data of the n-th asset at the moment of the previous state detection cycle to the position data at the current moment, that is, WZ c (n)(k)=WZ c (n)(k-1);
[0137] Step a5-3: Identify assets with a location confidence status value of 0, activate the Bluetooth signal receiving function of the Bluetooth module of the asset status detector installed on these assets, and collect Bluetooth signals broadcast by other asset status detectors at the location. When the number of collected Bluetooth signals is greater than or equal to 3 and the strength value of each Bluetooth signal is greater than -75dB, apply these received Bluetooth signals in combination with the location data of the asset installed by the asset status collector that broadcast these signals, and calculate the location of the asset installed by the asset status checker that received these Bluetooth signals using a multilateral positioning method based on RSSI (Received Signal Strength Indication). After the calculation is completed, assign the obtained location data to the location data value of the asset at the time of the current asset status detection cycle, then set the location confidence status data value of the asset to 1, set the number of consecutive cumulative movement cycles of the asset to 0, and activate the broadcast signal function of the Bluetooth module of the asset status detector installed on the asset.
[0138] Step a5-4: Repeat step a5-3 until there are no assets with a location trusted state data value of 0 or these assets with a location trusted state data value of 0 can not collect Bluetooth signals broadcast by 3 or more other asset state detectors and the signal strength is greater than -75dB.
[0139] Step a6, update the robot patrol list
[0140] Group the asset IDs with a location trust status data value of 0 into a set, and use U ng (k) indicates that if the location confidence status data of M assets is 0, then U ng The value of (k) can be denoted as U ng (k)={ng1,ng2,…ng m ,…,ng M}, where ng m ∈{1,2,3,…,N}, m∈{0,1,2,…,M}, M≤N. Set U ng The assets in (k) are the objects that need to be located by the inspection robot, therefore U ng (k) is defined as the robot's inspection list; the set U is estimated using formula (4). ng Location estimation data for each asset in (k):
[0141]
[0142] Among them, WZ c_gs (ng m (k) represents the time of the k-th asset status detection cycle, numbered ng. mThe estimated position of the asset in coordinate system c; g is the gravitational acceleration value;
[0143] Step a7, update asset status
[0144] The system transmits the location confidence status data, location data, location estimation data, and attitude data of each asset to the Metaverse system. Upon receiving the data, the Metaverse venue system sets the attitude attributes of each asset's 3D model to its attitude data. For assets with a location confidence status value of 1, the system sets the location attribute of the 3D model to its location data and sets its location confidence status attribute to 1. For assets with a location confidence status value of 0, the system sets the location attribute of the 3D model to its estimated location data and sets its location confidence status attribute to 0. The Metaverse venue system then renders and displays the data based on these set attribute values, thus completing the real-time mapping of the position and attitude of each asset in the scene.
[0145] Robotic fixed-point inspection process:
[0146] Step b1, perform the inspection robot scheduling judgment:
[0147] Receive U ng The value of (k) when set U ng If (k) is not empty, jump to step b2; otherwise, wait for time T and then re-execute this step.
[0148] Step b2, Sorting asset locations:
[0149] Will U ng The estimated location values of each asset in (k) are sorted sequentially by their x-axis, y-axis, and z-axis coordinates, either from smallest to largest or from largest to smallest. The sorted set is represented by U. ng_yx (k) is denoted by ng_yx. m It means ng_yx m ∈{1,2,3,…,N};
[0150] Step b3, Dispatch the patrol robot:
[0151] The metaverse venue system dispatches three or more patrol robots to enter the U-shaped area. ng_yx (k) The estimated location of the asset corresponding to the first element in the set is the center of the sphere, within a sphere with a radius of 6 meters;
[0152] Step b4: Perform asset location calculation
[0153] Start U ng_yx(k) The first element in the set corresponds to the signal receiving function of the Bluetooth module of the asset status detector installed on the asset. When the module receives Bluetooth signals broadcast from 3 or more patrol robots and the signal strength is greater than -75dB, it obtains the current position data of the patrol robots. Combined with the Bluetooth signal data received by the asset status detector installed on the asset, it calculates the current position data value of the asset using a multi-lateral positioning algorithm based on RSSI. After the calculation is completed, the obtained position data is assigned to the position data of the asset at the current asset status detection cycle, the value of the asset's reliable position status data is set to 1, the number of consecutive cumulative movement cycles of the asset is set to 0, and the signal broadcasting function of the Bluetooth module of the asset status detector installed on the asset is started. The set U is then deleted. ng (k) Midpoint of elements and set U ng_yx (k) For elements with the same value as the first element, repeat step b1.
[0154] Example
[0155] This method was applied to a 1.6 million m2 convention center project, involving 112,000 assets. A total of 112,000 asset status detectors were installed, 10 communication gateways were deployed, 3 patrol robots were used, and a metaverse venue system was constructed. The Bluetooth modules of the asset status detectors broadcast signals with a period of 500ms; the Bluetooth iBeacon modules of each robot also broadcast signals with a period of 500ms, with a maximum travel speed of 2 m / s; the system completes an asset status check every 2 seconds. Statistical analysis of data from three months of system operation shows that 96.78% of asset status checks could be completed using the "self-organized dynamic and static combined status detection" method, while only 3.22% of asset status checks required "robot fixed-point scheduling patrol detection" to detect assets that could not be completed using this method. Assets detected through "robot fixed-point scheduling patrol detection" were updated on average after 21.56 minutes.
[0156] This invention provides a method for constructing a real-time virtual-real inter-scene in a metaverse venue. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.
Claims
1. A method for constructing a metaverse venue scene with real-time virtual-real inter-realistic mapping, characterized in that, include: The asset self-organizing dynamic and static combined status detection process and the robot fixed-point scheduling inspection process are executed in parallel. The asset self-organizing dynamic and static combined state detection process is to use the asset state detector to self-organize and calculate the attitude and position of each asset based on the motion state of the asset in a cycle of T. At the same time, the robot inspection list is generated for assets that cannot be detected, which guides the robot to perform the inspection process at fixed points. T is the asset state detection cycle. The robot fixed-point scheduling inspection and detection process involves scheduling a robot to inspect the assets in the robot inspection list in order to complete the status detection of the assets. The asset self-organization dynamic and static combined status detection process specifically includes the following steps: Step a1: Construct a digital scene; Step a2, asset modeling and initialization; Step a3: The asset status detector collects and processes data; Step a4, calculate asset pose; Step a5: Perform asset self-organization positioning; Step a6: Update the robot patrol list; Step a7: Update asset status; Step a5 includes: Step a5-1, calculate the magnitude of the displacement vector of each asset according to formula (3): (3) in, Indicates the first At the time of each asset status monitoring cycle, numbered as follows: The magnitude of the asset's displacement; For the first The time period for each asset status monitoring cycle is numbered as follows: The acceleration vector of the asset is expressed in the form of three axial components. , ,and Represented as: ; Step a5-2: Iterate through the modulus of each asset's displacement vector. For the value, perform the following operations: Step a5-2-1, when At time A, the location reliability data of the nth asset will be... Setting the value to 0 stops the Bluetooth module broadcast signal function of the asset status detector installed on the nth asset; where A and B are constants. Step a5-2-2, when and At that time, the location trusted status data of the nth asset will be... Setting the value to 0 stops the Bluetooth module broadcast signal function of the asset status detector installed on the nth asset; Step a5-2-3, when and When, the nth asset The value is incremented by 1, and the location data of the nth asset is set to... Where g is the value of gravitational acceleration; Step a5-2-4, when At that time, the position data of the nth asset at the previous state detection cycle is assigned to the position data at the current time, that is... ; Step a5-3: Identify assets with a location confidence state value of 0. Activate the Bluetooth signal receiving function of the Bluetooth module of the asset status detector installed on the asset with a location confidence state value of 0. Collect Bluetooth signals broadcast by other asset status detectors at the same location. When the number of collected Bluetooth signals is greater than or equal to a certain number and the strength value of each Bluetooth signal is greater than a threshold, apply the location data of the asset installed on the asset status detector using the received Bluetooth signals. Calculate the asset's location using a multi-lateral positioning method based on RSSI. After calculation, assign the obtained location data to the asset's location data value at the time of the current asset status detection cycle. Then, set the asset's location confidence state data value to 1 and set the asset's continuous cumulative movement cycle count value... Set to 0 and enable the Bluetooth module broadcast signal function of the asset status detector installed on the asset; Step a5-4: Repeat step a5-3 until there are no assets with a location trusted state data value of 0 or assets with a location trusted state data value of 0 cannot collect a certain number of Bluetooth signals broadcast by other asset state detectors and the signal strength is greater than the threshold.
2. The method according to claim 1, characterized in that, Step a1 includes: performing 3D modeling of the physical scene and establishing a coordinate system c: selecting a point in the scene as the origin o, with the vertical upward direction as the positive z-axis, the east direction as the positive x-axis, and the north direction as the positive y-axis; installing a communication gateway in the scene, with the communication gateway connecting to each asset status detector in the south via LORA communication, and forwarding data to the Metaverse Venue System in the north via Ethernet communication or mobile communication network.
3. The method according to claim 2, characterized in that, Step a2 includes: Step a2-1, Asset Modeling: Suppose there are N assets in the scenario. Each asset is numbered individually, and the asset number is represented by... express, To model each asset in the scene, the model of the nth asset is denoted as [missing information]. Establish a coordinate system for the asset numbered n. : Select the center of the asset as the origin The forward direction of the asset is The positive axis direction is perpendicular to the asset surface and extends outwards. The positive axis direction, the right front direction of the asset is... The positive axis direction; install asset status detectors at the center of each asset, with the center of the asset status detector aligned with the center of the asset; construct three attributes for the 3D model of each asset: position attribute, position confidence status attribute, and attitude attribute. The value of the position attribute indicates the position of the 3D asset model in coordinate system c, including three coordinate values: x, y, and z. When rendering the 3D asset model, the position is rendered according to the value of the position attribute. When constructing the 3D model, an indicator model is built. When the value of the location confidence state attribute is 1, the indicator model is highlighted; when it is 0, the indicator model is not highlighted. The values of the attitude attributes represent the rotational state of the asset's 3D model in coordinate system c, including yaw, pitch, and roll angles. The yaw angle represents the angle of rotation of the asset's 3D model around the z-axis of coordinate system c, the pitch angle represents the angle of rotation of the asset's 3D model around the x-axis of coordinate system c, and the roll angle represents the angle of rotation of the asset's 3D model around the y-axis of coordinate system c. Counterclockwise rotation is positive, and clockwise rotation is negative. When rendering the asset's 3D model, the attitude is rendered according to the values of the attitude attributes. Step a2-2, Asset Status Initialization: Asset status data includes location data, location confidence status data, and attitude data: The location data represents the position coordinates of the asset's center point in coordinate system c. express, ,in For the asset number, To indicate the first At any point in the asset status monitoring cycle Number The central point of the assets is at the The asset status detection cycle is located in coordinate system c. axis coordinate values axis coordinate values and Axis coordinate values; The location credibility status data represents the credibility status of asset location data. A value of 1 indicates that the asset location data is credible, and a value of 0 indicates that the asset location data is untrustworthy. Let the location credibility status of each asset be represented by... It means that among them For the asset number, To indicate the first Each detection cycle time; The attitude data represents the asset's attitude in coordinate system c, using... express, ,in, For the first The time period for each asset status monitoring cycle is numbered as follows: The heading angle of the asset, that is, the angle by which the asset rotates about the z-axis of coordinate system c; For the first The time period for each asset status monitoring cycle is numbered as follows: The pitch angle of the asset, that is, the angle by which the asset rotates about the x-axis of coordinate system c; For the first The time period for each asset status monitoring cycle is numbered as follows: The roll angle of an asset is the angle by which the asset rotates about the y-axis of coordinate system c. Once the asset status detectors for each asset are installed, asset status detection begins, and this moment is designated as the 0th asset status detection cycle. Position data of each asset relative to coordinate system c is acquired through measurement. and attitude data And the location and reliable status data of each asset at this moment. The value is set to 1, as shown below: , (1) in, Indicates the number is The position data of the asset in coordinate system c during the 0th asset status detection cycle; Indicates the number is The x-axis coordinate value of the asset's position data in coordinate system c during the 0th asset status detection cycle is obtained through measurement; Indicates the number is The y-axis coordinate of the asset's position data in coordinate system c during the 0th asset status detection cycle is obtained through measurement; Indicates the number is The z-axis coordinate of the asset's position data in coordinate system c during the 0th asset status detection cycle is obtained through measurement; Indicates the number is The attitude data of the asset in coordinate system c during the 0th asset state detection cycle; Indicates the number is The heading angle of the asset's attitude data in coordinate system c during the 0th asset state detection cycle is obtained by measurement; Indicates the number is The pitch angle of the asset's attitude data in coordinate system c during the 0th asset state detection cycle is obtained by measurement; Indicates the number is The roll angle of the asset's attitude data in coordinate system c during the 0th asset state detection cycle is obtained by measurement; Indicates the number is The asset's location and trusted status data in the 0th asset status detection period are set to 1. Steps a2-3: Start the equipment operation: Enable the Bluetooth signal broadcast function of each asset status detector. The broadcast data includes the MAC address of the Bluetooth positioning module in the asset status detector. Use the MAC address values of the Bluetooth positioning modules of the asset status detectors installed on each asset... This indicates that n is the asset number; Create a variable for each asset Record the number of consecutive cumulative asset movement cycles. The initial value is set to 0.
4. The method according to claim 3, characterized in that, Step a3 includes: cyclically collecting data detected by the six-axis inertial sensor in the asset status detector with a period of T, and calculating the acceleration and angular velocity values of the asset relative to coordinate system c, respectively. and ,in For the first The time period for each asset status monitoring cycle is numbered as follows: The acceleration vector of the assets, For the first The time period for each asset status monitoring cycle is numbered as follows: The acceleration value of the asset on the x-axis in coordinate system c. For the first The time period for each asset status monitoring cycle is numbered as follows: The acceleration value of the asset on the y-axis in coordinate system c. For the first The time period for each asset status monitoring cycle is numbered as follows: The acceleration value of the asset on the z-axis in coordinate system c. For the first The time period for each asset status monitoring cycle is numbered as follows: The rotational angular velocity vector of the asset. For the first The time period for each asset status monitoring cycle is numbered as follows: The heading angle and angular velocity values of the assets. For the first The time period for each asset status monitoring cycle is numbered as follows: The pitch angle and angular velocity values of the asset. For the first The time period for each asset status monitoring cycle is numbered as follows: The roll angle and angular velocity values of the asset.
5. The method according to claim 4, characterized in that, Step a4 includes: calculating the attitude data values of each asset according to formula (2): (2)。 6. The method according to claim 5, characterized in that, Step a6 includes: Group the asset IDs with a location confidence status value of 0 into a set. If the location confidence status data of M assets is 0, then The value is denoted as ,in , , , The set represents the asset number whose trusted state value is 0 at the m-th location; The assets in the text are objects that need to be located by robots. Defined as a robot inspection list; the set is estimated using formula (4). Location estimation data for each asset in the middle: (4) in, For the first Asset status monitoring cycle time, numbered as The estimated position data of the asset in coordinate system c.
7. The method according to claim 6, characterized in that, Step a7 includes: The system transmits the location confidence status data, location data, location estimation data, and attitude data of each asset to the Metaverse Venue System. After receiving the data, the Metaverse Venue System sets the attitude attributes of the 3D models of each asset to the attitude data, sets the location attributes of the 3D models of assets with a location confidence status value of 1 to the location data, sets the location confidence status attributes of the 3D models to 1, sets the location attributes of the 3D models of assets with a location confidence status value of 0 to the location estimation data, and sets the location confidence status attributes of the 3D models to 0. Then, the Metaverse Venue System renders and displays the data based on the set attribute values of each asset, thus completing the real-time mapping of the position and attitude of each asset in the scene.
8. The method according to claim 7, characterized in that, The robot's fixed-point scheduling, inspection, and detection process specifically includes the following steps: Step b1, perform robot scheduling judgment: take over The value of the set If the value is not empty, proceed to step b2; otherwise, wait for time T and then re-execute step b1. Step b2, Sorting asset locations: Will The estimated location values of each asset are sorted sequentially by their x-axis, y-axis, and z-axis coordinates. The sorted set is then used... Represents a set The elements in express, This represents the asset number at the m-th position after sorting, where the trust state value is 0. ; Step b3, scheduling the robot: The metaverse venue system dispatch robot enters to The estimated location of the asset corresponding to the first element in the set is within the sphere's center. Step b4, perform asset location calculation: start up The first element in the set corresponds to the signal receiving function of the Bluetooth module of the asset status detector installed on the asset. When the Bluetooth module receives a certain number of Bluetooth signals broadcast by the robot and the signal strength of each signal is greater than a threshold, it acquires the robot's current position data. Combined with the Bluetooth signal data received by the asset status detector installed on the asset, it calculates the asset's current position data value using a multi-directional positioning algorithm based on RSSI. After the calculation is completed, the obtained position data is assigned to the asset's position data at the current asset status detection cycle, the asset's reliable position status data value is set to 1, the number of consecutive cumulative movement cycles of the asset is set to 0, and the signal broadcasting function of the Bluetooth module of the asset status detector installed on the asset is activated. The set is then deleted. Median of Elements and Sets For the first element with the same value, repeat step b1.
Citation Information
Patent Citations
Mobile robot reliable location method under dynamic environment
CN105022394A
Coal mine underground personnel and material dispatching method based on digital twinning
CN114677054A
