A method and system for in-situ visualization of gunite thickness based on mixed reality

Abstract

The disclosure provides a mixed reality-based in-situ visualization method and system for shotcrete thickness, relating to the field of civil engineering and intelligent construction, aiming to realize in-situ visualization of shotcrete thickness detection by using mixed reality technology, the method comprising: constructing a three-dimensional grid representation of the shotcrete operation site, establishing a virtual mechanical arm corresponding to the physical mechanical arm in the shotcrete operation site by using the spatial anchor point technology, and keeping the virtual mechanical arm consistent with the physical mechanical arm in pose and spatial position; generating virtual shotcrete particles and setting particle parameters for the virtual shotcrete particles; when a virtual shotcrete particle is detected to collide with the environment grid, mapping the three-dimensional coordinates of the collision point to the grid cell in the two-dimensional height field grid, and calculating the current deposition thickness value of the grid cell; based on the deposition thickness value of the grid cell, obtaining the spraying thickness of each position point in the grid cell, and rendering the spraying thickness of each position point to the target surface through the corresponding color.

Description

Technical Field

[0001] This disclosure relates to the fields of civil engineering and intelligent construction, and in particular to a method and system for in-situ visualization of shotcrete thickness based on mixed reality. Background Technology

[0002] Shotcrete application, a core technology in construction, tunneling, and mining engineering, aims to reinforce target surfaces by spraying concrete or mortar. In practice, the thickness of the sprayed concrete or mortar needs to be monitored to ensure the application meets standards.

[0003] Existing methods for shotcrete thickness detection mainly include core drilling, laser scanning, and ultrasonic testing. While core drilling offers high accuracy, it is a destructive method, costly, and has a limited number of testing points, making it unsuitable for large-area continuous monitoring. Laser scanning is significantly affected by ambient lighting conditions, with accuracy decreasing noticeably in complex environments such as tunnels, and it cannot achieve real-time detection. Ultrasonic testing requires strict surface roughness control, has low efficiency, and is ill-suited to the fast-paced demands of field operations.

[0004] In addition, traditional shotcrete thickness testing can only be carried out after the entire construction process is completed, lacking real-time and intuitive quality monitoring methods, making it difficult to detect and correct problems such as uneven shotcrete, insufficient edge coverage, or excessive shotcrete in a timely manner. Summary of the Invention

[0005] A first aspect of this disclosure provides a method for in-situ visualization of shotcrete thickness based on mixed reality, the method comprising:

[0006] A three-dimensional mesh representation of the shotcrete operation site is constructed, wherein the three-dimensional mesh representation includes the environmental mesh corresponding to the target surface to be sprayed;

[0007] Using spatial anchor point technology, a virtual robotic arm corresponding to the physical robotic arm in the shotcrete operation site is established, and the virtual robotic arm is kept consistent with the physical robotic arm in terms of pose and spatial position.

[0008] Virtual sprayed particles are generated, and particle parameters are set for the virtual sprayed particles based on the posture of the physical robotic arm when performing spraying operations and the spraying parameters of the robotic arm nozzle on the physical robotic arm. The particle parameters include at least the initial velocity of the particle, the gravity influence factor, and the spraying angle of the particle.

[0009] When a collision between the virtual sprayed particle and the environmental mesh is detected, the three-dimensional coordinates of the collision point are mapped to the corresponding mesh cell in the two-dimensional height field mesh, and the current deposition thickness value of the mesh cell is obtained based on the physical parameters of the virtual sprayed particle; wherein, the two-dimensional height field mesh corresponds to the environmental mesh, and the physical parameters include at least the particle mass and adhesion probability of the virtual sprayed particle, and the adhesion probability characterizes the probability that the virtual sprayed particle adheres to the target surface;

[0010] Based on the accumulated deposition thickness value of each grid cell, the coating thickness at each location point in the grid cell is obtained, and the coating thickness at each location point is rendered onto the target surface using a corresponding color; wherein, different coating thicknesses within different thickness ranges correspond to different colors.

[0011] For example, the construction of a three-dimensional mesh representation of the shotcrete operation site includes an environmental mesh corresponding to the target surface to be sprayed, comprising:

[0012] Using the spatial mapping component of a mixed reality device, the depth information of the shotcrete operation site is automatically scanned, and the three-dimensional mesh representation is generated based on the depth information;

[0013] A collider component is added to the three-dimensional mesh representation; wherein the collider component is used to perform collision detection on the environment mesh.

[0014] For example, the step of using spatial anchoring technology to establish a virtual robotic arm corresponding to the physical robotic arm in the shotcrete operation site, and ensuring that the virtual robotic arm and the physical robotic arm maintain consistency in pose and spatial position, includes:

[0015] An initial virtual robotic arm is constructed, and a virtual QR code is set for the virtual robotic arm; wherein the position of the virtual QR code on the virtual robotic arm is consistent with the position of the physical QR code on the physical robotic arm.

[0016] The position of the physical QR code is detected in real time by an image acquisition device, and the pose of the virtual robotic arm in the three-dimensional mesh representation is adjusted according to the real-time position until the real-time poses of the virtual QR code and the physical QR code coincide in the mixed reality view, so that the virtual robotic arm is aligned with the physical robotic arm.

[0017] A persistent spatial anchor point is created for the alignment position when the virtual robotic arm is aligned with the physical robotic arm, and the virtual robotic arm is bound to the spatial anchor point to lock and maintain the alignment relationship between the virtual robotic arm and the physical robotic arm.

[0018] For example, generating virtual sprayed particles and setting particle parameters for the virtual sprayed particles based on the posture of the physical robotic arm during spraying operations and the spraying parameters of the robotic arm nozzles on the physical robotic arm includes:

[0019] Create a particle system in Unity and bind the particle system to the virtual model of the robotic arm nozzle;

[0020] Based on the real-time position of the physical robotic arm during the spraying operation and the spraying parameters, the emission rate, initial velocity, and emission angle of the virtual spraying particles are set.

[0021] Gravity, air resistance, lifespan, and rebound coefficient are added to the virtual sprayed particles to simulate the physical properties of real sprayed materials;

[0022] The lifecycle characterizes the time it takes for virtual jet particles that have not collided to disappear; the rebound coefficient is negatively correlated with the adhesion coefficient of the virtual jet particles.

[0023] For example, the step of mapping the three-dimensional coordinates of the collision point to the corresponding grid cell in the two-dimensional height field grid each time a collision between the virtual sprayed particle and the environmental grid is detected includes:

[0024] A two-dimensional height field data structure is established to obtain the two-dimensional height field grid, and the two-dimensional height field grid is divided to obtain multiple grid cells;

[0025] When a collision between the virtual sprayed particle and the environmental mesh is detected, the three-dimensional coordinates of the collision point are extracted;

[0026] The three-dimensional coordinates are converted into relative coordinates with respect to the origin of the two-dimensional height field grid;

[0027] Based on the size of the mesh cell and the relative coordinates, the mesh cell to which the three-dimensional coordinates are mapped is obtained in multiple two-dimensional height field meshes.

[0028] For example, obtaining the current deposition thickness value of the grid cell based on the physical parameters of the virtual sprayed particles includes:

[0029] The depositional contribution can be obtained using the following formula;

[0030] ;in, The mass of the particle is... The adhesion probability is... The time factor represents the duration for which the robotic arm nozzle continuously sprays paint in the same pose.

[0031] The current deposition thickness value of the grid cell is obtained based on the thickness value deposited by the grid cell before the current collision and the deposition contribution.

[0032] For example, obtaining the coating thickness at each location point in the grid cell based on the accumulated deposition thickness value of each grid cell, and rendering the coating thickness at each location point onto the target surface using a corresponding color, includes:

[0033] By iterating through the deposition thickness values ​​of multiple grid cells, the maximum and minimum deposition thickness values ​​are obtained;

[0034] Based on the maximum deposition thickness value and the minimum deposition thickness value, the deposition thickness values ​​of the multiple grid cells are normalized to obtain the normalized thickness value of each grid cell;

[0035] Based on the normalized thickness value of the grid cell, the coating thickness at each location point in the grid cell is obtained;

[0036] For the difference between the coating thickness and the target deposition thickness at each location point, the coating thickness at that location point is rendered onto the target surface using a corresponding color.

[0037] The target deposition thickness value includes the maximum deposition thickness value and the minimum deposition thickness value; or, the target deposition thickness value is a preset thickness range.

[0038] For example, the mesh cell is a polygon, and the normalized thickness value of the mesh cell includes the normalized thickness values ​​of each endpoint of the mesh cell; obtaining the coating thickness at each location point in the mesh cell based on the normalized thickness value of the mesh cell includes:

[0039] Based on the normalized thickness values ​​of each endpoint of the grid cell, a linear interpolation is performed between the maximum deposition thickness value and the minimum deposition thickness value to obtain the coating thickness at each location point in the grid cell.

[0040] For example, rendering the coating thickness at each of the said locations onto the target surface using a corresponding color includes:

[0041] The corresponding colors of each of the aforementioned locations are rendered onto the environment mesh, and the environment mesh rendered with the aforementioned colors is holographically projected onto the target surface to display the coating thickness of the target surface in a mixed reality manner.

[0042] At least one of the particle parameters, the collision trajectory of the virtual sprayed particles, and the physical parameters is rendered into the generated display interface, and the display interface is holographically projected onto the spraying operation site to display the spraying operation process in a mixed reality manner.

[0043] A second aspect of this disclosure provides a mixed reality-based in-situ visualization system for shotcrete thickness, comprising:

[0044] An environmental perception module is used to construct a three-dimensional mesh representation of the spraying operation site, wherein the three-dimensional mesh representation includes the environmental mesh corresponding to the target surface to be sprayed;

[0045] Spatial registration module: Used to establish a virtual robotic arm in the three-dimensional mesh representation that corresponds to the physical robotic arm in the shotcrete operation site using spatial anchor point technology, and to make the virtual robotic arm consistent with the physical robotic arm in pose and spatial position;

[0046] Particle simulation module: used to generate virtual sprayed particles, and set particle parameters for the virtual sprayed particles based on the posture of the physical robotic arm when performing spraying operations and the spraying parameters of the robotic arm nozzles on the physical robotic arm. The particle parameters include at least the initial velocity of the particles, the gravity influence factor, and the spraying angle of the particles.

[0047] Thickness calculation module: When a collision between the virtual sprayed particle and the environmental mesh is detected, the three-dimensional coordinates of the collision point are mapped to the corresponding mesh cell in the two-dimensional height field mesh, and the current deposition thickness value of the mesh cell is obtained based on the physical parameters of the virtual sprayed particle; wherein, the two-dimensional height field mesh corresponds to the environmental mesh, and the physical parameters include at least the particle mass and adhesion probability of the virtual sprayed particle, and the adhesion probability characterizes the probability that the virtual sprayed particle adheres to the target surface;

[0048] Visualization rendering module: used to obtain the coating thickness at each location point in the grid cell based on the accumulated deposition thickness value of each grid cell, and to render the coating thickness at each location point onto the target surface using a corresponding color; wherein, coating thicknesses in different thickness ranges correspond to different colors.

[0049] By employing embodiments of this disclosure, the coating thickness at each location point in the two-dimensional height field grid can be rendered onto the target surface using corresponding colors during the shotcrete operation. This allows for a direct visual assessment of the coating thickness at each location point through color display, making the coating thickness detection results visible. Furthermore, the shotcrete operation process performed by the physical robotic arm can be reproduced in real time using environmental grids and virtual shotcrete particles, enabling the actual shotcrete operation process and its reproduction to be synchronized. This avoids the problem in related technologies where shotcrete thickness detection can only be performed after the entire construction process is completed.

[0050] On the other hand, because it can scan and reconstruct a high-precision spatial mesh model (three-dimensional mesh representation) of the real shotcrete operation site in real time, and use virtual shotcrete particles in this high-precision spatial mesh model to simulate the spraying of real physical shotcrete particles during the spraying process, the digital means are used to reproduce the spraying process of real physical shotcrete particles in the high-precision spatial mesh model. This will not damage the sprayed surface, and is not affected by ambient lighting conditions or the roughness of the target surface, which can improve its environmental adaptability and make the slurry thickness detection applicable to a wider range of scenarios.

[0051] The above description is merely an overview of the technical solution disclosed herein. In order to better understand the technical means of this disclosure and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this disclosure more apparent and understandable, specific embodiments of this disclosure are described below. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments or related technologies of this disclosure, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. It should be noted that the scale in the drawings is for illustration only and does not represent the actual scale.

[0053] Figure 1 A flowchart illustrating the steps of a mixed reality-based in-situ visualization method for shotcrete thickness is shown in this embodiment of the present disclosure.

[0054] Figure 2A , Figure 2B and Figure 3 The following are schematic diagrams illustrating application scenarios of the mixed reality-based in-situ visualization method for shotcrete thickness according to embodiments of this disclosure.

[0055] Figure 4 A flowchart illustrating the steps involved in constructing a virtual robotic arm is shown.

[0056] Figure 5 A schematic diagram of the particle parameter setting interface is shown;

[0057] Figure 6 This diagram illustrates the steps involved in determining the corresponding grid cell in a two-dimensional height field grid.

[0058] Figure 7 A schematic diagram illustrating the specific process of calculating the deposition thickness value is shown;

[0059] Figure 8 A schematic diagram of the framework of a mixed reality-based in-situ visualization system for shotcrete thickness is shown.

[0060] Figure label:

[0061] 600, Environmental grid; 700, Target surface; 601, Grid cell; 800, Display interface; 900, Physical robotic arm. Detailed Implementation

[0062] To make the above-mentioned objectives, features, and advantages of this disclosure more apparent and understandable, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0063] In related technologies, in order to improve the spraying quality of concrete or mortar materials and make the spraying thickness meet the standards, local test spraying is generally carried out by manually operating spraying equipment before construction, and the coverage effect is judged by experience. This method has the following main defects: (1) It is detached from the real environment: offline simulation relies on simplified models and cannot reflect the geometric complexity and dynamic changes of the real physical scene; (2) construction personnel cannot directly observe the spraying effect and dynamically adjust the parameters on the real work site; (3) physical test spraying and repeated corrections lead to a significant increase in material, time and labor costs.

[0064] Meanwhile, the coating thickness detection methods used in related technologies, such as core drilling, laser scanning, and ultrasonic testing, all have their own shortcomings.

[0065] In view of this, the present disclosure proposes an in-situ visualization method and system for sprayed grout thickness based on mixed reality. This method provides an innovative direction for sprayed grout simulation through mixed reality (MR) technology. First, the method utilizes the depth sensor and spatial mapping function of the mixed reality device to scan and reconstruct a high-precision spatial mesh model (three-dimensional mesh representation) of the real sprayed grout operation site in real time, providing a real physical basis for sprayed grout simulation. Then, by generating virtual sprayed grout particles, during the spraying process, the trajectory of the virtual sprayed grout particles is rendered in real time according to the spraying parameters of the nozzle on the physical robotic arm, and the deposition effect (spraying thickness) of the virtual sprayed grout particles on the target surface is obtained. The deposition effect is then rendered onto the real target surface in real time using corresponding colors.

[0066] In this way, by establishing a physical interaction model between virtual sprayed particles and the real target surface, the deposition thickness of the sprayed material at different locations on the target surface can be accurately calculated, and the thickness distribution can be displayed intuitively through color coding, enabling operators to monitor the sprayed quality in real time during construction.

[0067] Therefore, on the one hand, the spraying thickness at various points on the target surface can be rendered in real time during the spraying operation, so that construction personnel can observe the spraying situation in real time and correct any positions that do not meet the standard thickness (hereinafter referred to as the preset thickness range) in a timely manner, avoiding the problem in related technologies that the spraying thickness detection can only be carried out after the entire construction process is completed.

[0068] On the other hand, since the coating thickness at each location point in the two-dimensional height field can be rendered onto the target surface using corresponding colors, the coating thickness at each location point can be intuitively judged by displaying the colors, making the detection results of the coating thickness visible.

[0069] On the other hand, by utilizing the depth sensor and spatial mapping function of mixed reality devices, a high-precision spatial mesh model (3D mesh representation) of the real shotcrete operation site can be scanned and reconstructed in real time. In this high-precision spatial mesh model, virtual shotcrete particles are used to simulate the spraying of real physical shotcrete particles during the spraying process. This digital method allows the real physical shotcrete particle spraying process to be reproduced in the high-precision spatial mesh model without damaging the sprayed surface, and is not affected by ambient lighting conditions or the roughness of the target surface. This improves its environmental adaptability and allows the slurry thickness detection to be flexibly applied to a wider range of scenarios.

[0070] The following description, in conjunction with the accompanying drawings, provides an exemplary illustration of the in-situ visualization method and system for shotcrete thickness based on mixed reality provided in this application.

[0071] Please refer to Figure 1, Figure 2A and Figure 3 As shown, Figure 1 The diagram illustrates a step-by-step flowchart of a mixed reality-based in-situ visualization method for shotcrete thickness according to an embodiment of this disclosure. Figure 2A , Figure 2B and Figure 3 The following are schematic diagrams illustrating application scenarios of the mixed reality-based in-situ visualization method for shotcrete thickness according to embodiments of this disclosure: Figures 1-3 As shown, the in-situ visualization method for shotcrete thickness based on mixed reality in this disclosure mainly includes the following steps:

[0072] Step S100: Construct a three-dimensional mesh representation of the shotcrete operation site;

[0073] The three-dimensional mesh represents the environmental mesh corresponding to the target surface to be coated.

[0074] In this embodiment, the shotcrete operation site refers to the environment of the target surface where the shotcrete operation is performed, such as... Figure 2A As shown, the construction site for the shotcrete operation can be inside a building structure such as a house or tunnel, or it can be the environment where the workpiece to be sprayed is located.

[0075] The target surface can refer to the surface to be sprayed, which can be a flat surface, a curved surface, a spherical surface, etc. The target surface can be a metallic material, such as the surface of a metal workpiece; or it can be a non-metallic material, such as the surface of a building, such as ceramic or cement mortar. There is no limitation on the target surface here.

[0076] In some embodiments, the three-dimensional mesh representation may refer to the three-dimensional scene of the shotcrete operation site, such as the three-dimensional representation of the interior of a building structure such as a house or tunnel, which includes the environmental mesh corresponding to the target surface to be sprayed.

[0077] For example, such as Figure 2A As shown, the environmental mesh 600 can be a three-dimensional representation of the target surface 700, containing depth information of the target surface. It can be used not only to represent the two-dimensional plane of the target surface but also to represent its roughness. For example... Figure 2A and Figure 2B As shown, the environmental mesh can be a mesh composed of multiple uneven three-dimensional meshes.

[0078] Among them, such as Figure 2B As shown, the environmental grid 600 may include multiple grid cells 601 ( Figure 2B (The portion shown in the dashed box) Different mesh cells can correspond to different locations on the target surface. In some examples, the mesh cells in the environment mesh can have a different shape than the target surface. For example, as... Figure 2BAs shown, the target surface is a rectangular surface, and the mesh cells in the environment mesh can be triangular pyramidal cells.

[0079] in, Figure 2A An example is shown in the scene diagram when mixed reality is displayed. Figure 2A The example uses only a portion of the target surface's environmental mesh to provide a clear and intuitive understanding of the relationship between the environmental mesh and the target surface in mixed reality.

[0080] in, Figure 2B This diagram illustrates the relationship between the environmental mesh and the shotcrete operation site when the environmental mesh covers the entire target surface. The target surface at the shotcrete operation site corresponds to the environmental mesh. The dimensions and shapes of the multiple mesh cells within the environmental mesh may not be identical.

[0081] In some examples, when constructing a 3D mesh representation, sensors can be used to collect depth information at the shotcrete operation site and convert that depth information into a 3D mesh representation.

[0082] The sensors may include image acquisition devices, such as 3D cameras, or laser devices, without limitation.

[0083] Step S200: Using spatial anchor point technology, establish a virtual robotic arm corresponding to the physical robotic arm in the shotcrete operation site, and ensure that the virtual robotic arm and the physical robotic arm are consistent in pose and spatial position.

[0084] In this embodiment, the physical robotic arm at the shotcrete operation site refers to the robotic arm used to perform the shotcrete operation, such as... Figure 3 As shown, the physical robotic arm 900 refers to equipment that actually exists at the shotcrete operation site.

[0085] Among these methods, spatial anchoring technology can be used to create a virtual robotic arm that corresponds to the physical robotic arm. This virtual robotic arm can be understood as a mapping of the physical robotic arm in virtual space.

[0086] In this embodiment, the created virtual robotic arm can track the pose of the physical robotic arm and maintain the same pose and spatial position as the physical robotic arm. Thus, the virtual robotic arm can reproduce the pose changes of the physical robotic arm during the shotcreting operation in real time in virtual space.

[0087] Among them, pose can be understood as the motion posture of the physical robotic arm, and spatial position can be understood as the positional relationship between the physical robotic arm and the target surface.

[0088] Step S300: Generate virtual spray particles, and set particle parameters for the virtual spray particles based on the posture of the physical robotic arm during spraying operations and the spraying parameters of the robotic arm nozzles on the physical robotic arm. The particle parameters include at least the initial velocity of the particles, the gravity influence factor, and the spraying angle of the particles.

[0089] In this embodiment, the virtual sprayed particles correspond to the physical sprayed particles ejected from the nozzle of the robotic arm. The physical sprayed particles refer to the real sprayed particles that exist at the spraying operation site and are sprayed onto the target surface. The virtual sprayed particles can be understood as the mapping of the physical sprayed particles in virtual space.

[0090] like Figure 3 As shown, a robotic arm nozzle is installed on the physical robotic arm. The change in the pose of the physical robotic arm is related to the direction of the spray cone of the robotic arm nozzle. For example, the pose of the physical robotic arm determines the direction of the spray cone of the robotic arm nozzle. Therefore, in the aforementioned step S200, by binding the virtual robotic arm to the physical robotic arm, the direction of the spray cone of the robotic arm nozzle can be tracked in real time, thereby obtaining the spraying position of the sprayed particles on the target surface.

[0091] After generating virtual sprayed particles, the initial velocity, gravity influence factor, and particle spraying angle of the virtual sprayed particles can be set based on the posture of the physical robotic arm during spraying operations and the spraying parameters of the robotic arm nozzle, so as to reproduce the real spraying operation process in virtual space.

[0092] The initial velocity of the virtual sprayed particles can be determined based on the spraying parameters of the robotic arm nozzle. For example, if the spraying parameters of the robotic arm nozzle include the initial spraying velocity of the physical sprayed particles, then the initial velocity of the virtual sprayed particles is the same as the initial spraying velocity of the physical sprayed particles.

[0093] The gravity influence factor characterizes the effect of gravity on the physical sprayed particles during shotcreting operations. For example, a gravity influence factor of 1 indicates that the particles are completely affected by gravity. It is understandable that the gravity influence factor is related to the initial spraying speed; the higher the initial spraying speed, the smaller the gravity influence factor can be, indicating a weaker degree of influence of gravity on the physical sprayed particles.

[0094] The particle ejection angle can refer to the emission angle of the ejection cone of the robotic arm nozzle, which can represent the positional relationship with the target surface. The emission angle is related to the deposition position of the physical sprayed particles on the target surface.

[0095] Therefore, by setting particle parameters, virtual shotcrete particles can reproduce the spray trajectory of physical shotcrete particles in real time at the shotcrete operation site.

[0096] In this embodiment, setting a gravity influence factor can make the virtual shotcrete particles have more realistic physical effects, thereby improving the accuracy of reproducing the spray trajectory of physical shotcrete particles in the shotcrete operation site.

[0097] Step S400: When a collision between a virtual sprayed particle and the environment mesh is detected, the three-dimensional coordinates of the collision point are mapped to the corresponding mesh cell in the two-dimensional height field mesh, and the current deposition thickness value of the mesh cell is obtained based on the physical parameters of the virtual sprayed particle; wherein, the two-dimensional height field mesh corresponds to the environment mesh, and the physical parameters include at least the particle mass and adhesion probability of the virtual sprayed particle, and the adhesion probability characterizes the probability that the virtual sprayed particle adheres to the target surface.

[0098] In this embodiment, a two-dimensional height field mesh corresponding to the environmental mesh can be obtained. This two-dimensional height field mesh can be understood as a two-dimensional layer corresponding to the environmental mesh. This two-dimensional layer is used to calculate the deposition thickness value of virtual jet particles in each mesh cell of the environmental mesh. It can serve to associate the environmental mesh and the target surface. It can map the deposition position and thickness value of virtual jet particles colliding with the environmental mesh to the corresponding deposition position on the target surface.

[0099] The two-dimensional height field mesh can also include multiple mesh cells, and the mesh cells in the two-dimensional height field mesh can be determined according to the size of the target surface and the accuracy of the coating thickness detection. For example, if the accuracy requirement for coating thickness detection is high, the size of the mesh cells can be smaller, and the two-dimensional height field mesh can include a larger number of mesh cells.

[0100] The shape of the grid cells in the two-dimensional height field grid can be polygonal, which can be adapted to the shape of the grid cells in the environment grid. For example, the shape of the grid cells in the two-dimensional height field grid is triangular, and the shape of the grid cells in the environment grid is triangular pyramidal.

[0101] In this context, the size of the grid cells in the two-dimensional height field grid can be different; for example, different grid cells in the two-dimensional height field grid can have different sizes.

[0102] Here, the dimension can refer to the area of ​​the grid cell.

[0103] In this embodiment, a collision detection component can be provided, which can be used to detect collisions between virtual spray particles and the environmental mesh.

[0104] Specifically, when a collision between a virtual sprayed particle and the environment mesh is detected, the three-dimensional coordinates of the collision point can be obtained. These three-dimensional coordinates refer to the coordinates of the environment mesh in the three-dimensional mesh representation. For example, the three-dimensional coordinates can be represented as follows: .

[0105] Specifically, the three-dimensional coordinates of the collision point can be mapped to the corresponding grid cells in the two-dimensional height field grid. In this way, the grid cells corresponding to the three-dimensional coordinates in the two-dimensional height field grid can be determined based on the three-dimensional coordinates of the collision point, thereby mapping the three-dimensional coordinates of the collision point to the two-dimensional height field grid.

[0106] In this embodiment, the two-dimensional height field mesh can be associated with the environmental mesh and the target surface. Collision events detected on the environmental mesh can be mapped to the target surface in situ through the two-dimensional height field mesh, thereby establishing a mapping relationship between the environmental mesh and the target surface through the two-dimensional height field mesh, and calculating the coating thickness in the two-dimensional height field mesh.

[0107] In this process, after determining the grid cell corresponding to the three-dimensional coordinate in the two-dimensional height field grid, the current deposition thickness value of the mapped grid cell can be obtained based on the physical parameters of the virtual sprayed particles. These physical parameters may include the particle mass and adhesion probability of the virtual sprayed particles, with the adhesion probability representing the probability that the virtual sprayed particles adhere to the target surface.

[0108] The particle mass of the virtual spray particles can be related to the deposition thickness; for example, the greater the mass, the greater the contribution to the deposition thickness.

[0109] The adhesion probability of virtual sprayed particles can also be related to the deposition thickness; the higher the adhesion probability, the greater its contribution to the deposition thickness.

[0110] Therefore, in this embodiment, the current deposition thickness value of the grid cell in the two-dimensional height field grid can be determined at least based on the particle mass of the virtual jet particles and the adhesion probability of the virtual jet particles.

[0111] In this process, the current deposition thickness value of a grid cell in the two-dimensional height field grid is continuously accumulated. For example, when a virtual jet particle is detected colliding with the environmental grid for the first time, the current deposition thickness value of the grid cell can be calculated. When a virtual jet particle is detected colliding with the environmental grid for the second time, the second calculated deposition thickness value can be added to the first deposition thickness value of the grid cell, thus obtaining the second current deposition thickness value of the grid cell. This process continues until the current deposition thickness value of the grid cell is obtained after continuous accumulation.

[0112] In this embodiment, the adhesion probability and particle mass of the virtual sprayed particles are consistent with the adhesion probability and particle mass of the physical sprayed particles ejected by the robotic arm nozzle.

[0113] In this embodiment, during the process of the physical robotic arm performing the shotcrete operation, the deposition thickness value corresponding to each grid cell in the two-dimensional height field grid can be continuously updated.

[0114] In some examples, the deposition thickness value corresponding to each grid cell in the two-dimensional height field grid can refer to the deposition thickness value at the endpoint of the grid cell. In this way, when calculating the deposition thickness value, it can be used to calculate the deposition thickness value at the edge of the grid cell, thereby reducing the amount of computation.

[0115] Step S500: Based on the accumulated deposition thickness value of each grid cell, obtain the coating thickness at each location point in the grid cell, and render the coating thickness at each location point onto the target surface using the corresponding color; wherein, the coating thickness in different thickness ranges corresponds to different colors.

[0116] In this embodiment, each grid cell in the two-dimensional height field grid may include multiple location points, which may correspond to multiple different locations in a certain location region of the target surface.

[0117] In the process of the physical robotic arm performing the spraying operation, the spraying thickness at each position point in the grid cell can be determined based on the deposition thickness value corresponding to the grid cell in the two-dimensional height field grid.

[0118] As described in the previous embodiments, the deposition thickness value corresponding to each grid cell can be the deposition thickness value at the endpoint of the grid cell. In this way, the coating thickness at each location point in the grid cell can be calculated based on the deposition thickness value at the endpoint of the grid cell, such as by performing linear or nonlinear interpolation on the deposition thickness value at the endpoint of the grid cell.

[0119] After obtaining the coating thickness at each location point within the grid cell, the coating thickness at each location point can be rendered onto the target surface using the corresponding color, thereby achieving a mixed reality display effect. In this way, the coating thickness of the physical spray particles on the target surface can be displayed in a more intuitive and visual way.

[0120] In this system, the position points in the two-dimensional height field grid correspond one-to-one with the position points on the target surface. In this way, the spraying thickness of the virtual spray particles at the corresponding position points of the grid cells can be displayed in situ at the deposition position of the physical spray particles on the target surface, thereby making the spray thickness visualized in situ.

[0121] For example, such as Figure 3The image shown depicts the view from a worker's perspective at the spraying operation site. In this image, in addition to the building surface at the spraying operation site, the colors rendered by steps S100-S500 above are also visible on the building surface. As a result, the construction worker does not need to leave the spraying operation site, nor does he need to return to the electronic equipment to observe the spray thickness. That is, the construction worker does not need to switch between the building surface and the electronic equipment. Therefore, it does not affect the construction worker's work, but allows the construction worker to observe the spray thickness in real time and adjust the construction parameters such as nozzle distance and pressure in real time according to the spray thickness to correct the spray thickness in areas where the thickness does not meet the standard.

[0122] In this embodiment, different coating thicknesses correspond to different colors. For example, the coating thickness in the first thickness range corresponds to dark blue, the coating thickness in the second thickness range corresponds to light blue, the coating thickness in the third thickness range corresponds to light purple, the coating thickness in the fourth thickness range corresponds to dark purple, and the coating thickness in the fifth thickness range corresponds to red.

[0123] Among them, the first thickness range is smaller than the second thickness range, the second thickness range is smaller than the third thickness range, the third thickness range is smaller than the fourth thickness range, and the fourth thickness range is smaller than the fifth thickness range.

[0124] In this way, the thinnest point corresponds to dark blue, and the thickest point corresponds to red. Thus, through the gradient of dark blue-light blue-light purple-dark purple-red, the coating thickness is converted into a heat map representation, thereby improving the intuitiveness of thickness observation.

[0125] The in-situ visualization method for sprayed grout thickness in this embodiment allows for real-time rendering of the grout thickness at various points within a two-dimensional height field grid onto the target surface using corresponding colors during the grouting operation. This enables intuitive judgment of the grout thickness at each point through color display, thus visualizing the grout thickness detection results. Furthermore, the grouting operation process performed by the physical robotic arm can be reproduced in real-time using environmental grids and virtual grout particles, allowing the actual grouting operation and its reproduction to proceed synchronously. This allows construction personnel to observe the grout thickness on the target surface in real-time during the grouting operation, avoiding the problem in related technologies where grout thickness detection can only be performed after the entire construction process is completed.

[0126] On the other hand, by utilizing the depth sensor and spatial mapping function of mixed reality devices, a high-precision spatial mesh model (three-dimensional mesh representation) of the real shotcrete operation site can be scanned and reconstructed in real time. In this high-precision spatial mesh model, virtual shotcrete particles are used to simulate the spraying of real physical shotcrete particles during the spraying process. This digital method allows the real physical shotcrete particle spraying process to be reproduced in the high-precision spatial mesh model without damaging the sprayed surface, and is not affected by ambient lighting conditions or the roughness of the target surface. This improves its environmental adaptability and allows the slurry thickness detection to be applied to a wider range of scenarios.

[0127] In some embodiments, in step S100 above, the spatial mapping component of the mixed reality device can be used to automatically scan the depth information of the shotcrete operation site and generate a three-dimensional mesh representation based on the depth information; and a collision body component can be added to the three-dimensional mesh representation; wherein the collision body component is used to perform collision detection of the environmental mesh.

[0128] Specifically, in the Unity development environment, the spatial mapping component IMixedRealitySpatialAwarenessMeshObserver of the device can be called. This spatial mapping component will automatically scan the surrounding environment of the shotcrete operation site and generate a geometric mesh. The scanned mesh is enumerated through the API, and the MeshFilter data is serialized into a model file type to obtain the above-mentioned three-dimensional mesh representation.

[0129] In this regard, the scanned 3D mesh representation can be represented by a Mesh Collider component in Unity, which enables it to perform collision detection with virtual jet particles.

[0130] Unity is a real-time 3D interactive content creation and operation platform, enabling creators in fields such as game development, art, architecture, automotive design, and film to turn their ideas into reality. Unity provides a complete software solution for creating, operating, and monetizing any real-time interactive 2D and 3D content, supporting platforms including mobile phones, tablets, PCs, game consoles, augmented reality, and virtual reality devices.

[0131] Among them, IMixedRealitySpatialAwarenessMeshObserver is a core component of the Microsoft MixedReality Toolkit (MRTK) for spatially aware mesh observation. It is mainly responsible for acquiring spatial mesh data in the environment in real time and providing interactive capabilities. Its core functions mainly include: real-time spatial awareness, collision detection, and dynamic interaction.

[0132] Real-time spatial perception uses IMU (Inertial Measurement Unit) and VIO (Visual Inertial Odometry) technologies to track the device's position and attitude, constructing an environmental spatial grid model.

[0133] Collision detection supports raycasting, camera collision detection, and object interaction responses, such as triggering events (e.g., triggering OnTriggerEnter / OnTriggerExit events when a ray enters / exits an object's area).

[0134] Dynamic interaction can add physical properties (such as material and mass) to mesh objects, enabling more realistic interactive feedback.

[0135] In some embodiments, such as Figure 4 As shown, Figure 4 A flowchart illustrating the steps involved in constructing a virtual robotic arm is shown, as follows: Figure 4 As shown, step S200 may include the following steps:

[0136] Step S201: Construct the initial virtual robotic arm and set a virtual QR code for the virtual robotic arm; wherein, the position of the virtual QR code on the virtual robotic arm is consistent with the position of the physical QR code on the physical robotic arm.

[0137] Step S202: Detect the position of the physical QR code in real time using an image acquisition device, and adjust the pose of the virtual robotic arm in the 3D mesh representation according to the real-time position until the real-time poses of the virtual QR code and the physical QR code coincide in the mixed reality view, so that the virtual robotic arm is aligned with the physical robotic arm.

[0138] Step S203: Create a persistent spatial anchor point for the alignment position when the virtual robotic arm is aligned with the physical robotic arm, and bind the virtual robotic arm to the spatial anchor point to lock and maintain the alignment relationship between the virtual robotic arm and the physical robotic arm.

[0139] In this embodiment, a physical QR code can be affixed to a fixed reference position of the physical robotic arm as an identifier for the physical robotic arm.

[0140] In the Unity editor, you can create a virtual QR code model, set it as a sub-object of the virtual robotic arm, and precisely adjust its local position and posture to make it completely consistent with the relative position and orientation of the physical QR code on the physical robotic arm.

[0141] The image acquisition device can be a camera, and the location of the camera is not limited, as long as it can capture images of the physical robotic arm and the physical QR code affixed to the physical robotic arm.

[0142] In this embodiment, the image of the physical QR code can be captured in real time by a camera, and the pose of the physical robotic arm can be obtained based on the captured image of the physical QR code. This allows the pose of the virtual robotic arm to be automatically adjusted in the virtual space, so that the real-time pose of the virtual QR code and the physical QR code detected by the camera are superimposed in the mixed reality view, thereby achieving precise alignment between the virtual robotic arm and the physical robotic arm.

[0143] For example, in Unity, the ARMarkerManager component can be used to scan using the device's camera. When a physical QR code matching a preset pattern is detected, the markersChanged event is triggered. By subscribing to this event, the newly detected ARMarker object can be obtained in the callback function of the C# script. The position and pose (transform.position and transform.rotation) of this ARMarker object represent the precise pose of the physical QR code in Unity's world space. By obtaining this pose and adjusting the pose of the virtual robotic arm's root object in real time, the pose of the virtual QR code coincides with the pose of the ARMarker, thus achieving automatic alignment between the virtual and physical robotic arms.

[0144] In this embodiment, after the virtual robotic arm is aligned with the physical robotic arm, a persistent spatial anchor point can be created at the current alignment position, and the virtual robotic arm model can be bound to the spatial anchor point to lock and maintain the established virtual-physical spatial registration relationship, thereby achieving stable tracking without relying on continuous QR code (physical QR code) recognition.

[0145] For example, after aligning the virtual robotic arm with the physical robotic arm, a WorldAnchor component can be added to the virtual robotic arm to lock it. The WorldAnchorStore.GetAsync() asynchronous method is called to obtain an instance of WorldAnchorStore in the world anchor storage area on the device. Its Save() method is then called to save the spatial anchor information of the current alignment position to the device (i.e., to create a persistent spatial anchor at the current alignment position), thereby realizing the persistence of the registration relationship between the physical robotic arm and the virtual robotic arm.

[0146] This method allows for stable tracking without relying on continuous recognition of physical QR codes.

[0147] In some embodiments, in step S300, a particle system can be created in Unity and bound to a virtual model of the robotic arm nozzle; the emission rate, initial velocity, and emission angle of the virtual sprayed particles are set according to the real-time position and spraying parameters of the physical robotic arm when performing spraying operations; and gravity, air resistance, lifetime, and rebound coefficient are added to the virtual sprayed particles to simulate the physical properties of real sprayed materials; wherein, the lifetime characterizes the disappearance time of virtual sprayed particles that have not collided; the rebound coefficient is negatively correlated with the adhesion coefficient of the virtual sprayed particles.

[0148] In this embodiment, the emission rate, initial velocity, and emission angle of the virtual sprayed particles can be dynamically set according to the real-time position and spraying parameters of the physical robotic arm during the spraying operation. When the real-time position and spraying parameters of the physical robotic arm during the spraying operation change, the emission rate, initial velocity, and emission angle of the virtual sprayed particles can be set according to the changed real-time position and spraying parameters.

[0149] The virtual jet particles' emission rate, initial velocity, and emission angle can be kept consistent with those of the physical jet particles ejected from the robotic arm's nozzle, in order to simulate the real jet trajectory of the physical jet particles.

[0150] In this embodiment, please refer to Figure 5 As shown, Figure 5 A schematic diagram of the particle parameter setting interface is shown, such as... Figure 5 As shown, the emission rate can be set in the Emission module of the particle system, the angle and radius of the emission cone can be set in the Shape module, and the initial velocity can be set in the main module of the particle system.

[0151] In this embodiment, to simulate the physical effects of real-world jet particles, a Gravity Modifier can be enabled in the main module of the particle system, and an appropriate value can be set (e.g., 1.0 indicates complete influence by gravity). Simultaneously, Drag can be enabled in the Limit Velocity over Lifetime module to simulate the velocity decay of jet particles as they fly through the air.

[0152] In this embodiment, a Start Lifetime can be set in the main module of the particle system to ensure that particles that do not collide disappear after a certain period of time (i.e., the disappearance time of virtual spray particles that do not collide), thereby saving computational resources. In the Collision module, the type is set to World, and the Bounce coefficient is set to a very small value (such as 0.1) to simulate the physical characteristic that most of the spray material adheres to the target surface rather than bounces.

[0153] Among them, the rebound coefficient is negatively correlated with the adhesion coefficient of the virtual sprayed particles. That is, the larger the rebound coefficient, the smaller the adhesion coefficient of the virtual sprayed particles, and the less likely the sprayed particles are to adhere to the target surface; the smaller the rebound coefficient, the larger the adhesion coefficient of the virtual sprayed particles, and the easier the sprayed particles are to adhere to the target surface.

[0154] Therefore, through the above settings, the physical properties of the virtual sprayed particles can be made consistent with those of the physical sprayed particles, thereby making the spray trajectory of the virtual sprayed particles highly consistent with that of the physical sprayed particles. This allows for the accurate reproduction of the spray trajectory of the physical sprayed particles in virtual space, improving the accuracy of spray thickness detection.

[0155] The gravity, air resistance, lifespan, and rebound coefficient of the virtual sprayed particles can be determined based on the gravity, air resistance, lifespan, and rebound coefficient faced by the physical sprayed particles. For example, in practice, the above physical characteristics of the physical sprayed particles can be detected and set as the physical characteristics of the virtual sprayed particles.

[0156] In step S400, each time a virtual sprayed particle is detected to collide with the environmental mesh, the three-dimensional coordinates of the collision point are mapped to the corresponding mesh cell in the two-dimensional height field mesh. In some embodiments, this can be done according to... Figure 6 The steps shown determine the corresponding grid cells in the two-dimensional height field grid.

[0157] Step S401: Establish a two-dimensional height field data structure, obtain a two-dimensional height field mesh, and divide the two-dimensional height field mesh into multiple mesh cells; wherein, the two-dimensional height field data structure is represented as follows: ;in, , The horizontal coordinate is used.

[0158] In the C# script, a two-dimensional floating-point array can be created as a two-dimensional height field grid to store thickness data. The array dimension and coverage of this data structure are determined according to the actual working area. The two-dimensional height field grid can be divided into multiple grid cells. The size of the multiple grid cells can not be exactly the same, and the shape type of the multiple grid cells can be the same.

[0159] As described in the previous embodiments, a collider component can be added to the 3D mesh representation to perform collision detection of the environment mesh. In this embodiment, an OnParticleCollision(GameObject other) callback function can be created in a C# script to respond to collision events between virtual spray particles and the environment mesh;

[0160] Next, you can use particleSystem.GetCollisionEvents(other,collisionEvents) in the callback function to get all collision events.

[0161] Step S402: When a collision between a virtual sprayed particle and the environment mesh is detected, extract the three-dimensional coordinates of the collision point; wherein, the three-dimensional coordinates are represented as... .

[0162] Step S403: Convert the three-dimensional coordinates to relative coordinates relative to the origin of the two-dimensional height field grid. The relative coordinates are expressed as follows: .

[0163] Step S404: Based on the size and relative coordinates of the mesh cells, obtain the mesh cells to which the three-dimensional coordinates are mapped in multiple two-dimensional height field meshes.

[0164] Specifically, for three-dimensional coordinates, the corresponding components in the relative coordinate vector can be selected according to the preset mesh orientation. These two components are then divided by the preset mesh cell size to convert them into indices for the two-dimensional height field mesh. This allows us to obtain the mesh cells to which the three-dimensional coordinates are mapped.

[0165] Specifically, it can be based on three-dimensional coordinates Obtain the collision point relative to the grid origin. relative coordinates Among them, the grid origin It can refer to the geometric center of the grid cell where the collision point is located in the environmental grid.

[0166] Next, based on the size and relative coordinates of the mesh cells, the mesh cells to which the three-dimensional coordinates are mapped can be obtained in multiple two-dimensional height field meshes according to the following formula (1):

[0167] ;

[0168] Formula (1);

[0169] In formula (1), i and j represent the indices of the two-dimensional height field grid, for example, i represents the grid cell in the i-th row and j-th column.

[0170] In formula (1), cell Size represents the size of the grid cell, which can refer to the area of ​​the grid cell.

[0171] In this embodiment, in step S400, when obtaining the current deposition thickness value of the grid cell based on the physical parameters of the virtual sprayed particles, the deposition contribution can be obtained according to the following formula (2):

[0172] Formula (2);

[0173] In formula (2), The mass of the particle is... The adhesion probability is... The time factor represents the duration for which the robotic arm nozzle continuously sprays paint in the same pose.

[0174] In this embodiment, a time factor can be set in the particle system, or a time factor can be introduced separately when calculating the deposition contribution.

[0175] Among them, the time factor can characterize the duration for which physical jet particles continuously act on the same location.

[0176] In this context, "×" represents a multiplication operation.

[0177] Accordingly, the current deposition thickness of a grid cell can be obtained based on the thickness value and deposition contribution of the grid cell before the current collision.

[0178] For example, the current deposition thickness value of the grid cell can be obtained according to the following formula (3):

[0179] H[i,j]=h[i,j]+H formula (3)

[0180] In formula (3), H is the deposition contribution of a single particle, H[i,j] represents the current deposition thickness of the grid cell, and h[i,j] represents the deposition thickness of the grid cell before the current collision.

[0181] In step S500, based on the accumulated deposition thickness value of each grid cell, the spraying thickness at each location point in the grid cell needs to be obtained, and the spraying thickness at each location point is rendered onto the target surface using a corresponding color. In some embodiments, this can be combined with... Figure 7 As shown, Figure 7 A schematic diagram of the specific process of step S500 is shown, as follows: Figure 7 As shown, the specific steps may include:

[0182] Step S501: Traverse the deposition thickness values ​​of multiple grid cells to obtain the maximum and minimum deposition thickness values.

[0183] In this embodiment, the maximum deposition thickness value refers to the deposition thickness value corresponding to the grid cell with the largest deposition thickness value among multiple grid cells.

[0184] The minimum deposition thickness value refers to the deposition thickness value corresponding to the grid cell with the smallest deposition thickness value among multiple grid cells.

[0185] Step S502: Based on the maximum and minimum deposition thickness values, normalize the deposition thickness values ​​of multiple grid cells to obtain the normalized thickness value of each grid cell.

[0186] In this embodiment, when normalizing the deposition thickness values ​​of multiple grid cells based on the maximum and minimum deposition thickness values, the maximum deposition thickness value can be normalized to 1, the minimum deposition thickness value can be normalized to 0, and the deposition thickness value of the grid cell can be normalized to a value between 0 and 1 based on the deposition thickness value of the grid cell and the difference between the maximum and minimum deposition thickness values, thereby obtaining the normalized thickness value of the grid cell.

[0187] For example, assuming the maximum deposition thickness is 50 mm and the minimum deposition thickness is 10 mm, with 50 mm normalized to 1 and 10 mm normalized to 0, the 40 mm thickness difference between the maximum and minimum deposition thicknesses is converted into a difference of 1. Therefore, each 1 mm corresponds to a step size of 0.025 in the (0,1) range. Assuming the current grid's deposition thickness is 12 mm, the normalized thickness value is 0.05.

[0188] Of course, there are other ways to perform normalization, which will not be elaborated here.

[0189] Step S503: Based on the normalized thickness value of the mesh cell, obtain the coating thickness at each location point in the mesh cell.

[0190] In this embodiment, after normalizing the deposition thickness value of each grid cell, the deposition thickness value of each grid cell can be calibrated to the same numerical range, so that the deposition thickness of multiple grid cells can be compared at the same scale, thereby improving the accuracy of subsequent color rendering.

[0191] In this embodiment, the deposition thickness value of the grid cell refers to the deposition thickness value at the endpoint of the grid cell. In practice, in order to improve the fine detection of the coating thickness, the coating thickness at each position point in the grid cell can be obtained based on the normalized thickness value at the endpoint of the grid cell.

[0192] Since a mesh cell contains multiple location points—specifically, in a virtual scene, a mesh cell typically contains multiple pixels, and each pixel can be considered a location point—the normalized coating thickness of each pixel within the mesh cell can be obtained based on the normalized thickness values ​​at the endpoints of the mesh cell. Then, color coding is applied to the corresponding location points based on their normalized coating thicknesses.

[0193] For example, a normalized thickness value of 0 corresponds to pure blue (RGB: 0, 0, 255), representing the thinnest region. A normalized thickness value of 1 corresponds to pure red (RGB: 255, 0, 0), representing the thickest region. For any normalized thickness value between 0 and 1, the corresponding color will be obtained by linear interpolation between blue and red. This means that as the thickness gradually increases, the color will smoothly transition from blue to purple, magenta, and finally to red.

[0194] In some examples, the coating thickness at each location point in the grid cell can be obtained by linear interpolation between the maximum and minimum deposition thickness values ​​based on the normalized thickness values ​​at each endpoint of the grid cell.

[0195] In this embodiment, the normalized thickness value t of all endpoints of the mesh cell can be passed to a custom shader, and a color mapping function can be implemented in the fragment shading function of the shader. This function calculates the color that each pixel in the grid cell should display based on the input normalized thickness value t, by linearly interpolating between the maximum and minimum deposition thickness values. In other words, it calculates the normalized coating thickness at each location point and then obtains the display color of each location point based on the normalized coating thickness.

[0196] Specifically, based on the input normalized thickness value t, linear interpolation is performed between the normalized value 1 corresponding to the maximum deposition thickness value and the normalized value 0 corresponding to the minimum deposition thickness value.

[0197] For example, the grid cell includes three endpoints, and the normalized thickness values ​​t of the three endpoints are assumed to be 0.05, 0.075 and 0.05, respectively. The maximum deposition thickness is 50 mm and the minimum deposition thickness is 10 mm, and their corresponding normalized values ​​are 1 and 0, respectively. Then, linear interpolation can be performed using 0.05, 0.075, 0.05, 1 and 0 to obtain the normalized coating thickness at each location point in the grid cell.

[0198] For example, a grid cell may contain 10 pixels, which means it contains 10 location points. The normalized coating thickness of these 10 location points can be obtained by linear interpolation using 0.05, 0.075, 0.05, 0, and 1, respectively, resulting in coating thicknesses of 0.06, 0.065, 0.055, etc.

[0199] For example, taking a triangle as the mesh cell, the shader can use the values ​​of the three endpoints to perform linear interpolation from blue to red.

[0200] For example, if a mesh cell contains 10 pixels, then for each of these 10 pixels, the shader calculates a precise normalized spray thickness based on its relative position within the mesh cell. Suppose a pixel has a calculated normalized spray thickness of 0.2, then its color will be blue. Another pixel has a calculated normalized spray thickness of 0.5, and its color will be closer to purple, falling within the medium thickness range. Yet another pixel has a calculated normalized spray thickness of, say 0.8, and its color will be close to red, indicating a larger thickness.

[0201] Therefore, the shader can render the corresponding color representation for each location point based on the normalized spray thickness at each location point. This process can be called the color encoding process.

[0202] The linear interpolation process in this embodiment can be referred to in the linear interpolation of related technologies, and will not be elaborated here.

[0203] Step S504: For the difference between the coating thickness and the target deposition thickness value at each location point, render the coating thickness at that location point onto the target surface using the corresponding color;

[0204] The target deposition thickness value includes the maximum deposition thickness value and the minimum deposition thickness value; or, the target deposition thickness value is a preset thickness range.

[0205] In this embodiment, after obtaining the coating thickness at each location point, the color corresponding to that location point can be determined based on the difference between the coating thickness at that location point and the target deposition thickness value, and that color can be rendered onto the target surface.

[0206] In some examples, the target deposition thickness value can include a maximum deposition thickness value and a minimum deposition thickness value. The maximum deposition thickness value can be set to red, and the minimum deposition thickness value to blue. The coating thickness at a given location is typically between the maximum and minimum deposition thickness values, and its color can be a mixture of red and blue in appropriate proportions, such as a gradient of purple.

[0207] The difference between the spray thickness at a location point and the target deposition thickness value can include a first difference between it and the maximum deposition thickness value, and a second difference between it and the minimum deposition thickness value. Based on the first and second differences, the color ratio between blue and red can be determined, thereby obtaining the color corresponding to the location point.

[0208] The ratio between red (the color corresponding to the maximum deposition thickness) and blue (the color corresponding to the minimum deposition thickness) can be the inverse ratio of the first difference to the second difference. For example, if the first difference is greater than the second difference, indicating that the coating thickness at the location point is closer to the minimum deposition thickness, then its color can be closer to blue. By increasing the blue ratio and decreasing the red ratio, the color of the location point can be light purple.

[0209] Of course, in some examples, the target deposition thickness value may include a preset thickness range, the color corresponding to the preset thickness range may be green, the color for a thickness less than the preset thickness range may be blue, and the color for a thickness greater than the preset thickness range may be red.

[0210] The preset thickness range can refer to the standard deposition thickness of the coating material sprayed on the target surface; the specific thickness can be determined according to the coating requirements of the target surface, and is not limited here.

[0211] In some examples, the colors of each location point can be rendered into a two-dimensional height field mesh, and the colored two-dimensional height field mesh can be projected onto the target surface using holographic projection technology.

[0212] In some examples, the colors of each location point can be rendered onto the surface of an environment mesh, and then the rendered colored environment mesh can be projected onto the target surface using holographic projection technology. This makes the rendered colors appear more three-dimensional on the target surface, matching the surface roughness of the target surface, achieving a three-dimensional viewing effect, and thus optimizing the display effect of mixed reality.

[0213] In some embodiments, such as Figure 3 As shown, at least one of the particle parameters, the collision trajectory of the virtual sprayed particles, and the physical parameters can be rendered into the generated display interface 800, and the display interface 800 can be holographically projected onto the spraying operation site to display the spraying operation process in a mixed reality manner.

[0214] For example, the display interface 800 can be understood as a virtual display interface at the shotcrete operation site. This virtual display interface includes multiple windows, and particle parameters, the collision trajectory of virtual shotcrete particles, and physical parameters can be displayed in different windows respectively. Then, holographic projection technology can be used to project the display interface onto the shotcrete operation site, thus realizing the display of a mixed reality operation interface.

[0215] like Figure 3 As shown, from the perspective of a construction worker, in addition to seeing the color representing the thickness rendered on the wall, they can also see the projected display interface 800. Different windows in the display interface can display particle parameters, the collision trajectory of virtual sprayed particles, and physical parameters.

[0216] In some examples, gesture control functionality can also be provided, for example, construction workers can perform gesture control on a display interface shown at the shotcrete operation site.

[0217] For example, image acquisition equipment can be used to capture the gestures of construction workers at the shotcrete operation site, and the captured gestures can be recognized. Based on the recognized gestures, the specific operation to be performed by the construction worker can be determined. For example, whether it is a click operation, a swipe operation, etc., thereby controlling the electronic device to execute the control logic corresponding to the operation, so that the view displayed on the shotcrete operation site changes according to the operation. In this way, not only can the visualization of shotcrete thickness detection be satisfied, but construction workers can also set and view the physical parameters and particle parameters of virtual shotcrete particles without relying on input devices.

[0218] In summary, the embodiments described above have the following advantages and positive effects:

[0219] (1) By utilizing mixed reality spatial perception and height field thickness calculation technology, the virtual shotcrete thickness and the real physical environment were accurately integrated. By generating a three-dimensional mesh representation through real-time environmental scanning and establishing a collision detection system, the deposition process of shotcrete particles and the thickness accumulation effect strictly follow the physical laws of the real shotcrete process. Based on the two-dimensional height field mesh, the cumulative thickness value of each mesh position was accurately recorded. This method avoids the problems of destructive sampling and delayed feedback in traditional thickness detection methods, ensuring the real-time nature of thickness calculation and the accuracy of construction quality monitoring, while improving the operator's ability to predict and control the shotcrete coverage effect.

[0220] (2) Through dynamic color mapping and in-situ heatmap rendering technology, efficient and intuitive thickness visualization performance is achieved. The height field data is updated in real time based on particle collision events, and the spraying thickness is converted into an intuitive heatmap display through a color gradient algorithm; combined with a threshold warning mechanism and dynamic rendering optimization, the system maintains a stable visualization effect when running on mixed reality devices. This efficient visualization method not only reduces the cognitive load of construction personnel, but also significantly improves the efficiency of quality monitoring in complex construction environments, and is especially suitable for in-situ operation scenarios with complex lighting conditions such as tunnels and foundation pits.

[0221] (3) It has the advantages of real-time thickness monitoring and in-situ decision support, realizing preventive control and dynamic parameter adjustment of shotcrete quality. By recording the collision position, deposition thickness and time data of virtual shotcrete particles, a dynamic thickness distribution map covering the target surface is generated. Construction personnel can identify areas with insufficient or excessive shotcrete thickness in real time and adjust construction parameters such as nozzle distance and pressure in a timely manner. This method transforms traditional post-event quality inspection into process-controllable real-time monitoring, reducing material waste and rework costs, and shortening the time for discovering construction quality problems from hours to seconds.

[0222] In summary, this disclosure presents a high-precision in-situ shotcrete thickness monitoring system through the deep integration of mixed reality technology, real-time thickness calculation, and intelligent visualization. It overcomes the technical challenges of real-time shotcrete thickness calculation and in-situ visualization, significantly improving the scientific rigor and operability of construction quality control. In fields such as tunnel support and foundation pit reinforcement, construction personnel can monitor shotcrete thickness distribution in situ and optimize construction processes using mixed reality equipment without relying on traditional core drilling or offline testing equipment. This technology reduces engineering quality risks, improves construction efficiency in complex environments, and provides a reusable technical framework for intelligent construction and digital construction, accelerating the development of the engineering construction field towards refinement and intelligence.

[0223] Based on the same inventive concept, a mixed reality-based in-situ visualization system for shotcrete thickness is also provided, such as... Figure 8 As shown, Figure 8 A schematic diagram of the framework of a mixed reality-based in-situ visualization system for shotcrete thickness is shown, such as... Figure 8 As shown, the system mainly includes the following modules:

[0224] An environmental perception module is used to construct a three-dimensional mesh representation of the spraying operation site, wherein the three-dimensional mesh representation includes the environmental mesh corresponding to the target surface to be sprayed;

[0225] Spatial registration module: Used to establish a virtual robotic arm in the three-dimensional mesh representation that corresponds to the physical robotic arm in the shotcrete operation site using spatial anchor point technology, and to make the virtual robotic arm consistent with the physical robotic arm in pose and spatial position;

[0226] Particle simulation module: used to generate virtual sprayed particles, and set particle parameters for the virtual sprayed particles based on the posture of the physical robotic arm when performing spraying operations and the spraying parameters of the robotic arm nozzles on the physical robotic arm. The particle parameters include at least the initial velocity of the particles, the gravity influence factor, and the spraying angle of the particles.

[0227] Thickness calculation module: When a collision between the virtual sprayed particle and the environmental mesh is detected, the three-dimensional coordinates of the collision point are mapped to the corresponding mesh cell in the two-dimensional height field mesh, and the current deposition thickness value of the mesh cell is obtained based on the physical parameters of the virtual sprayed particle; wherein, the two-dimensional height field mesh corresponds to the environmental mesh, and the physical parameters include at least the particle mass and adhesion probability of the virtual sprayed particle, and the adhesion probability characterizes the probability that the virtual sprayed particle adheres to the target surface;

[0228] Visualization rendering module: used to obtain the coating thickness at each location point in the environment grid based on the accumulated deposition thickness value of each grid cell, and to render the coating thickness at each location point to the environment grid with a corresponding color; wherein, coating thicknesses in different thickness ranges correspond to different colors.

[0229] For example, the environment sensing module includes:

[0230] The scanning unit is used to automatically scan the depth information of the shotcrete operation site using the spatial mapping component of the mixed reality device, and generate the three-dimensional mesh representation based on the depth information;

[0231] A component adding unit is used to add a collider component to the three-dimensional mesh representation; wherein the collider component is used to perform collision detection on the environment mesh.

[0232] For example, the spatial registration module includes:

[0233] A QR code setting unit is used to construct an initial virtual robotic arm and set a virtual QR code for the virtual robotic arm; wherein the position of the virtual QR code on the virtual robotic arm is consistent with the position of the physical QR code on the physical robotic arm.

[0234] The pose alignment unit is used to detect the position of the physical QR code in real time through an image acquisition device, and adjust the pose of the virtual robotic arm in the three-dimensional mesh representation according to the real-time position until the real-time poses of the virtual QR code and the physical QR code coincide in the mixed reality view, so that the virtual robotic arm is aligned with the physical robotic arm.

[0235] Anchor point binding unit is used to create a persistent spatial anchor point for the alignment position when the virtual robotic arm is aligned with the physical robotic arm, and bind the virtual robotic arm to the spatial anchor point to lock and maintain the alignment relationship between the virtual robotic arm and the physical robotic arm.

[0236] For example, the particle simulation module includes:

[0237] A particle generation unit is used to create a particle system in Unity and bind the particle system to a virtual model of the robotic arm nozzle;

[0238] The first setting unit is used to set the emission rate, initial velocity, and emission angle of the virtual sprayed particles based on the real-time position of the physical robotic arm during the spraying operation and the spraying parameters.

[0239] The second setting unit is used to add gravity, air resistance, life cycle and rebound coefficient to the virtual sprayed particles to simulate the physical properties of real sprayed materials.

[0240] The lifecycle characterizes the time it takes for virtual jet particles that have not collided to disappear; the rebound coefficient is negatively correlated with the adhesion coefficient of the virtual jet particles.

[0241] For example, the thickness calculation module includes a position mapping submodule, which includes:

[0242] A data structure generation unit is used to establish a two-dimensional height field data structure, obtain the two-dimensional height field grid, and divide the two-dimensional height field grid to obtain multiple grid cells;

[0243] The coordinate acquisition unit is used to extract the three-dimensional coordinates of the collision point each time a collision between the virtual sprayed particle and the environmental mesh is detected;

[0244] A coordinate transformation unit is used to convert the three-dimensional coordinates into relative coordinates relative to the origin of the two-dimensional height field grid;

[0245] A coordinate mapping unit is used to obtain the grid cell to which the three-dimensional coordinates are mapped in a plurality of two-dimensional height field grids, based on the size of the grid cell and the relative coordinates.

[0246] For example, the thickness calculation module includes a thickness calculation submodule, which is specifically used to: obtain the deposition contribution according to the following formula;

[0247] ;in, The mass of the particle is... The adhesion probability is... The time factor represents the duration of continuous spraying by the robotic arm nozzle in the same pose; and the current deposition thickness value of the grid cell is obtained based on the thickness value deposited by the grid cell before the current collision and the deposition contribution.

[0248] For example, the visualization rendering module includes:

[0249] The traversal cell is used to traverse the deposition thickness values ​​of multiple grid cells to obtain the maximum and minimum deposition thickness values;

[0250] A normalization unit is used to normalize the deposition thickness values ​​of multiple grid cells based on the maximum deposition thickness value and the minimum deposition thickness value, so as to obtain the normalized thickness value of each grid cell.

[0251] The position point thickness acquisition unit is used to acquire the coating thickness of each position point in the grid cell based on the normalized thickness value of the grid cell.

[0252] The rendering unit is used to render the coating thickness at each location point onto the target surface using a corresponding color, based on the difference between the coating thickness at each location point and the target deposition thickness value.

[0253] The target deposition thickness value includes the maximum deposition thickness value and the minimum deposition thickness value; or, the target deposition thickness value is a preset thickness range.

[0254] For example, the mesh cell is a polygon, and the normalized thickness value of the mesh cell includes the normalized thickness values ​​of each endpoint of the mesh cell; the location point thickness acquisition unit is specifically used to: perform linear interpolation between the maximum deposition thickness value and the minimum deposition thickness value based on the normalized thickness values ​​of each endpoint of the mesh cell to obtain the coating thickness at each location point in the mesh cell.

[0255] For example, the rendering unit includes:

[0256] The first rendering subunit is used to render the corresponding colors of each of the said locations to the environment mesh, and to holographically project the environment mesh with the rendered colors onto the target surface, so as to display the coating thickness of the target surface in a mixed reality manner.

[0257] The second rendering subunit is used to render at least one of the particle parameters, the collision trajectory of the virtual sprayed particles, and the physical parameters onto the generated display interface, and to holographically project the display interface onto the spraying operation site to display the spraying operation process in a mixed reality manner.

[0258] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0259] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0260] The above provides a detailed description of a mixed reality-based in-situ visualization method and system for shotcrete thickness. Specific examples have been used to illustrate the principles and implementation methods of this disclosure. The descriptions of the above embodiments are only for the purpose of helping to understand the method and its core ideas. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this disclosure. Therefore, the content of this specification should not be construed as a limitation of this disclosure.

[0261] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.

[0262] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

[0263] The terms "an embodiment," "embodiment," or "one or more embodiments" as used herein mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of this disclosure. Furthermore, please note that the examples of the phrase "in one embodiment" do not necessarily all refer to the same embodiment.

[0264] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this disclosure may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0265] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit them. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure.

Claims

1. A method for in-situ visualization of shotcrete thickness based on mixed reality, characterized in that, The method includes: A three-dimensional mesh representation of the shotcrete operation site is constructed, wherein the three-dimensional mesh representation includes the environmental mesh corresponding to the target surface to be sprayed; Using spatial anchor point technology, a virtual robotic arm corresponding to the physical robotic arm in the shotcrete operation site is established, and the virtual robotic arm is kept consistent with the physical robotic arm in terms of pose and spatial position. Virtual sprayed particles are generated, and particle parameters are set for the virtual sprayed particles based on the posture of the physical robotic arm when performing spraying operations and the spraying parameters of the robotic arm nozzle on the physical robotic arm. The particle parameters include at least the initial velocity of the particle, the gravity influence factor, and the spraying angle of the particle. When a collision between the virtual sprayed particle and the environmental mesh is detected, the three-dimensional coordinates of the collision point are mapped to the corresponding mesh cell in the two-dimensional height field mesh, and the current deposition thickness value of the mesh cell is obtained based on the physical parameters of the virtual sprayed particle; wherein, the two-dimensional height field mesh corresponds to the environmental mesh, and the physical parameters include at least the particle mass and adhesion probability of the virtual sprayed particle, and the adhesion probability characterizes the probability that the virtual sprayed particle adheres to the target surface; Based on the accumulated deposition thickness value of each grid cell, the coating thickness at each location point in the grid cell is obtained, and the coating thickness at each location point is rendered onto the target surface using a corresponding color; wherein, different coating thicknesses within different thickness ranges correspond to different colors.

2. The method for in-situ visualization of shotcrete thickness based on mixed reality according to claim 1, characterized in that, The construction of the three-dimensional mesh representation of the shotcrete operation site includes: Using the spatial mapping component of a mixed reality device, the depth information of the shotcrete operation site is automatically scanned, and the three-dimensional mesh representation is generated based on the depth information; A collider component is added to the three-dimensional mesh representation; wherein the collider component is used to perform collision detection on the environment mesh.

3. The method for in-situ visualization of shotcrete thickness based on mixed reality according to claim 1, characterized in that, The method of using spatial anchoring technology to establish a virtual robotic arm corresponding to the physical robotic arm at the shotcrete operation site, and ensuring that the virtual robotic arm and the physical robotic arm maintain consistency in pose and spatial position, includes: An initial virtual robotic arm is constructed, and a virtual QR code is set for the virtual robotic arm; wherein the position of the virtual QR code on the virtual robotic arm is consistent with the position of the physical QR code on the physical robotic arm. The position of the physical QR code is detected in real time by an image acquisition device, and the pose of the virtual robotic arm in the three-dimensional mesh representation is adjusted according to the position until the real-time poses of the virtual QR code and the physical QR code coincide in the mixed reality view, so that the virtual robotic arm is aligned with the physical robotic arm. A persistent spatial anchor point is created for the alignment position when the virtual robotic arm is aligned with the physical robotic arm, and the virtual robotic arm is bound to the spatial anchor point to lock and maintain the alignment relationship between the virtual robotic arm and the physical robotic arm.

4. The method for in-situ visualization of shotcrete thickness based on mixed reality according to claim 1, characterized in that, The process of generating virtual sprayed particles and setting particle parameters for the virtual sprayed particles based on the posture of the physical robotic arm during spraying operations and the spraying parameters of the robotic arm nozzles on the physical robotic arm includes: Create a particle system in Unity and bind the particle system to the virtual model of the robotic arm nozzle; Based on the real-time position of the physical robotic arm during the spraying operation and the spraying parameters, the emission rate, initial velocity, and emission angle of the virtual spraying particles are set. Gravity, air resistance, lifespan, and rebound coefficient are added to the virtual sprayed particles to simulate the physical properties of real sprayed materials; The lifecycle characterizes the time it takes for virtual jet particles that have not collided to disappear; the rebound coefficient is negatively correlated with the adhesion coefficient of the virtual jet particles.

5. The method for in-situ visualization of shotcrete thickness based on mixed reality according to claim 1, characterized in that, The step of mapping the three-dimensional coordinates of the collision point to the corresponding grid cell in the two-dimensional height field grid each time a collision between the virtual sprayed particle and the environmental grid is detected includes: A two-dimensional height field data structure is established to obtain the two-dimensional height field grid, and the two-dimensional height field grid is divided to obtain multiple grid cells; When a collision between the virtual sprayed particle and the environmental mesh is detected, the three-dimensional coordinates of the collision point are extracted; The three-dimensional coordinates are converted into relative coordinates with respect to the origin of the two-dimensional height field grid; Based on the size of the mesh cell and the relative coordinates, the mesh cell to which the three-dimensional coordinates are mapped is obtained in multiple two-dimensional height field meshes.

6. The method for in-situ visualization of shotcrete thickness based on mixed reality according to claim 1, characterized in that, The step of obtaining the current deposition thickness value of the grid cell based on the physical parameters of the virtual sprayed particles includes: The depositional contribution can be obtained using the following formula; ;in, The mass of the particle is... The adhesion probability is... The time factor represents the duration for which the robotic arm nozzle continuously sprays paint in the same pose. The current deposition thickness value of the grid cell is obtained based on the thickness value deposited by the grid cell before the current collision and the deposition contribution.

7. The method for in-situ visualization of shotcrete thickness based on mixed reality according to claim 1, characterized in that, The step of obtaining the coating thickness at each location point in the grid cell based on the accumulated deposition thickness value of each grid cell, and rendering the coating thickness at each location point onto the target surface using a corresponding color, includes: By iterating through the deposition thickness values ​​of multiple grid cells, the maximum and minimum deposition thickness values ​​are obtained; Based on the maximum deposition thickness value and the minimum deposition thickness value, the deposition thickness values ​​of the multiple grid cells are normalized to obtain the normalized thickness value of each grid cell; Based on the normalized thickness value of the grid cell, the coating thickness at each location point in the grid cell is obtained; For the difference between the coating thickness and the target deposition thickness at each location point, the coating thickness at that location point is rendered onto the target surface using a corresponding color. The target deposition thickness value includes the maximum deposition thickness value and the minimum deposition thickness value; or, the target deposition thickness value is a preset thickness range.

8. The method for in-situ visualization of shotcrete thickness based on mixed reality according to claim 7, characterized in that, The mesh unit is a polygon, and the normalized thickness value of the mesh unit includes the normalized thickness values ​​of each endpoint of the mesh unit; obtaining the coating thickness at each position point in the mesh unit based on the normalized thickness value of the mesh unit includes: Based on the normalized thickness values ​​of each endpoint of the grid cell, a linear interpolation is performed between the maximum deposition thickness value and the minimum deposition thickness value to obtain the coating thickness at each location point in the grid cell.

9. The method for in-situ visualization of shotcrete thickness based on mixed reality according to claim 1, characterized in that, The step of rendering the coating thickness at each of the aforementioned locations onto the target surface using corresponding colors includes: The corresponding colors of each of the aforementioned locations are rendered onto the environment mesh, and the environment mesh rendered with the aforementioned colors is holographically projected onto the target surface to display the coating thickness of the target surface in a mixed reality manner. At least one of the particle parameters, the collision trajectory of the virtual sprayed particles, and the physical parameters is rendered into the generated display interface, and the display interface is holographically projected onto the spraying operation site to display the spraying operation process in a mixed reality manner.

10. A mixed reality-based in-situ visualization system for shotcrete thickness, characterized in that, include: An environmental perception module is used to construct a three-dimensional mesh representation of the spraying operation site, wherein the three-dimensional mesh representation includes the environmental mesh corresponding to the target surface to be sprayed; Spatial registration module: Used to establish a virtual robotic arm in the three-dimensional mesh representation that corresponds to the physical robotic arm in the shotcrete operation site using spatial anchor point technology, and to make the virtual robotic arm consistent with the physical robotic arm in pose and spatial position; Particle simulation module: used to generate virtual sprayed particles, and set particle parameters for the virtual sprayed particles based on the posture of the physical robotic arm when performing spraying operations and the spraying parameters of the robotic arm nozzles on the physical robotic arm. The particle parameters include at least the initial velocity of the particles, the gravity influence factor, and the spraying angle of the particles. Thickness calculation module: When a collision between the virtual sprayed particle and the environmental mesh is detected, the three-dimensional coordinates of the collision point are mapped to the corresponding mesh cell in the two-dimensional height field mesh, and the current deposition thickness value of the mesh cell is obtained based on the physical parameters of the virtual sprayed particle; wherein, the two-dimensional height field mesh corresponds to the environmental mesh, and the physical parameters include at least the particle mass and adhesion probability of the virtual sprayed particle, and the adhesion probability characterizes the probability that the virtual sprayed particle adheres to the target surface; Visualization rendering module: used to obtain the coating thickness at each location point in the grid cell based on the accumulated deposition thickness value of each grid cell, and to render the coating thickness at each location point onto the target surface using a corresponding color; wherein, coating thicknesses in different thickness ranges correspond to different colors.

Images