Engineering construction auxiliary system and method, storage medium and electronic equipment

Abstract

The invention relates to the field of intelligent construction, in particular to an engineering construction auxiliary system and method, a storage medium and electronic equipment. The engineering construction auxiliary system comprises a terminal device which comprises a voice acquisition device used for acquiring a voice instruction and transmitting the voice instruction to a cloud data platform, and an AR device used for displaying received target construction auxiliary information; the cloud data platform comprises a voice recognition engine used for analyzing the voice instruction to obtain an engineering auxiliary instruction, and a decision module used for generating initial construction auxiliary information and sending the initial construction auxiliary information to the edge computing node; and the edge computing node comprises a real-time positioning engine which is used for converting the coordinate system to obtain the target construction auxiliary information and sending the target construction auxiliary information to the AR equipment. According to the engineering construction auxiliary system provided by the invention, intelligence and visualization of the engineering construction process can be realized, and the efficiency, accuracy and collaboration of the construction process are improved.

Description

Technical Field

[0001] This disclosure relates to the field of intelligent construction, specifically to an engineering construction assistance system, an engineering construction assistance method, a storage medium, and an electronic device. Background Technology

[0002] New energy projects refer to systematic engineering projects that utilize renewable energy sources such as wind and solar power, and achieve clean power production and efficient utilization through technologies such as wind power, photovoltaics, and energy storage.

[0003] However, the construction of these new energy projects presents the following challenges: large project scale, involving numerous precision equipment and requiring high construction accuracy; complex construction environments, often located in extreme environments such as deserts and plateaus, significantly affected by weather conditions; and high requirements for multidisciplinary collaboration, involving civil engineering, mechanical engineering, electrical engineering, and other fields. Traditional construction methods rely on construction personnel's judgment based on two-dimensional drawings and experience, which can easily lead to installation deviations.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this disclosure is to provide an engineering construction assistance system, engineering construction assistance method, storage medium, and electronic device, which aim to realize the intelligence and visualization of the engineering construction process and improve the efficiency, accuracy, and collaboration of the construction process.

[0006] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0007] According to one aspect of this disclosure, an engineering construction assistance system is provided, comprising: The terminal device includes a voice acquisition device and an augmented reality (AR) device; the voice acquisition device is used to acquire voice commands and transmit the voice commands to a cloud data platform; the AR device is used to display the received target construction assistance information to assist in engineering construction. The cloud data platform includes a speech recognition engine and a decision module; the speech recognition engine is used to parse the speech command to obtain engineering assistance commands; the decision module is used to generate initial construction assistance information in the BIM model coordinate system corresponding to the auxiliary display command when the engineering assistance command is an auxiliary display command, and send the initial construction assistance information to the edge computing node; An edge computing node, including a real-time positioning engine, is used to transform the received initial construction assistance information into the coordinate system of the AR device to obtain the target construction assistance information in the coordinate system of the AR device, and send the target construction assistance information to the AR device.

[0008] Optionally, the speech recognition engine is configured according to the following steps: Create an initial speech recognition model; Construct an engineering domain lexicon, and use the engineering domain lexicon to train the initial speech recognition model to obtain the target speech recognition model; A speech training set under construction environment noise is constructed, and the speech training set is used to calibrate the target speech recognition model to obtain the speech recognition engine.

[0009] Optionally, the cloud data platform further includes: BIM model database is used to store BIM model data corresponding to the BIM model of the construction project. A construction management database is used to store construction management data associated with the sub-models of the BIM model; the sub-models are obtained by splitting the BIM model according to construction stages.

[0010] Optionally, the terminal device further includes a positioning device, and the real-time positioning engine is configured to: Obtain BIM model data from the BIM model database; and Acquire the three-dimensional scene data collected by the positioning device; Align the 3D scene data and the BIM model data spatially, and establish the transformation relationship between the BIM model coordinate system, the site coordinate system and the AR device coordinate system; The coordinate system transformation is performed according to the transformation relationship.

[0011] Optionally, the decision module is further configured to: When the engineering auxiliary instruction is an auxiliary operation instruction, the auxiliary operation instruction is executed.

[0012] According to a second aspect of this disclosure, an engineering construction assistance system is provided, the method comprising: When a voice command is received, it is parsed to obtain engineering auxiliary commands; When the engineering auxiliary instruction is an auxiliary display instruction, initial construction auxiliary information in the BIM model coordinate system corresponding to the auxiliary display instruction is generated; The initial construction assistance information is transformed into the AR device coordinate system to obtain the target construction assistance information in the AR device coordinate system. The AR device displays construction assistance information for the target to aid in the construction process.

[0013] Optionally, the auxiliary display instructions include one or more of the following: BIM model interaction instructions, construction management data retrieval instructions, equipment parameter retrieval instructions, equipment virtual pre-assembly instructions, and positioning and navigation instructions.

[0014] Optionally, when the engineering auxiliary instruction is an auxiliary operation instruction, the auxiliary operation instruction is executed; wherein, the engineering auxiliary instruction further includes auxiliary operation instructions, which include one or more of the following: equipment parameter labeling instruction, engineering acceptance instruction, and acceptance problem labeling instruction.

[0015] According to a third aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the engineering construction assistance method as described in the above embodiments.

[0016] According to a fourth aspect of the present disclosure, an electronic device is provided, characterized in that it includes: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the engineering construction assistance method as described in the above embodiments.

[0017] The exemplary embodiments disclosed herein may have some or all of the following beneficial effects: In the technical solutions provided by some embodiments of this disclosure, on the one hand, the engineering construction assistance system is equipped with a voice recognition engine, and construction personnel can interact through voice, which improves the intelligence of engineering construction assistance and thus greatly improves the efficiency of engineering construction; on the other hand, by displaying construction assistance information corresponding to the assistance display instructions through augmented reality (AR) devices, the engineering construction assistance information is visualized, which can help construction personnel to carry out precise construction in the engineering construction process; furthermore, by building an engineering construction assistance system and managing terminal devices in a unified manner, the collaboration of engineering construction can be improved.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings: Figure 1This schematic diagram illustrates the composition of an engineering construction assistance system according to an exemplary embodiment of the present disclosure; Figure 2 The illustration shows a flowchart of a speech recognition engine configuration method in an exemplary embodiment of the present disclosure. Figure 3 The schematic diagram illustrates a process flow of an engineering construction assistance method according to an exemplary embodiment of the present disclosure; Figure 4 This illustration schematically shows a process flow diagram of an engineering construction assistance method applied to the construction of a solar thermal power plant in an exemplary embodiment of the present disclosure; Figure 5 The schematic diagram illustrates the structure of a computer system of an electronic device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0020] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art.

[0021] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0022] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0023] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0024] New energy projects refer to systematic engineering projects that utilize renewable energy sources such as wind and solar power, and achieve clean electricity production and efficient utilization through technologies such as wind power, photovoltaics, and energy storage. However, the construction of such new energy projects presents the following challenges: large project scale, involving numerous precision equipment and requiring high construction accuracy; complex construction environments, often located in extreme environments such as deserts and plateaus, significantly affected by weather conditions; and high requirements for multi-disciplinary collaboration, involving multiple fields such as civil engineering, machinery, and electrical engineering.

[0025] Traditional construction methods rely on construction workers' judgment based on two-dimensional drawings and experience, which easily leads to installation deviations. Specifically, construction workers have difficulty visualizing the three-dimensional space from two-dimensional drawings, easily causing installation deviations; information transmission between different disciplines is not timely, resulting in low collaboration efficiency; quality inspection relies on manual measurement, which is inefficient and difficult to guarantee accuracy; safety training is mostly in text or video format, with poor immersion and limited training effectiveness; comparison and analysis of construction progress and plans are lagging, making real-time adjustments difficult; the interaction between BIM models and AR scenes is mostly static display, lacking dynamic data association and real-time update mechanisms; construction navigation relies mostly on 2D drawings or simple coordinate guidance, failing to achieve accurate path planning in three-dimensional space; voice interaction technology is insufficiently applied in industrial construction scenarios, especially lacking the ability to recognize professional terminology and complex instructions; and the various system modules (AR display, BIM data, navigation function) are independent of each other, failing to form an integrated collaborative workflow.

[0026] Augmented reality (AR) technology, by integrating virtual information with real-world scenes, has been applied in some industrial sectors, but a systematic solution has not yet been developed for engineering construction, especially in areas such as high-precision positioning in complex environments and large-scale collaborative equipment installation, where there are technological gaps.

[0027] Therefore, there is an urgent need for an AR-assisted construction system and method that integrates intelligent voice interaction, deep BIM fusion, and precise navigation. This disclosure achieves intelligentization and visualization of the construction process through the organic integration of AR voice recognition, BIM dynamic interaction, and 3D navigation functions, thereby improving the accuracy, efficiency, and collaboration of the construction process.

[0028] The implementation details of the technical solutions of the embodiments of this disclosure are described in detail below.

[0029] Figure 1 This schematic diagram illustrates the composition of an engineering construction assistance system according to an exemplary embodiment of this disclosure. For example... Figure 1 As shown, the construction support system 100 includes a terminal device 101, a cloud data platform 102, and an edge computing node 103. Wherein: Terminal device 101 includes a voice acquisition device and an AR device; the voice acquisition device is used to acquire voice commands and transmit the voice commands to a cloud data platform; the AR device is used to display the received target construction assistance information to assist in engineering construction. The cloud data platform 102 includes a speech recognition engine and a decision module; the speech recognition engine is used to parse the speech command to obtain engineering auxiliary commands; the decision module is used to generate initial construction auxiliary information in the BIM model coordinate system corresponding to the auxiliary display command when the engineering auxiliary command is an auxiliary display command, and send the initial construction auxiliary information to the edge computing node; Edge computing node 103 includes a real-time positioning engine, which is used to transform the received initial construction assistance information into the coordinate system to obtain the target construction assistance information in the AR device coordinate system, and send the target construction assistance information to the AR device.

[0030] Specifically, the engineering construction auxiliary system disclosed herein adopts a three-layer architecture of "cloud-edge-terminal": the cloud deploys a cloud data platform as a data middleware, which also serves the functions of data storage and analysis; the edge terminal is mainly responsible for real-time positioning and calculation; and the terminal is used to interact with users.

[0031] In one embodiment of this disclosure, the terminal device 101 may include a voice acquisition device for acquiring voice commands and transmitting the voice commands to a cloud data platform. The engineering construction assistance system supports voice acquisition by collecting user voice information through a voice module. For example, a noise-canceling microphone array, employing multi-microphone beamforming technology, can acquire voice in the mechanical noise environment of a construction site and enhance the voice signal. It can support a dynamic sampling rate of 8kHz to 48kHz, adapting to voice input at different distances.

[0032] Based on the above method, in engineering construction scenarios, construction workers need to wear protective equipment. Voice interaction can greatly reduce the manual operation of construction workers and improve the convenience of operation.

[0033] In addition, the terminal device can also include an image acquisition module, meaning the engineering construction assistance system can also support image acquisition. The acquired video information is then transmitted to a cloud data platform, enabling construction workers to interact through gestures, greatly improving the convenience of interaction. Furthermore, the system can support the collaboration of voice commands and gesture operations, achieving efficient interaction in complex construction scenarios. It can also incorporate eye tracking to automatically focus on BIM model details, reducing the user's operational burden.

[0034] In one embodiment of this disclosure, the terminal device 101 may further include an AR (Augmented Reality) device for displaying received target construction assistance information to aid in engineering construction. Specifically, an AR device is a smart hardware that integrates and interacts with computer-generated virtual information in real time with the real world, primarily used for first-person perspective virtual-real fusion display. Examples include AR glasses or AR helmets, which integrate depth sensors and optical positioning systems to achieve precise alignment between the AR scene and the real environment. Transparent waveguide display technology is used to present voice interaction feedback, ensuring construction workers maintain environmental awareness.

[0035] It should be noted that some AR devices can also integrate the functions of voice acquisition devices and image acquisition modules. For example, some AR devices have cameras. In this case, the functions of voice acquisition and AR display can be completed by using different modules of the AR device separately. This disclosure does not make detailed limitations here.

[0036] In one embodiment of this disclosure, the device can take a series of measures to improve its adaptability to extreme environments. For example, for strong light environments, a high-brightness AR display screen can be used to automatically adjust the contrast; for dusty environments, the device adopts IP65 protection and performs regular cleaning and calibration of optical components; for low-temperature environments, a heating module can be used to maintain the device's operating temperature.

[0037] In one embodiment of this disclosure, the cloud data platform 102 may include a speech recognition engine for parsing the speech commands to obtain engineering assistance commands.

[0038] Figure 2 This illustration schematically depicts a flowchart of a speech recognition engine configuration method according to an exemplary embodiment of this disclosure. Figure 2 As shown, the configuration steps for the speech recognition engine are as follows: Step S201: Create an initial speech recognition model; Step S203: Construct an engineering domain lexicon, and use the engineering domain lexicon to train the initial speech recognition model to obtain the target speech recognition model; Step S205: Construct a speech training set under construction environment noise, and use the speech training set to calibrate the target speech recognition model to obtain the speech recognition engine.

[0039] Specifically, the initial speech recognition model can be a neural network, a machine learning model, etc., and this disclosure does not impose any specific limitations. For example, an end-to-end model based on Transformer can be used.

[0040] To adapt to the engineering and construction field, an engineering-specific lexicon is constructed for model training. This lexicon is divided into three levels: basic instructions, technical terms, and engineering-specific vocabulary. Technical terms include equipment terminology and instruction phrases, while engineering-specific vocabulary includes construction procedures. For example, the engineering-specific lexicon includes a basic instruction library (200 entries), a technical terminology library (1500 entries), and an engineering-specific vocabulary library (800 entries). This improves the sensitivity to recognizing technical terms in voice commands and enhances the accuracy of speech recognition.

[0041] Considering the complex construction environment and significant noise levels, a dedicated speech training set under construction noise conditions can be constructed to perform noise robustness training on the trained model, thereby calibrating the speech recognition model and further improving its accuracy.

[0042] In addition, a dialect training set can be constructed by combining Mandarin with major dialects to make it dialect-adaptable, thereby supporting the mixed recognition of Mandarin and multiple dialects at the same time. Through transfer learning, it can adapt to the terminology system of different projects and improve its applicability.

[0043] When applying the application, voice interaction can be activated using a wake word to prevent accidental triggering, and continuous voice input is supported. The voice recognition engine also possesses multi-turn dialogue capabilities, able to understand ambiguous commands by combining the context of the construction scenario, or to predict needs through the location of construction personnel, equipment status, and historical interaction records, performing context awareness and dynamic adaptation. For example, in "show the installation requirements for this," "this" refers to the equipment currently in the field of view. The voice recognition engine also supports offline recognition capabilities, allowing key commands to be recognized locally in environments without a network connection, which helps ensure construction continuity.

[0044] A speech recognition engine is built based on the trained speech recognition model. When a speech command is received from a speech acquisition device, the speech command is parsed and converted into an engineering auxiliary command.

[0045] In one embodiment of this disclosure, the cloud data platform 102 further includes a decision module, which is used to generate initial construction assistance information in the BIM model coordinate system corresponding to the auxiliary display instruction when the engineering assistance instruction is an auxiliary display instruction, and send the initial construction assistance information to the edge computing node.

[0046] Specifically, if the parsed auxiliary display instructions are indeed auxiliary display instructions, meaning relevant information needs to be displayed on the AR device, such as BIM model interaction instructions, construction management data retrieval instructions, equipment parameter retrieval instructions, equipment virtual pre-assembly instructions, and positioning and navigation instructions, then the decision-making module can utilize the data stored in the cloud data platform for data analysis to generate corresponding initial construction assistance information. The data analysis process differs for different auxiliary display instructions, which will be explained in detail in the subsequent engineering construction assistance methods.

[0047] In one embodiment of this disclosure, the decision module is further configured to execute the auxiliary operation instruction when the engineering auxiliary instruction is an auxiliary operation instruction.

[0048] Specifically, engineering auxiliary instructions can also be auxiliary operation instructions, which are operation instructions that do not need to be displayed on the AR device but can be executed directly. Examples include one or more of the following: equipment parameter annotation instructions, engineering acceptance instructions, and acceptance problem annotation instructions. In this case, no coordinate system transformation is required; the auxiliary operation instruction is executed directly.

[0049] In one embodiment of this disclosure, the cloud data platform 102 further includes: a BIM model database for storing BIM model data corresponding to the BIM model of the construction project (e.g., containing IFC / FBX format); and a construction management database for storing construction management data associated with sub-models of the BIM model; wherein the sub-model is obtained by splitting the BIM model according to construction stages.

[0050] Specifically, the cloud-based data platform contains BIM model data for the construction project, encompassing information from all disciplines. To facilitate lightweight management of the BIM model and reduce memory usage on terminal devices, the BIM model can be split into sub-models for different construction stages. Simultaneously, construction management data for each stage can be linked to these sub-models. This construction management data includes one or more of the following: construction schedule, material information, and quality inspection data. Therefore, structured data such as construction schedules, material information, and quality standards can be directly imported to establish a linked database.

[0051] In one embodiment of this disclosure, the cloud data platform 102 further includes an instruction generation module, which is used to generate the engineering assistance instructions. Specifically, in addition to being triggered by the user, the engineering construction assistance instructions can also be automatically generated by the system based on conditions.

[0052] It should be noted that the cloud-based data platform supports hierarchical access based on permissions and can also be integrated with a model version control system to record design changes and construction adjustments, ensuring data traceability. The cloud-based data platform uses protocols to achieve cross-device data synchronization, supports real-time sharing of annotation information, establishes message queues to handle high-concurrency commands, and ensures low-latency responses for multi-disciplinary collaboration.

[0053] Edge computing node 103 includes a real-time positioning engine, which is used to transform the received initial construction assistance information into the coordinate system to obtain the target construction assistance information in the AR device coordinate system, and send the target construction assistance information to the AR device.

[0054] In one embodiment of this disclosure, the terminal device further includes a positioning device, and the real-time positioning engine is configured to: acquire BIM model data from the BIM model database; acquire three-dimensional scene data collected by the positioning device; spatially align the three-dimensional scene data and the BIM model data, and establish a transformation relationship between the BIM model coordinate system, the site coordinate system and the AR device coordinate system; and perform coordinate system transformation according to the transformation relationship.

[0055] Specifically, the positioning device in the terminal equipment collects 3D scene data. This positioning device includes BeiDou / GPS dual-mode positioning, collecting high-precision 3D point cloud data and geographic information of the construction site to establish a benchmark for the site coordinate system. Through an extended Kalman filter algorithm, it can provide centimeter-level outdoor positioning accuracy, and combined with the geographic coordinates of the BIM model, it achieves global alignment. The positioning device also includes an IMU and LiDAR, which can capture user actions and environmental 3D data in real time for dynamically adjusting the AR perspective and displayed information.

[0056] The 3D scene data in the site coordinate system and the BIM model data in the BIM model coordinate system are spatially aligned, and then converted into data in the AR device coordinate system. This establishes the conversion relationship between the BIM model coordinate system, the site coordinate system and the AR device coordinate system, and builds a real-time positioning engine that supports seamless switching between cross-regional construction scenarios.

[0057] The BIM model coordinate system is a relative three-dimensional Cartesian coordinate system within the BIM model. The site coordinate system, also known as the world coordinate system, is a global and absolute reference system used to define the entire physical world. The AR device coordinate system is the local coordinate system on the AR device, which dynamically changes as the device moves and rotates, used to describe the device's pose in space.

[0058] After building a real-time positioning engine, the initial construction assistance information can be transformed into a coordinate system to obtain target construction assistance information in the AR device coordinate system for display.

[0059] In one embodiment of this disclosure, the edge computing node 103 may further include an AR rendering acceleration module. For example, during the BIM model interaction process using the engineering construction assistance system, a lightweight BIM model rendering can be performed using the Unity or Unreal Engine, combined with edge GPU acceleration to achieve a smooth AR experience.

[0060] In one embodiment of this disclosure, the edge computing node 103 may further include a speech preprocessing unit. Specifically, after the speech acquisition device of the terminal acquires speech information, it can perform noise reduction, endpoint detection, and feature extraction on the original speech signal to preprocess the speech, thereby reducing the computing pressure on the cloud. In addition, the speech preprocessing unit also supports offline keyword wake-up to ensure basic interaction in a network-free environment.

[0061] In one embodiment of this disclosure, the edge computing node 103 may also deploy an edge decision module to perform simple recognition and reasoning, enabling local recognition in a network-free environment to ensure construction continuity. For example, the edge decision module may include a lightweight speech recognition engine, which can then realize some local basic command recognition, or it may include a lightweight action analysis module to analyze the actions of construction personnel in real time and predict their operational intentions based on the context.

[0062] Figure 3 This schematic diagram illustrates a flow chart of an engineering construction assistance method according to an exemplary embodiment of this disclosure. Figure 3 As shown, the auxiliary method for engineering construction includes the following steps: Step S301: When a voice command is acquired, the voice command is parsed to obtain an engineering auxiliary command; Step S303: When the engineering auxiliary instruction is an auxiliary display instruction, generate the initial construction auxiliary information in the BIM model coordinate system corresponding to the auxiliary display instruction; Step S305: Transform the initial construction assistance information into a coordinate system to obtain the target construction assistance information in the AR device coordinate system; Step S307: Display the target construction assistance information through the AR device to assist in the construction of the project.

[0063] The engineering auxiliary instructions can be auxiliary display instructions or auxiliary operation instructions. Auxiliary display instructions include one or more of the following: BIM model interaction instructions, construction management data retrieval instructions, equipment parameter retrieval instructions, equipment virtual pre-assembly instructions, and positioning and navigation instructions. The engineering auxiliary instructions also include auxiliary operation instructions, which include one or more of the following: equipment parameter annotation instructions, engineering acceptance instructions, and acceptance problem annotation instructions. The engineering construction auxiliary methods in steps S201 to S207 have been described in detail in the modules of the aforementioned engineering construction auxiliary system, and will not be repeated here. Specific embodiments are described below using different engineering auxiliary instructions.

[0064] In one embodiment of this disclosure, the engineering construction assistance system supports dynamic calling of BIM models. Specifically, when the assistance display instruction is a BIM model interaction instruction, generating initial construction assistance information in the BIM model coordinate system corresponding to the assistance display instruction includes: extracting model data corresponding to the BIM model interaction instruction from the BIM model database as the initial construction assistance information.

[0065] The specific process of the construction assistance method for this project is as follows: Construction personnel speak "Display parameters of heliostat No. 3" through the microphone in the terminal device; the edge computing node preprocesses the voice signal; the voice recognition engine of the cloud data platform parses the voice signal to obtain the engineering assistance instructions; the decision module of the cloud data platform retrieves the corresponding parameters from the BIM model database; the edge computing node renders AR annotations and broadcasts the parameters by voice; the AR device displays the parameters in augmented reality.

[0066] It should be noted that the above is only an exemplary description of BIM model interaction commands. In other embodiments of this disclosure, the engineering construction assistance system can also be used to trigger BIM model interactions such as detailed display, attribute query, section view, and time dimension evolution of the BIM model through voice commands, such as "display the heat absorber installation step animation". This disclosure does not make specific limitations in this regard.

[0067] In one embodiment of this disclosure, the engineering construction assistance system supports the display of construction management data or equipment parameter data. That is, the assistance display commands can include commands to call construction management data and commands to call equipment parameters. Similar to BIM model interaction, the system extracts the data corresponding to the commands from the corresponding database as initial construction assistance information. For example, real-time construction progress and quality inspection data can be retrieved via voice commands, such as "Query the solar collector installation qualification rate." Alternatively, key parameters of the equipment, such as angle, elevation, and levelness, can be dynamically displayed in an AR scene during construction guidance. The system can be configured to support a hybrid gesture + voice interaction during retrieval; for example, the voice command "Zoom in on flange interface" can be combined with a gesture drag for virtual alignment.

[0068] In one embodiment of this disclosure, the engineering construction assistance system also supports AR simulation interaction where the auxiliary display command is a virtual pre-assembly command for equipment. Assembly animations are pre-configured based on the BIM model and related assembly parameters, and then used during construction to call the animation data as initial construction assistance information for display.

[0069] In one embodiment of this disclosure, the engineering construction assistance system also supports positioning and navigation. Specifically, when the auxiliary display instruction is a positioning and navigation instruction, generating initial construction assistance information in the BIM model coordinate system corresponding to the auxiliary display instruction includes: calculating the current position of the terminal device; querying the target position corresponding to the auxiliary display instruction; and generating a three-dimensional navigation path from the current position to the target position as the initial construction assistance information.

[0070] The specific process of the construction assistance method is as follows: The user issues a voice command, "Navigate to the installation location of thermal storage tank No. 2"; the voice recognition engine of the cloud data platform parses the voice signal to obtain the engineering assistance command, its decision module obtains the current position calculated by the real-time positioning engine of the edge computing node, and at the same time extracts the target position of the thermal storage tank corresponding to the engineering assistance command based on BIM model data, optimizes the construction path through reinforcement learning algorithm, and generates the optimal path from the current position to the target position; the AR device interface displays a three-dimensional path guide with elevation prompts, including elevation change prompts, and combines real-time scanning data to mark obstacles in the navigation path and provide alternative routes, such as "There is hoisting equipment ahead, it is recommended to detour".

[0071] It should be noted that, in addition to assisting with display, engineering construction auxiliary systems can also be used to assist with operation, that is, to directly execute auxiliary operation instructions without necessarily displaying them on the terminal.

[0072] In one embodiment of this disclosure, the engineering construction assistance system supports information annotation. Specifically, when the auxiliary operation instruction is a device parameter annotation instruction, executing the auxiliary operation instruction includes: acquiring the device parameter information corresponding to the device parameter annotation instruction; and triggering an update of the BIM model database based on the device parameter information. Specifically, key parameters of device installation can be dynamically displayed in an AR scene, and dynamic annotation and updating of these parameters can be triggered by voice commands such as "adjust angle".

[0073] In addition, the engineering construction assistance system also supports multi-terminal collaborative annotation, that is, when the BIM model database is updated, an auxiliary display instruction is generated so that the AR device can display the target construction assistance information corresponding to the auxiliary display instruction.

[0074] The specific process of the construction assistance method for this project is as follows: Engineer A uses his AR glasses to verbally annotate "support needs to be added here"; the annotation information is synchronized to the cloud via the data bus, triggering the BIM model update; Engineer B's AR device displays the annotation in real time.

[0075] In one embodiment of this disclosure, the engineering construction assistance system also supports engineering acceptance, that is, the engineering acceptance procedure is initiated by voice, the engineering assistance instruction is parsed as an engineering acceptance instruction, the AR system compares the real scene with the BIM model, and generates a deviation report.

[0076] In one embodiment of this disclosure, the engineering construction assistance system also supports the annotation of acceptance issues, that is, the acceptance issues can be annotated by voice, such as "the flange gap here exceeds the standard by 3mm". The engineering assistance command is parsed as an acceptance issue annotation command and automatically associated with the corresponding location in the BIM model.

[0077] It should be noted that the above-mentioned engineering construction assistance methods can be implemented repeatedly. That is, during the construction process, the user and the system engage in multiple rounds of dialogue to complete different functions. For example, in the scenario of a solar thermal power plant project, construction workers wear AR devices and activate the system by saying the wake-up word "solar thermal assistant"; the construction workers issue the voice command "navigate to the installation position of heliostat No. 12 in area A", and the system generates a 3D navigation path in the AR view, including ground guide lines and elevation prompts; after the construction workers arrive at the location, they issue the voice command "display installation benchmark model", and the virtual model of the heliostat and the installation benchmark line are superimposed in the AR scene; the construction workers issue the voice command "measure the current base level", and the system obtains the actual data through laser scanning, displays the level deviation (such as "+0.5°") in the AR, and provides a voice prompt "the left side is too high and needs adjustment"; during the adjustment process, the construction workers issue the voice command "display angle changes in real time", and the AR interface dynamically updates the angle parameters until the standard is met, and then displays a green confirmation mark; after the installation is completed, the construction workers issue the voice command "record the installation data and submit for acceptance", and the system automatically takes real-time pictures, records parameters, and uploads them to the cloud.

[0078] In one embodiment of this disclosure, all AR interaction data, voice command recordings, and construction process data can also be automatically archived to the cloud to form a traceable digital archive.

[0079] It should be noted that, in addition to user-triggered engineering assistance commands via voice or gesture control, the engineering construction assistance system can also include a command generation module to automatically generate engineering assistance commands based on judgment conditions. For example, it can proactively push instructions for the next process based on the schedule, then execute a voice broadcast such as "Next step: connect the solar collector piping," and display the operation instructions for the corresponding equipment. Alternatively, it can analyze construction deviation data in real time, automatically generate rectification suggestions, and push them to the AR terminal.

[0080] In addition, the engineering construction support system can also be used for safety management, such as AR-based safety training, simulating hazardous scenarios and emergency response procedures; real-time identification of hazardous areas during construction and issuing warnings via AR devices; and AR marking of violations and prompting corrective measures. All of these can be achieved by configuring the functions of each module within the engineering construction support system.

[0081] Figure 4 This schematic diagram illustrates a process flow of an engineering construction support method applied to the construction of a concentrated solar power plant, as shown in an exemplary embodiment of this disclosure. Figure 4 As shown, taking the application of this construction support system in the field of solar thermal power as an example, firstly, a BIM model of the solar thermal power plant is obtained; then, the BIM model is decomposed and lightweighted to obtain various sub-models. On the one hand, a BIM database of model association information, i.e., a construction management database, is built based on the decomposed sub-models to construct a cloud data platform for the solar thermal power plant; on the other hand, a professional vocabulary for solar thermal construction is constructed for subsequent training of the AR speech recognition model; simultaneously, high-precision point cloud geographic information coordinates of the construction site are collected, and then AR hardware terminals are configured, including the trained AR speech recognition model; the transformation relationship between the BIM model coordinate system, the site coordinate system, and the AR device coordinate system is established to obtain the cloud data platform for the solar thermal power plant, thereby completing the construction of the construction support system, and then utilizing AR speech recognition technology. The voice recognition model, acting as a speech recognition engine, combines AR voice interaction with BIM dynamic interaction to achieve various engineering construction assistance functions, such as: intelligent navigation during the 3D intelligent navigation phase of solar thermal power plant construction, including intelligent navigation and inspection of heliostat fields, intelligent navigation and inspection of receiver towers, intelligent navigation and inspection of turbine buildings, navigation and inspection of other core parts of the solar thermal power plant, and customized navigation and inspection of the solar thermal power plant; simulating hazardous scenarios and emergency response procedures; real-time early warning during construction; AR and BIM-assisted multi-disciplinary collaboration; AR equipment-assisted remote expert collaboration; AR visualization-assisted optimization of design and construction schemes; AR simulation interaction and construction pre-simulation; AR system comparison of real-world scenes and BIM models to assist on-site construction; digital acceptance and archiving of projects under the BIM database; and final acceptance and digital handover.

[0082] In one embodiment of this disclosure, the hardware layer of the engineering construction assistance system may include: AR glasses / helmets, providing a first-person perspective of virtual-real fusion display; mobile terminal devices, supporting portable AR applications; high-precision positioning devices, including GPS, Beidou, LiDAR, etc., to achieve centimeter-level positioning; and data acquisition devices, such as high-definition cameras, total stations, etc., for on-site data acquisition.

[0083] The software layer of the engineering construction support system may include: a 3D model engine, responsible for loading and rendering virtual models of various components of the solar thermal power plant; a spatial registration module, which enables precise alignment between the virtual model and the real construction scene; a real-time interaction module, which supports natural interaction methods such as gestures and voice; a data management module, which processes and stores data such as design models, construction progress, and quality inspection; and a multi-terminal collaboration module, which enables information sharing and collaborative work among different participants.

[0084] The data layer of the engineering construction support system may include: BIM model data, containing detailed design information for various disciplines of the solar thermal power plant; geographic information data, including topography, coordinate system, and other information of the construction site; construction process data, including schedule, material information, quality records, etc.; and real-time sensor data, including environmental parameters, equipment status, and other data from sensors at the construction site.

[0085] Engineering construction auxiliary systems can be used for: (a) Visualization of pre-construction planning For example, integrating the BIM model of a solar thermal power plant with the real environment of the construction site can intuitively demonstrate the layout effect of the design scheme in the actual site; AR technology can be used to conduct pre-construction simulations to identify potential spatial conflicts and construction difficulties in advance; and it can help construction personnel understand the complex node structure and improve the efficiency of construction scheme handover.

[0086] (ii) Real-time guidance during construction

[0087] Heliostat array installation guidance: AR glasses display the precise installation position and angle parameters of the heliostats in real time, guiding construction personnel to make accurate adjustments; Pipeline and cable laying guidance: 3D models of pipelines are overlaid on the construction site to intuitively indicate the laying path and connection method; Equipment hoisting assistance: real-time display of equipment hoisting trajectory, positioning position and attitude requirements to improve hoisting accuracy and safety.

[0088] (III) Quality Inspection and Acceptance

[0089] AR technology is used to compare the actual construction results with the design model to quickly identify the deviations and values; AR markings are used to mark the quality control points of key processes to guide inspection personnel to conduct inspections according to specifications; and data during the acceptance process is recorded and uploaded in real time to form a traceable quality archive.

[0090] (iv) Safety training and early warning

[0091] AR-based safety training simulates dangerous scenarios such as working at heights and operating equipment to enhance the safety awareness of construction workers; real-time safety warnings are displayed through AR devices to show information such as dangerous areas and safe distances, and timely reminders are given for violations.

[0092] (v) Multi-disciplinary collaborative management

[0093] It supports construction personnel from different specialties to share real-time information through AR devices, enabling efficient communication; it visualizes construction progress, allowing for a direct comparison between planned and actual progress via AR, and timely detection of progress deviations; it provides remote expert support, enabling on-site personnel to transmit real-time images to remote experts via AR devices, with experts providing guidance through annotation and other methods.

[0094] Based on the above methods, the ease of operation can be improved, with voice interaction reducing manual operations by 80%, solving the operational difficulties faced by construction workers wearing protective equipment; BIM utilization can be increased, with dynamic interactive functions improving BIM model retrieval efficiency by 60% and shortening the time for acquiring professional information by 50%; construction accuracy can be improved, with 3D navigation and real-time parameter annotation controlling the installation error of key equipment within ±0.05°; collaboration efficiency can be improved, with multi-terminal real-time interaction improving cross-disciplinary communication efficiency by 70% and shortening the construction plan adjustment cycle by 40%; the level of intelligence can be improved, with context-aware voice interaction enabling the system command response accuracy to reach over 95%, lowering the operational threshold; and construction efficiency can be improved, reducing the time for understanding drawings, reducing rework rates, and shortening the overall construction cycle. Shorter; Enhanced collaboration capabilities, enabling real-time information sharing among multiple disciplines and improving communication efficiency; Digital traceability, building complete AR construction archives to provide precise data support for later operation and maintenance; Improved construction accuracy, through real-time AR guidance and immersive 3D perception of the project under construction, thereby improving construction accuracy and precision and avoiding rework; Increased construction efficiency, reducing the time for understanding traditional drawings and lowering the rework rate, with statistics showing that construction efficiency can be increased by more than 30%; Enhanced collaboration capabilities, breaking down information barriers and achieving efficient collaboration among multiple disciplines and stakeholders; Reduced training costs, through immersive AR training, accelerating the onboarding speed of new employees and reducing the risks of on-site training; Strengthened safety management, identifying safety hazards in advance, providing real-time warnings of dangerous situations, and reducing the incidence of safety accidents.

[0095] In summary, the construction assistance system, through the fusion of virtual and real technologies, effectively addresses many pain points in traditional construction methods, significantly improving construction accuracy, efficiency, and safety, while reducing project costs, providing a novel solution for efficient construction. With the continuous development of AR technology and its deepening engineering applications, its application prospects in the field of new energy engineering construction will be even broader, potentially driving the transformation and upgrading of energy engineering construction methods towards intelligence and digitalization.

[0096] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0097] In exemplary embodiments of this disclosure, a storage medium capable of implementing the above-described methods is also provided. It may be a portable compact disc read-only memory (CD-ROM) and include program code, and can run on an augmented reality device, such as a mobile phone. However, the program product of this disclosure is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0098] In an exemplary embodiment of this disclosure, an electronic device capable of implementing the above-described method is also provided. Figure 5 The schematic diagram illustrates the structure of a computer system of an electronic device according to an exemplary embodiment of the present disclosure.

[0099] It should be noted that, Figure 5 The computer system 500 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0100] like Figure 5 As shown, the computer system 500 includes a Central Processing Unit (CPU) 501, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 502 or programs loaded from storage section 508 into Random Access Memory (RAM) 503. The RAM 503 also stores various programs and data required for system operation. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An Input / Output (I / O) interface 505 is also connected to the bus 504.

[0101] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. Removable media 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 510 as needed so that computer programs read from them can be installed into storage section 508 as needed.

[0102] In particular, according to embodiments of this disclosure, the processes described below with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit (CPU) 501, it performs various functions defined in the system of this disclosure.

[0103] It should be noted that the computer-readable medium shown in the embodiments of this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0104] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0105] The units described in the embodiments of this disclosure can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the unit itself.

[0106] In another aspect, this disclosure also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to perform the methods described in the above embodiments.

[0107] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0108] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0109] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.

[0110] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An engineering construction auxiliary system, characterized in that, include: Terminal devices, including voice acquisition devices and augmented reality (AR) devices; The voice acquisition device is used to acquire voice commands and transmit the voice commands to the cloud data platform; the AR device is used to display the received target construction assistance information to assist in engineering construction. The cloud-based data platform includes a speech recognition engine and a decision-making module; The speech recognition engine is used to parse the speech command to obtain engineering assistance command; the decision module is used to generate initial construction assistance information in the BIM model coordinate system corresponding to the auxiliary display command when the engineering assistance command is an auxiliary display command, and send the initial construction assistance information to the edge computing node; An edge computing node, including a real-time positioning engine, is used to transform the received initial construction assistance information into the coordinate system of the AR device to obtain the target construction assistance information in the coordinate system of the AR device, and send the target construction assistance information to the AR device.

2. The engineering construction auxiliary system according to claim 1, characterized in that, The speech recognition engine is configured according to the following steps: Create an initial speech recognition model; Construct an engineering domain lexicon, and use the engineering domain lexicon to train the initial speech recognition model to obtain the target speech recognition model; A speech training set under construction environment noise is constructed, and the speech training set is used to calibrate the target speech recognition model to obtain the speech recognition engine.

3. The engineering construction auxiliary system according to claim 1, characterized in that, The cloud data platform also includes: BIM model database is used to store BIM model data corresponding to the BIM model of the construction project. A construction management database is used to store construction management data associated with the sub-models of the BIM model; the sub-models are obtained by splitting the BIM model according to construction stages.

4. The engineering construction auxiliary system according to claim 3, characterized in that, The terminal device further includes a positioning device, and the real-time positioning engine is configured to: Obtain BIM model data from the BIM model database; and Acquire the three-dimensional scene data collected by the positioning device; Align the 3D scene data and the BIM model data spatially, and establish the transformation relationship between the BIM model coordinate system, the site coordinate system and the AR device coordinate system; The coordinate system transformation is performed according to the transformation relationship.

5. The engineering construction auxiliary system according to claim 1, characterized in that, The decision module is also configured to: When the engineering auxiliary instruction is an auxiliary operation instruction, the auxiliary operation instruction is executed.

6. An engineering construction assistance method, applied to any one of the engineering construction assistance systems as described in claims 1 to 5, characterized in that, The method includes: When a voice command is received, it is parsed to obtain engineering auxiliary commands; When the engineering auxiliary instruction is an auxiliary display instruction, initial construction auxiliary information in the BIM model coordinate system corresponding to the auxiliary display instruction is generated; The initial construction assistance information is transformed into the AR device coordinate system to obtain the target construction assistance information in the AR device coordinate system. The AR device displays construction assistance information for the target to aid in the construction process.

7. The engineering construction auxiliary method according to claim 6, characterized in that, The auxiliary display instructions include one or more of the following: BIM model interaction instructions, construction management data call instructions, equipment parameter call instructions, equipment virtual pre-assembly instructions, and positioning and navigation instructions.

8. The engineering construction auxiliary method according to claim 6, characterized in that, When the engineering auxiliary instruction is an auxiliary operation instruction, the auxiliary operation instruction is executed; wherein, the engineering auxiliary instruction further includes auxiliary operation instructions, which include one or more of the following: equipment parameter annotation instruction, engineering acceptance instruction, and acceptance problem annotation instruction.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the engineering construction assistance method as described in any one of claims 6 to 8.

10. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the engineering construction assistance method as described in any one of claims 6 to 8.

Images