Construction scheme generation method and device, equipment and storage medium

Abstract

The invention discloses a construction scheme generation method and device, equipment and a storage medium, and relates to the technical field of natural language processing, and the method comprises the steps: responding to a voice question proposed by a user for a target construction site, and generating a corresponding information obtaining prompt word according to the voice question; performing information acquisition reasoning according to the information acquisition cue word and the large language model to obtain an information acquisition strategy; acquiring target information of the target construction site based on the information acquisition strategy, and generating a corresponding data search prompt word according to the target information; performing data acquisition reasoning according to the data search prompt word and the large language model to obtain a data acquisition strategy; and obtaining a construction constraint condition of the target construction site based on the data obtaining strategy, and generating a target construction scheme according to the construction constraint condition and the large language model. Through the method, the understanding ability of the large language model in the professional field of buildings is improved, and the scientificity and accuracy of the target construction scheme are ensured while the construction efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology, and in particular to a construction plan generation method, apparatus, equipment and storage medium. Background Art

[0002] During the construction of civil engineering projects, whether workers can strictly follow the construction methods is an important factor affecting the progress, quality and safety of the project.

[0003] However, in actual projects, workers' technical levels are often uneven, and steps are often missed or misoperated. The construction site environment is changeable and the construction materials are complex. It is extremely inconvenient to browse paper documents or query electronic documents, and it is difficult to quickly find the required construction method information.

[0004] Therefore, how to improve construction efficiency under the conditions of complex construction environment and numerous construction materials is a problem that needs to be solved urgently. Summary of the Invention

[0005] The main purpose of this application is to provide a construction plan generation method, device, equipment and storage medium, aiming to solve the technical problem of low construction efficiency caused by complex construction environment and numerous construction materials.

[0006] To achieve the above objectives, this application proposes a construction plan generation method, which includes:

[0007] Responding to a user's voice question regarding a target construction site and generating corresponding information acquisition prompt words according to the voice question;

[0008] Perform information acquisition reasoning based on the information acquisition prompt words and the large language model to obtain an information acquisition strategy;

[0009] Acquire target information of the target construction site based on the information acquisition strategy, and generate corresponding information search prompt words according to the target information;

[0010] Performing data acquisition reasoning based on the data search prompt words and the large language model to obtain a data acquisition strategy;

[0011] The construction constraints of the target construction site are acquired based on the data acquisition strategy, and a target construction plan is generated according to the construction constraints and the large language model.

[0012] In one embodiment, the target information includes a first target location and initial marking information. The step of acquiring the target information of the target construction site based on the information acquisition strategy includes:

[0013] If the information acquisition strategy requires obtaining visual information, the construction site image is acquired in real time by calling the image acquisition function;

[0014] The construction site image is input into a target detection model to generate the initial annotation information at the first target position.

[0015] In one embodiment, the target information further includes second target location information. After the step of inputting the construction site image into the target detection model, the following steps are further included:

[0016] Identify user gesture key points in a single frame image;

[0017] The second target position information is determined based on the user gesture key points.

[0018] In one embodiment, the step of acquiring target information of the target construction site based on the information acquisition strategy and generating corresponding data search prompt words according to the target information further includes:

[0019] If the information acquisition strategy does not require visual information, converting the voice question into target information of the target construction site through the large language model;

[0020] Generate corresponding data search prompt words according to the target information.

[0021] In one embodiment, the construction constraint includes a first construction constraint, and the step of acquiring the construction constraint of the target construction site based on the data acquisition strategy and generating a target construction plan based on the construction constraint and the large language model includes:

[0022] If the data acquisition strategy is to acquire data, the first construction constraint condition of the target construction site is acquired in real time by calling a data query function, where the first construction constraint condition corresponds to the target information;

[0023] A target construction plan is generated according to the first construction constraint and the large language model.

[0024] In one embodiment, the construction constraint further includes a second construction constraint, and the step of acquiring the construction constraint of the target construction site based on the data acquisition strategy and generating a target construction plan based on the construction constraint and the large language model further includes:

[0025] If the data acquisition strategy is that no data acquisition is required, generating the second construction constraint condition based on the knowledge of the large language model itself;

[0026] A target construction plan is generated according to the second construction constraint and the large language model.

[0027] In one embodiment, before the step of responding to a voice question raised by a user regarding a target construction site and generating a corresponding information acquisition prompt word according to the voice question, the method further includes:

[0028] Obtain user wake-up voice and user voiceprint information;

[0029] Comparing the user voiceprint information with the preset target voiceprint information to determine whether the user voiceprint information is consistent with the target voiceprint information;

[0030] If the user voiceprint information is consistent with the target voiceprint information, determining whether the wake-up voice is consistent with the preset target wake-up word;

[0031] If the wake-up voice is consistent with the target wake-up word, the verification result is that the user voice information conforms to the preset voice information.

[0032] In addition, to achieve the above-mentioned purpose, the present application also proposes a construction plan generating device, which includes:

[0033] A voice prompt word acquisition module is used to respond to a user's voice question about a target construction site and generate corresponding information acquisition prompt words according to the voice question;

[0034] A first strategy acquisition module is used to perform information acquisition reasoning based on the information acquisition prompt words and the large language model to obtain an information acquisition strategy;

[0035] a data prompt word generation module, configured to obtain target information of the target construction site based on the voice question and the information acquisition strategy, and generate corresponding data search prompt words according to the target information;

[0036] A second strategy acquisition module is used to perform data acquisition reasoning based on the data search prompt word and the large language model to obtain a data acquisition strategy;

[0037] A construction plan determination module is used to obtain the construction constraints of the target construction site based on the data acquisition strategy, and generate a target construction plan based on the construction constraints and the large language model.

[0038] In addition, to achieve the above-mentioned purpose, the present application also proposes a construction plan generation device, which includes: a memory, a processor, and a computer program stored on the memory and runnable on the processor, and the computer program is configured to implement the steps of the construction plan generation method described above.

[0039] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium. A computer program is stored on the storage medium, and when the computer program is executed by the processor, the steps of the construction plan generation method described above are implemented.

[0040] One or more technical solutions proposed in this application have at least the following technical effects:

[0041] The system responds to user voice questions regarding a target construction site and generates corresponding information acquisition prompts based on the voice questions; performs information acquisition reasoning based on the information acquisition prompts and the large language model to obtain an information acquisition strategy; obtains target information of the target construction site based on the information acquisition strategy and generates corresponding data search prompts based on the target information; performs data acquisition reasoning based on the data search prompts and the large language model to obtain a data acquisition strategy; obtains construction constraints of the target construction site based on the data acquisition strategy and generates a target construction plan based on the construction constraints and the large language model. By receiving user voice questions regarding the target construction site, the system can quickly identify user needs in complex construction environments and with a wide variety of construction materials, avoiding the tedious process of traditional manual input or query. It generates corresponding information acquisition prompts based on the voice questions to avoid ambiguous queries. By generating data search prompts, the system can specifically guide the data acquisition process of the large language model, improving the large language model's understanding ability in the construction field, thereby ensuring that the acquired data is closely related to construction needs. Furthermore, it ensures the accuracy of construction constraints, provides data support for the formulation of target construction plans, and realizes intelligent construction management. While improving construction efficiency, it also ensures the scientificity and accuracy of the target construction plan. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0044] Figure 1 This is a flow chart of the first embodiment of the construction plan generation method of this application;

[0045] Figure 2 This is a flow chart of the second embodiment of the construction plan generation method of this application;

[0046] Figure 3This is a flow chart of the third embodiment of the construction plan generation method of this application;

[0047] Figure 4 This is a flow chart of the fourth embodiment of the construction plan generation method of this application;

[0048] Figure 5 This is a schematic diagram of the module structure of the construction plan generating device according to an embodiment of the present application;

[0049] Figure 6 This is a schematic diagram of the device structure of the hardware operating environment involved in the construction plan generation method in the embodiment of this application.

[0050] The purpose, features and advantages of this application will be further explained with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0051] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0052] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0053] In real-world construction, workers' skill levels often vary, leading to frequent omissions and errors. Furthermore, construction site environments are highly dynamic, and construction materials are complex and diverse. Flipping through both paper and electronic documents is extremely inconvenient, making it difficult to quickly find the required construction method information. Currently, some smart construction sites are equipped with intelligent devices such as smart helmets and smart glasses, but these devices only have simple functions like positioning and facial recognition, making them inadequate for complex tasks like guiding workers according to construction methods. Large language models demonstrate strong comprehension and expression capabilities and hold great potential for knowledge management. However, the original models provided by large model vendors are trained only on general datasets and are unable to effectively understand images of construction scenes. Furthermore, the original large models do not address gesture-based questioning. To achieve these two capabilities, the model requires further pre-training or fine-tuning using specific datasets, which consumes significant computing resources, is cumbersome, and has high training costs.

[0054] Therefore, how to improve construction efficiency under the conditions of complex construction environment and numerous construction materials is a problem that needs to be solved urgently.

[0055] The present application provides a solution that responds to voice questions raised by users regarding a target construction site and generates corresponding information acquisition prompts based on the voice questions; performs information acquisition reasoning based on the information acquisition prompts and a large language model to obtain an information acquisition strategy; obtains target information of the target construction site based on the information acquisition strategy and generates corresponding data search prompts based on the target information; performs data acquisition reasoning based on the data search prompts and a large language model to obtain a data acquisition strategy; obtains construction constraints of the target construction site based on the data acquisition strategy and generates a target construction plan based on the construction constraints and the large language model. By receiving voice questions raised by users regarding a target construction site, the user's needs can be quickly identified under conditions of a complex construction environment and a large number of construction materials, avoiding the tedious process of traditional manual input or query. Corresponding information acquisition prompts are generated based on the voice questions to avoid fuzzy queries. By generating data search prompts, the data acquisition process of the large language model can be guided in a targeted manner, improving the large language model's understanding ability in the field of construction expertise, thereby ensuring that the acquired data is closely related to construction needs. Furthermore, it ensures the accuracy of construction constraints, provides data support for the formulation of target construction plans, and realizes intelligent construction management. While improving construction efficiency, it also ensures the scientificity and accuracy of the target construction plan.

[0056] It should be noted that the execution subject of this embodiment may be a computing service device or smart terminal with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, mobile phone, etc., or an electronic device capable of performing the above functions. This embodiment and the following embodiments will be described below using a smart terminal as an example.

[0057] Based on this, the embodiment of the present application provides a construction plan generation method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the construction plan generation method of this application.

[0058] In this embodiment, the construction plan generation method includes steps S10 to S50:

[0059] Step S10: responding to a voice question raised by a user regarding a target construction site and generating corresponding information acquisition prompt words according to the voice question.

[0060] It should be noted that voice questions can include user commands and inquiries about the construction site. Information acquisition prompts can be understood as structured information query instructions generated by the smart terminal through natural language processing after parsing the user's voice questions. These instructions are used to clarify data requirements and focus on raw data collection. For example, a prompt manager can be provided in the smart terminal. The prompt manager can be understood as a dedicated module for dynamically generating and managing prompts.

[0061] Step S20: performing information acquisition reasoning based on the information acquisition prompt words and the large language model to obtain an information acquisition strategy.

[0062] It should be noted that information acquisition reasoning can be a process of generating an information acquisition strategy through logical chain analysis by combining information acquisition prompt words and context, wherein the information acquisition strategy can be an executable operation instruction. It can be understood that the operation instruction clarifies the retrieval conditions and execution order.

[0063] Step S30: acquiring target information of the target construction site based on the information acquisition strategy, and generating corresponding data search prompt words according to the target information.

[0064] It should be noted that target information can be understood as information actually collected or retrieved from the target construction site according to the information acquisition strategy. It is understood that the output of target information can include various forms, such as voice guidance, images, etc. Data search prompts can be refined search instructions further generated based on the target information. For example, data search prompts can be used to locate related documents or supplementary data.

[0065] Step S40 , performing data acquisition reasoning based on the data search prompt words and the large language model to obtain a data acquisition strategy.

[0066] For example, data acquisition reasoning based on data search prompts and large language models can include multimodal retrieval (e.g., simultaneous processing of heterogeneous data such as text, drawings, and tables), semantic expansion (automatically supplementing synonyms), etc. Data acquisition strategies can be specific operational instructions for guiding intelligent terminals to perform document retrieval, data association, and integration.

[0067] Step S50 : obtaining the construction constraints of the target construction site based on the data acquisition strategy, and generating a target construction plan based on the construction constraints and the large language model.

[0068] It should be noted that construction constraints can be specific conditions or restrictions that must be followed during construction, such as safety standards, environmental protection requirements, and technical specifications. For example, documents such as these can be pre-stored as construction materials in the large language model. The target construction plan can be understood as a specific construction plan or strategy developed based on the construction constraints.

[0069] In this embodiment, a user's voice question regarding a target construction site is responded to, and corresponding information acquisition prompts are generated based on the voice question; information acquisition inference is performed based on the information acquisition prompts and the large language model to obtain an information acquisition strategy; target information of the target construction site is acquired based on the information acquisition strategy, and corresponding data search prompts are generated based on the target information; data acquisition inference is performed based on the data search prompts and the large language model to obtain a data acquisition strategy; construction constraints of the target construction site are acquired based on the data acquisition strategy, and a target construction plan is generated based on the construction constraints and the large language model. By receiving a user's voice question regarding the target construction site, the user's needs can be quickly identified under conditions of a complex construction environment and a large number of construction materials, avoiding the tedious process of traditional manual input or query. Corresponding information acquisition prompts are generated based on the voice question to avoid ambiguous queries. By generating data search prompts, the data acquisition process of the large language model can be guided in a targeted manner, improving the large language model's understanding ability in the construction professional field, thereby ensuring that the acquired data is closely related to construction needs. Furthermore, it ensures the accuracy of construction constraints, provides data support for the formulation of target construction plans, and realizes intelligent construction management. While improving construction efficiency, it also ensures the scientificity and accuracy of the target construction plan.

[0070] Reference Figure 2 , Figure 2 This is a flow chart of the second embodiment of the construction plan generation method of this application, based on the above Figure 1 The first embodiment shown is a second embodiment of the construction plan generation method of the present application.

[0071] In the second embodiment, before step S10, the method further includes:

[0072] Step S010: Obtain user wake-up voice and user voiceprint information.

[0073] It should be noted that the user wake-up voice can be a specific voice command or phrase issued by the user, which can be used to activate the voice recognition system of the smart terminal. The user voiceprint information is the user's voice feature data, which can be used to uniquely identify a user, similar to the role of fingerprints in identity recognition.

[0074] Step S020: Compare the user voiceprint information with the preset target voiceprint information to determine whether the user voiceprint information is consistent with the target voiceprint information.

[0075] It should be noted that the target voiceprint information can be understood as pre-set voiceprint feature data corresponding to a specific user, which is used to compare with the user's voiceprint information to determine the user's identity.

[0076] Step S030: If the user voiceprint information is consistent with the target voiceprint information, it is determined whether the wake-up voice is consistent with the preset target wake-up word.

[0077] It should be noted that the target wake-up word can be understood as a preset specific word or phrase used to wake up the speech recognition system.

[0078] In step S040, if the wake-up voice is consistent with the target wake-up word, the verification result is that the user voice information is consistent with the preset voice information.

[0079] Exemplarily, the verification result can be understood as the result obtained by comparing the user's voiceprint information and wake-up voice with the preset information (target voiceprint information and target wake-up word).

[0080] In this embodiment, by comparing the user's voiceprint information with the preset target voiceprint information, unauthorized users can be effectively prevented from accessing the system. Through multiple verifications (matching of voiceprint information and wake-up words), the system's recognition accuracy of user identity can be greatly improved, and the cases of misidentification and false wake-up can be reduced. The user's identity authentication steps are simplified, and the user only needs to complete the verification through natural voice communication, which improves the convenience and fluency of use.

[0081] In one embodiment, the step of obtaining the target information of the target construction site based on the information acquisition strategy includes: if the information acquisition strategy requires obtaining visual information, obtaining the construction site image in real time by calling the image acquisition function; inputting the construction site image into the target detection model, and generating initial annotation information at the first target position.

[0082] It should be noted that the visual information may be real-life data of the construction site presented in the form of images or videos. Construction site images may be used to analyze information such as construction progress and construction site conditions. For example, the construction site images may include real-time construction site videos or construction images. The image acquisition function may be an auxiliary function for shooting (named the "take_a_phote" function). For example, the image acquisition function may be a predefined program interface. The target detection model may be a machine learning or deep learning model for identifying and locating specific objects in images or videos, such as equipment, personnel, materials, etc. at the construction site. The initial annotation information may be understood as information about each target (such as equipment, personnel, etc.) at the construction site obtained after analysis by the target detection model, including its location, status, and category.

[0083] It should be noted that step S30 also includes: if the information acquisition strategy is not to acquire visual information, converting the voice question into target information of the target construction site through the large language model; and generating corresponding data search prompt words according to the target information.

[0084] Exemplarily, the prompt word manager generates the following system prompt words, or system prompt words with the same or similar semantics as the following content:

[0085]

[0086] User prompt words can be generated based on the voice questions raised by the user regarding the target construction site, and the large language model generates a reply based on the system prompt words and the user prompt words. By parsing the YAML output by the large language model, if the value of the final_answer field is "yes", the "take_a_phote" function is called once and then enters step S40. If the value of the final_answer field is "no", step S40 is directly entered. In this embodiment, by inputting the construction site image into the target detection model and generating initial annotation information at the first target position, real-time data support is provided for the customization of the construction plan. Through systematic target recognition and annotation, the compliance of the construction site can be monitored in real time, and the scientific nature of the construction plan is also guaranteed.

[0087] In one embodiment, after the step of inputting the construction site image into the target detection model, the method further includes: identifying key points of user gestures in a single frame image; and determining second target position information based on the key points of user gestures.

[0088] It should be noted that the gesture key point can be understood as a specific position point of the hand when the user performs a gesture action. For example, the fingertip of the index finger can be taken as the gesture key point, or the hand joint point can be taken as the gesture key point to analyze the gesture action. The index finger pointing position or the hand holding position can be judged as the second target position. The second target position can be a coordinate position determined by a gesture in the construction site image. Exemplarily, the step of determining the second target position based on the user gesture key point may include: traversing each frame of the construction site image; identifying the coordinates of the user gesture key point in each frame of the image, tracking the motion trajectory of the gesture key point coordinates through continuous frames; and determining the second target position based on the motion trajectory.

[0089] It is understood that if the construction site image is a video image, it is composed of a series of static images (frames), each of which is captured at a specific point in time. By comparing adjacent frames, the position changes of the same object (such as gesture key points) at different time points can be tracked. For example, optical flow can be used for gesture tracking. The motion trajectory can reflect the dynamic characteristics and changing patterns of the gesture, which helps to analyze the user's intention.

[0090] In this implementation, by identifying key points of a user's gesture in real time and tracking its trajectory, the target location can be accurately determined based on the gesture. This precise positioning effectively reduces resource waste and construction errors caused by misjudgment of location during the construction process, thereby improving overall construction accuracy. By automatically identifying and labeling target construction mark information, manual intervention and judgment can be reduced, improving construction efficiency.

[0091] Reference Figure 3 , Figure 3 This is a flow chart of the third embodiment of the construction plan generation method of this application, based on the above Figure 2 The second embodiment shown proposes the third embodiment of the construction plan generation method of the present application.

[0092] In the third embodiment, step S50 includes:

[0093] Step A501: If the data acquisition strategy is to acquire data, the first construction constraint of the target construction site is acquired in real time by calling the data query function. The first construction constraint corresponds to the target information.

[0094] It should be noted that the first construction constraint condition may be a quantitative restriction condition obtained from an external system (such as a BIM platform or a specification database) through a data query function.

[0095] Step A502: Generate a target construction plan based on the first construction constraint and the large language model.

[0096] Exemplarily, the prompt word manager generates the following system prompt words, or system prompt words with the same or similar semantics as the following content:

[0097]

[0098] In this embodiment, quantifiable parameters are extracted from the external system, which can convert vague engineering requirements into precise instructions executable by the intelligent terminal. By calling the data query function to obtain the first construction constraint condition of the target construction site in real time, the relevant construction documents can be quickly located, avoiding the inefficient process of manual one-by-one searching. Retrieving construction documents based on keywords can ensure that the information obtained is highly relevant to user needs, thereby improving the accuracy and pertinence of the information.

[0099] Reference Figure 4 , Figure 4 This is a flow chart of the fourth embodiment of the construction plan generation method of this application, based on the above Figure 2 The second embodiment shown proposes the fourth embodiment of the construction plan generation method of the present application.

[0100] In the fourth embodiment, step S50 includes:

[0101] Step B501: If the data acquisition strategy is that data acquisition is not required, a second construction constraint condition is generated based on the knowledge of the large language model itself.

[0102] It should be noted that the second construction constraint can be understood as an empirical constraint generated by relying on the internal knowledge of the large language model, which can quickly respond to general problems.

[0103] Step B502: Generate a target construction plan based on the second construction constraint and the large language model.

[0104] Exemplarily, the prompt word manager generates the following system prompt words, or system prompt words with the same or similar semantics as the following content:

[0105]

[0106]

[0107] In this embodiment, generating the second construction constraint condition based on the knowledge of the large language model itself can bypass the complex data interface call and parsing links, significantly reduce the system computing resource consumption, adapt to edge computing devices, have a fast computing speed, and is suitable for solving general problems.

[0108] In order to make the above embodiments clearer, the following implementation methods are supplemented. It should be noted that the following implementation methods are used to understand the embodiments of the present application and do not constitute a limitation of the present application. Based on the above embodiments and implementation methods, in one implementation method, the construction plan generation method of the present application can be applied to a construction plan generation system, wherein the construction plan generation system may include an intelligent terminal, a laser device and a three-way pan-tilt head, wherein the three-way pan-tilt head is used to control the laser device, and the intelligent terminal can control the three-way pan-tilt head through a wireless connection. Specifically, the intelligent terminal can respond to the user's voice questions regarding the target construction site and generate corresponding information acquisition prompt words based on the voice questions; perform information acquisition reasoning based on the information acquisition prompt words and the large language model to obtain an information acquisition strategy; obtain target information of the target construction site based on the information acquisition strategy, and generate corresponding data search prompt words based on the target information; perform data acquisition reasoning based on the data search prompt words and the large language model to obtain a data acquisition strategy; obtain the construction constraints of the target construction site based on the data acquisition strategy, and generate a target construction plan based on the construction constraints and the large language model. The intelligent terminal also includes a prompt word manager, a database, a function library, and a multimodal large model. The multimodal large model can include an audio-language model with fewer than 10 bytes of parameters, such as Qwen2-Audio-7B-Instruct, and a visual-language model with more than 50 bytes of parameters, such as Qwen2-VL-72B-Instruct. The database can be used to store construction data, and the storage format can include Markdown files and image files. Text information is written directly in the Markdown file, simple tables are written in Markdown syntax, and complex tables are written in Markdown files using HTML syntax. If a text has a corresponding image, the image path is recorded after the text using Markdown syntax, and the corresponding image can be found by following the path. This standardized grammatical format can enhance the large model's ability to understand information. The function library is used to store auxiliary functions, which may include the "take_a_phote" function, the "retrieve" function, the "load_image" function, the "add_marks" function, and the "laser_control" function. The smart terminal controls the camera to shoot the construction site by calling the "take_a_phote" function, stores the captured construction site image in the variable IMAGE1, processes IMAGE1 through the target detection model, marks the target construction items, and stores the processed image in the variable IMAGE2; further, processes IMAGE2 through the target detection model, marks the gesture key points, and stores the processed image in the variable IMAGE3.By calling the "retrieve" function, you can retrieve the corresponding construction documents based on keywords and obtain the corresponding construction constraints. By calling the "load_image" function, you can display the image to the user via the smart terminal. By calling the "add_marks" function, you can annotate target construction marks at target locations based on construction constraints. For example, calling the "add_marks" function will add target construction marks to IMAGE1 and IMAGE3, respectively. It should be noted that although the target construction marks are derived from analyzing IMAGE3, the base image of IMAGE3 is IMAGE1. The target locations on IMAGE3 and IMAGE1 are actually aligned, so adding the target construction marks to IMAGE1 will not affect the accuracy of the target construction marks. By calling the "laser_control" function, coordinates are calculated based on the camera hardware parameters and the relative position of the smart terminal and the three-way pan-tilt head. The smart terminal controls the pan-tilt head via a wireless connection, rotating it so that the laser beam is directed toward the target construction location. The target construction location can be understood as providing construction guidance through laser pointing in a construction assistance scenario.

[0109] Based on the above embodiments and implementation methods, in one implementation method, the construction plan generation method of the present application also includes: determining the construction assistance mode according to the user's question voice, and the construction assistance mode may include a teaching mode and a normal mode. For example, when the construction assistance mode is in the normal mode, the user points to the tied steel mesh and steel skeleton with his finger, and when the user's question voice is: "What is the maximum allowable error of these two? Mark it for me." The user's voice information is received, and the user's voice information is verified to be consistent with the preset voice information. By calling the "take_a_phote" function, IMAGE1, IMAGE2, and IMAGE3 are obtained. The user's hand, steel mesh, and steel skeleton can be identified in the construction site image. The intelligent terminal obtains the corresponding construction constraint conditions according to the user's question voice, calls "retrieve", retrieves the corresponding construction documents according to the keyword, and finds relevant regulations in the "Rebar Sub-project" chapter of the "Concrete Structure Engineering Construction Quality Acceptance Code". To obtain the construction constraint conditions corresponding to the keyword, the "add_marks" function is called. By calling the "add_marks" function, draw a red dot at the position of the steel mesh in IMAGE1 and mark it as "length, width: ±10mm, mesh: ±20mm". Draw a red dot at the position of the steel skeleton in IMAGE1 and mark it as "length: ±10mm, width, height: ±5mm".

[0110] For example, if a user asks, "When installing prefabricated bridge piers, I've completed cleaning the pier cap surface. What's the next step?" the system receives the user's voice message, verifies that it matches the preset voice message, and finds relevant content and accompanying images in the "Construction Methods for Industrialized Prefabricated Bridge Piers." The smart terminal then outputs a voice message informing the user that the next step is to install the mortar baffle and calls the "load_image" function to display the accompanying image on the smart terminal's display interface.

[0111] For example, when the construction assistance mode is in teaching mode, if a user asks, "I'm making hollow hexagonal bricks. What should I do now?", the "take_a_phote" function is called to obtain IMAGE1, IMAGE2, and IMAGE3. The construction site image can identify the user's hand, the concrete distributor, the vibrating table, the mold, and the lead wire. The "retrieve" function is called to search for relevant information in the "Standardized Complete Construction Method for Small Prefabricated Components." The smart terminal outputs a voice message informing the user that they should first fill the mold halfway with material, shake it flat with a vibrating table, then add the lead wire, and finally fill the mold completely. The "laser_control" function is called to control the three-way pan-tilt head, directing the laser beam first toward the distributor, then toward the mold, then toward the vibrating table, then toward the lead wire, then toward the distributor, and finally toward the mold.

[0112] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the construction plan generation method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0113] This application also provides a construction plan generation device, please refer to Figure 5 The construction plan generation device is applied to a construction plan generation device, and the construction plan generation device includes: a camera and a projector; the construction plan generation device includes:

[0114] The voice prompt word acquisition module 10 is used to respond to the user's voice question about the target construction site and generate corresponding information acquisition prompt words according to the voice question;

[0115] A first strategy acquisition module 20 is configured to perform information acquisition reasoning based on the information acquisition prompt words and the large language model to obtain an information acquisition strategy;

[0116] A data prompt word generation module 30 is used to obtain target information of the target construction site based on the voice question and the information acquisition strategy, and generate corresponding data search prompt words according to the target information;

[0117] A second strategy acquisition module 40 is configured to perform data acquisition reasoning based on the data search prompt words and the large language model to obtain a data acquisition strategy;

[0118] The construction plan determination module 50 is configured to obtain the construction constraints of the target construction site based on the data acquisition strategy, and generate a target construction plan according to the construction constraints and the large language model.

[0119] The construction plan generation device provided in this application, which utilizes the construction plan generation method of the aforementioned embodiment, can resolve the technical problem of low construction efficiency caused by complex construction environments and a multitude of construction materials. Compared to the prior art, the beneficial effects of the construction plan generation device provided in this application are the same as those of the construction plan generation method provided in the aforementioned embodiment, and the other technical features of the construction plan generation device are the same as those disclosed in the aforementioned embodiment method, and are not further described here.

[0120] The present application provides a construction plan generation device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the construction plan generation method in the above-mentioned embodiment one.

[0121] Reference below Figure 6 , which shows a schematic diagram of the structure of a construction plan generation device suitable for implementing the embodiments of the present application. The construction plan generation device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (such as in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The construction plan generation device shown is merely an example and should not impose any limitations on the functions and scope of use of the embodiments of the present application.

[0122] like Figure 6As shown, the construction plan generation device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. RAM 1004 also stores various programs and data required for the operation of the construction plan generation device. Processing device 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007, such as a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008, such as a liquid crystal display (LCD), speaker, vibrator, etc.; storage device 1003, such as a magnetic tape or hard disk; and communication device 1009. The communication device 1009 can allow the construction plan generating device to communicate with other devices wirelessly or by wire to exchange data. Figure 6 The construction plan generating apparatus having various systems is shown, but it should be understood that it is not required to implement or possess all of the systems shown. More or fewer systems may be implemented or possessed instead.

[0123] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0124] The construction plan generation device provided in this application, which utilizes the construction plan generation method described in the aforementioned embodiment, can resolve the technical problem of low construction efficiency caused by a complex construction environment and a multitude of construction materials. Compared to the prior art, the beneficial effects of the construction plan generation device provided in this application are the same as those of the construction plan generation method described in the aforementioned embodiment, and the other technical features of the construction plan generation device are the same as those disclosed in the aforementioned embodiment, and are not further elaborated here.

[0125] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0126] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0127] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, and the computer-readable program instructions are used to execute the construction plan generating method in the above-mentioned embodiment.

[0128] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0129] The above-mentioned computer-readable storage medium may be included in the construction plan generation device; or it may exist independently without being assembled into the construction plan generation device.

[0130] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the construction plan generation device, the construction plan generation device is enabled to: respond to the user's voice questions regarding the target construction site, and generate corresponding information acquisition prompt words based on the voice questions; perform information acquisition reasoning based on the information acquisition prompt words and the large language model to obtain an information acquisition strategy; obtain target information of the target construction site based on the information acquisition strategy, and generate corresponding data search prompt words based on the target information; perform data acquisition reasoning based on the data search prompt words and the large language model to obtain a data acquisition strategy; obtain the construction constraints of the target construction site based on the data acquisition strategy, and generate a target construction plan based on the construction constraints and the large language model.

[0131] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0132] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0133] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0134] The computer-readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described construction plan generation method. This computer-readable storage medium can address the technical issues of low construction efficiency due to complex construction environments and a multitude of construction materials. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the construction plan generation method provided in the above-described embodiments, and are not further elaborated here.

[0135] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A construction plan generation method, characterized in that: The method comprises: Responding to a user's voice question regarding a target construction site and generating corresponding information acquisition prompt words according to the voice question; Perform information acquisition reasoning based on the information acquisition prompt words and the large language model to obtain an information acquisition strategy; Acquire target information of the target construction site based on the information acquisition strategy, and generate corresponding information search prompt words according to the target information; Performing data acquisition reasoning based on the data search prompt words and the large language model to obtain a data acquisition strategy; The construction constraints of the target construction site are acquired based on the data acquisition strategy, and a target construction plan is generated according to the construction constraints and the large language model.

2. The method according to claim 1, wherein The target information includes a first target location and initial marking information. The step of acquiring the target information of the target construction site based on the information acquisition strategy includes: If the information acquisition strategy requires obtaining visual information, the construction site image is acquired in real time by calling the image acquisition function; The construction site image is input into a target detection model to generate the initial annotation information at the first target position.

3. The method according to claim 2, wherein The target information also includes second target location information. After the step of inputting the construction site image into the target detection model, the method further includes: Identify user gesture key points in a single frame image; The second target position information is determined based on the user gesture key points.

4. The method according to claim 1, wherein The step of acquiring target information of the target construction site based on the information acquisition strategy and generating corresponding data search prompt words according to the target information further includes: If the information acquisition strategy does not require visual information, converting the voice question into target information of the target construction site through the large language model; Generate corresponding data search prompt words according to the target information.

5. The method according to claim 1, wherein The construction constraint condition includes a first construction constraint condition. The step of obtaining the construction constraint condition of the target construction site based on the data acquisition strategy and generating a target construction plan according to the construction constraint condition and the large language model includes: If the data acquisition strategy is to acquire data, the first construction constraint condition of the target construction site is acquired in real time by calling a data query function, where the first construction constraint condition corresponds to the target information; A target construction plan is generated according to the first construction constraint and the large language model.

6. The method according to claim 1, wherein The construction constraint condition further includes a second construction constraint condition. The step of acquiring the construction constraint condition of the target construction site based on the data acquisition strategy and generating a target construction plan based on the construction constraint condition and the large language model further includes: If the data acquisition strategy is that no data acquisition is required, generating the second construction constraint condition based on the knowledge of the large language model itself; A target construction plan is generated according to the second construction constraint and the large language model.

7. The method according to claim 1, wherein Before the step of responding to the user's voice question regarding the target construction site and generating corresponding information acquisition prompt words according to the voice question, the method further includes: Obtain user wake-up voice and user voiceprint information; Comparing the user voiceprint information with the preset target voiceprint information to determine whether the user voiceprint information is consistent with the target voiceprint information; If the user voiceprint information is consistent with the target voiceprint information, determining whether the wake-up voice is consistent with the preset target wake-up word; If the wake-up voice is consistent with the target wake-up word, the verification result is that the user voice information conforms to the preset voice information.

8. A construction plan generating device, characterized in that: The construction plan generating device comprises: A voice prompt word acquisition module is used to respond to a user's voice question about a target construction site and generate corresponding information acquisition prompt words according to the voice question; A first strategy acquisition module is used to perform information acquisition reasoning based on the information acquisition prompt words and the large language model to obtain an information acquisition strategy; a data prompt word generation module, configured to obtain target information of the target construction site based on the voice question and the information acquisition strategy, and generate corresponding data search prompt words according to the target information; A second strategy acquisition module is used to perform data acquisition reasoning based on the data search prompt word and the large language model to obtain a data acquisition strategy; A construction plan determination module is used to obtain the construction constraints of the target construction site based on the data acquisition strategy, and generate a target construction plan based on the construction constraints and the large language model.

9. A construction plan generating device, characterized in that: The construction plan generating device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the construction plan generating method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the construction plan generation method according to any one of claims 1 to 7 are implemented.

Images