Method, device and equipment for generating three-axis linkage track of intelligent tower crane of RAG-like structure

Abstract

The invention discloses a three-axis linkage track generation method, device and equipment for an intelligent tower crane of an RAG-like structure, and belongs to the technical field of intelligent tower cranes. The method comprises the following steps: converting a three-dimensional environment map of a tower crane construction site environment into a semantic embedding vector; acquiring a hoisting task description input by a user; according to the hoisting task description and the semantic embedding vector, retrieving from a track knowledge base to obtain a target historical tower crane three-axis linkage path track; the track knowledge base stores a historical tower crane three-axis linkage path track in the tower crane historical task execution process, a corresponding historical hoisting task description and a historical semantic embedding vector of a corresponding tower crane construction site environment; and by taking the three-axis linkage of the tower crane as a constraint, reconstructing the target historical three-axis linkage path track of the tower crane according to the equipment parameters of the tower crane, the semantic embedding vector and the hoisting task description, and generating a target three-axis linkage path track of the tower crane corresponding to the hoisting task description. The flexibility of path planning can be improved.

Description

Technical Field

[0001] This application belongs to the field of intelligent tower crane technology, and in particular relates to a method, device and equipment for generating three-axis linkage trajectory of an intelligent tower crane with a RAG-like structure. Background Technology

[0002] In the field of motion planning and control, path planning has always been a core component of system performance. Especially in the typical application scenario of tower crane automatic driving, how to generate a safe and reliable trajectory that meets the three-axis linkage motion constraints in a complex construction environment is a key challenge to achieve efficient automated operation.

[0003] In related technologies, path planning schemes utilize path search algorithm libraries to automatically search for a path in three-dimensional space that satisfies both collision safety and optimal distance. However, this approach typically uses "shortest distance" or "minimum time" as optimization objectives, following preset path planning strategies to generate and adjust paths. But in practice, users often develop personalized preferences based on task characteristics, such as "avoiding detours," "prioritizing open areas," and "attempting to cross rather than avoid obstacles." Currently, these complex requirements can only be addressed by manually adjusting algorithm parameters, modifying decision thresholds, or even rewriting planning strategies, resulting in extremely poor flexibility and making it difficult to meet the dynamic operational needs of multi-task, multi-scenario tasks. Summary of the Invention

[0004] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a method, apparatus, and equipment for generating a three-axis linkage trajectory for an intelligent tower crane with a RAG-like structure, so as to improve the flexibility of three-axis linkage path planning for tower cranes.

[0005] Firstly, this application provides a method for generating a three-axis linkage trajectory for a smart tower crane with a RAG-like structure, including: A 3D environmental map of the tower crane construction site is converted into a semantic embedding vector; the 3D environmental map includes the spatial features of the tower crane and the features of obstacles. Obtain the lifting task description input by the user; the lifting task description is used to describe the lifting task objectives and the tower crane's three-axis linkage path planning requirements; The target historical tower crane three-axis linkage path trajectory is retrieved from the trajectory knowledge base based on the hoisting task description and the semantic embedding vector; the trajectory knowledge base stores the historical tower crane three-axis linkage path trajectory during the execution of historical tower crane tasks, the corresponding historical hoisting task description, and the corresponding historical semantic embedding vector of the tower crane construction site environment. Based on the tower crane's equipment parameters, the semantic embedding vector, and the hoisting task description, the target historical tower crane's three-axis linkage path trajectory is reconstructed to generate the target tower crane's three-axis linkage path trajectory corresponding to the hoisting task description.

[0006] According to the RAG-like intelligent tower crane three-axis linkage trajectory generation method of this application, the method converts a three-dimensional environment map of the tower crane construction site into a semantic embedding vector; the three-dimensional environment map includes the spatial features of the tower crane and obstacle features; obtains a lifting task description input by the user; the lifting task description describes the lifting task objective and the tower crane three-axis linkage path planning requirements; retrieves the target historical tower crane three-axis linkage path trajectory from the trajectory knowledge base based on the lifting task description and the semantic embedding vector; the trajectory knowledge base stores historical tower crane three-axis linkage path trajectories during the execution of historical tower crane tasks, corresponding historical lifting task descriptions, and corresponding historical semantic embedding vectors of the tower crane construction site environment; reconstructs the target historical tower crane three-axis linkage path trajectory based on the tower crane's equipment parameters, the semantic embedding vectors, and the lifting task description to generate the target tower crane three-axis linkage path trajectory corresponding to the lifting task description. This application embodiment converts a 3D map of the tower crane construction site environment into a semantic embedding vector, which captures the complex environmental features of the construction site. Combined with the user-input lifting task description, it can clarify the specific objectives and path planning requirements of the lifting task, thereby realizing personalized customization of the tower crane's three-axis linkage path trajectory. It retrieves the target's historical tower crane three-axis linkage path trajectory from the trajectory knowledge base, and reconstructs the target's historical tower crane three-axis linkage path trajectory by integrating tower crane equipment parameters, semantic embedding vectors, and lifting task descriptions, generating a target tower crane three-axis linkage path trajectory adapted to the current task. It integrates historical experience and user preferences, and optimizes the generation based on typical paths in similar environments. Compared with traditional tower crane three-axis linkage path planning methods based on fixed optimization objectives, it integrates the "semantic retrieval + path generation" mechanism in the RAG-like architecture, which can generate personalized paths that are closer to user needs, improve the flexibility of tower crane three-axis linkage path planning, and adapt to the operational needs of multiple tasks and multiple scenarios.

[0007] According to one embodiment of this application, the step of retrieving the target historical tower crane three-axis linkage path trajectory from the trajectory knowledge base based on the hoisting task description and the semantic embedding vector includes: Aligning the hoisting task description with the semantic embedding vector includes: constructing a Prompt based on the hoisting task description and inputting it into a large language model to obtain structured information of the hoisting task description; the structured information includes path planning objectives, path optimization indicators, constraints, and environmental characteristics. The Prompt is rewritten based on the structured information to obtain optimized knowledge base retrieval query information; the knowledge base retrieval query information includes the environmental characteristics and the path optimization indicators. Based on the query information and semantic embedding vector retrieved from the knowledge base, the target historical tower crane three-axis linkage path trajectory is obtained from the trajectory knowledge base.

[0008] In this embodiment, a Prompt is constructed based on the hoisting task description and input into a large language model to obtain structured information containing path planning objectives, path optimization indicators, constraints, and environmental characteristics. This aligns the task description with scene and path features, thereby maximizing "relevance" and "executability" during retrieval and generation. As a result, the target historical tower crane three-axis linkage path trajectory that matches the current task requirements can be matched, improving the accuracy of retrieval.

[0009] According to one embodiment of this application, before converting a three-dimensional environment map of the tower crane construction site environment into a semantic embedding vector, the process includes: Obtain the three-dimensional point cloud data of the tower crane construction site environment; An octree is initialized with the spatial range of the tower crane construction site environment as the root node. The nodes of the octree are recursively divided, and the three-dimensional point cloud data is mapped to the voxel nodes of the octree to obtain a global octree map. The global octree map is used as a three-dimensional environment map of the tower crane construction site.

[0010] In this embodiment, by using octrees to process 3D point cloud data, the construction site space can be divided and managed in layers. Massive point cloud data is mapped into voxel nodes in an orderly manner, preserving the 3D spatial features of the environment. Furthermore, voxelization simplifies the data structure, reduces the complexity of subsequent data processing, and reduces the consumption of computing resources. Moreover, it enables the subsequent path trajectory generation process to perceive high-rise features in the environmental space, adapt to different construction site conditions, and achieve stronger environmental perception and path strategy transfer capabilities.

[0011] According to one embodiment of this application, converting the three-dimensional environmental map of the tower crane construction site environment into a semantic embedding vector includes: The global octree map is divided into multiple local sub-regions; Multi-dimensional feature extraction is performed on the local sub-region to obtain the local semantic embedding vector of the local sub-region; the multi-dimensional features include at least one of the following: voxel node density distribution, spatial traversable direction, vertical / horizontal openness, and voxel node variation gradient. The semantic embedding vector is obtained by aggregating the local semantic embedding vectors of each local sub-region.

[0012] In this embodiment, by dividing local sub-regions and extracting multi-dimensional features in a targeted manner, the unique environmental attributes of different regions can be captured. Voxel density can reflect the density of obstacle distribution, the passable direction of space and vertical / horizontal openness can reflect the passability of the region, and the changing gradient can show the abrupt changes in the environment. The extraction of multi-dimensional features gives the local semantic embedding vector rich environmental semantic information. By aggregating the local vectors into the overall semantic embedding vector, the detailed features of the local sub-regions are preserved, and the expressive power of the semantic embedding vector is enhanced.

[0013] According to one embodiment of this application, the step of performing multi-dimensional feature extraction on the local sub-region to obtain the local semantic embedding vector of the local sub-region includes: Extract the feature vectors of voxel nodes in the local sub-region; The adjacency graph of the local sub-region is constructed based on the voxel nodes in the local sub-region. The adjacency graph includes nodes and edges. The nodes correspond to the voxel nodes of the local sub-region, and the edges indicate that the voxel nodes corresponding to two nodes are adjacent in physical space. The node attributes of the nodes are the feature vectors of the voxel nodes. The local semantic embedding vector of the local sub-region is obtained by extracting multi-dimensional features from the adjacency graph using a graph neural network.

[0014] In this embodiment, the basic attributes of each voxel node are captured by extracting the feature vectors of the voxel nodes, and an adjacency graph is constructed to represent the spatial relationship between voxel nodes in the form of a graph. This can effectively reflect the spatial topological information of the local sub-region. By processing the adjacency graph through a graph neural network, the powerful modeling capability of the graph neural network can be fully utilized to explore the attributes of the nodes in the local sub-region and their spatial connections with the surrounding nodes, so that the generated local semantic embedding vector can more accurately represent the environmental semantics of the local sub-region.

[0015] According to one embodiment of this application, the step of retrieving the target historical tower crane three-axis linkage path trajectory from the trajectory knowledge base based on the hoisting task description and the semantic embedding vector includes: Calculate the similarity between the semantic embedding vector and each historical semantic embedding vector in the trajectory knowledge base, and determine the target historical semantic embedding vector whose similarity is greater than the target similarity; Calculate the similarity between the hoisting task description and the target historical semantic embedding vector for each historical hoisting task description, and determine the historical tower crane three-axis linkage path trajectory corresponding to the historical hoisting task description with the highest similarity as the target historical tower crane three-axis linkage path trajectory.

[0016] In this embodiment, by first filtering historical cases with similar environmental features based on environmental semantic similarity, the search scope is narrowed and the search efficiency is improved. Then, the description of the hoisting task is further matched from the filtering results, so that the environment corresponding to the selected historical tower crane three-axis linkage path trajectory is similar to the current work site environment and matches the current task, thereby improving the relevance of the search results.

[0017] According to one embodiment of this application, the step of reconstructing the target historical tower crane three-axis linkage path trajectory based on the tower crane's equipment parameters, the semantic embedding vector, and the hoisting task description, to generate the target tower crane three-axis linkage path trajectory corresponding to the hoisting task description, includes: The equipment parameters of the tower crane, the semantic embedding vector, the hoisting task description, and the target historical tower crane three-axis linkage path trajectory are input into the path trajectory generation model to obtain the target tower crane three-axis linkage path trajectory corresponding to the hoisting task description output by the generation model.

[0018] In this embodiment, by integrating multi-dimensional information such as tower crane equipment parameters, semantic embedding vectors, hoisting task descriptions, and historical tower crane three-axis linkage path trajectories, the path trajectory generation model can fully draw on the experience of historical tower crane three-axis linkage path trajectories when reconstructing the trajectory. It can also combine the tower crane's own equipment parameters to ensure that the trajectory conforms to the equipment performance. Furthermore, it can adapt the semantic embedding vectors to the characteristics of the tower crane construction site environment, generating a target tower crane three-axis linkage path trajectory that meets user preferences. Thus, it can provide diverse and strategy-rich path results while meeting safety constraints, adapting to the operational needs of multiple tasks and scenarios.

[0019] According to one embodiment of this application, the method further includes: A multi-dimensional evaluation is performed on the three-axis linkage path trajectory of the target tower crane to obtain a first evaluation result; the multi-dimensional evaluation includes at least one of structural rationality evaluation, safety evaluation, and execution stability evaluation; The difference index between the target tower crane's three-axis linkage path trajectory and the target tower crane's historical three-axis linkage path trajectory in the trajectory knowledge base is calculated to obtain a second evaluation result; the difference index includes at least one of the following: spatial difference between trajectory points, velocity curve difference, and embedding vector difference. A quality score for the target tower crane's three-axis linkage path trajectory is generated based on the first evaluation result and the second evaluation result. If the quality score is greater than the target score threshold, the target tower crane three-axis linkage path trajectory, the hoisting task description corresponding to the target tower crane three-axis linkage path trajectory, and the semantic embedding vector are associated and stored in the trajectory knowledge base.

[0020] In this embodiment, by conducting structural rationality assessment, safety assessment, and execution stability assessment on the target tower crane's three-axis linkage path trajectory, the feasibility and reliability of the path trajectory in practical applications can be ensured from different perspectives. By calculating the difference index between the target tower crane's three-axis linkage path trajectory and the target tower crane's historical three-axis linkage path trajectories in the trajectory knowledge base, the innovativeness and adaptability of the path are further analyzed, helping to assess whether the path can better adapt to the specific needs of the current task. The quality score generated by combining the first and second assessment results provides a quantitative indicator for the quality of the path trajectory, making the advantages and disadvantages of the path intuitively apparent. When the quality score exceeds the target score threshold, the target tower crane's three-axis linkage path trajectory and its related information are stored in the trajectory knowledge base, thereby continuously optimizing the content of the trajectory knowledge base and providing a better historical reference for subsequent path planning.

[0021] According to one embodiment of this application, the method further includes: Obtain the three-axis linkage path trajectory of the target tower crane with a quality score greater than the target score threshold and the three-axis linkage path trajectory of the target tower crane with a quality score less than or equal to the target score threshold; Construct positive and negative sample sets based on the target tower crane three-axis linkage path trajectory with a quality score greater than the target score threshold and the target tower crane three-axis linkage path trajectory with a quality score less than or equal to the target score threshold; The path trajectory generation model is fine-tuned and trained using the positive and negative sample sets.

[0022] In this embodiment, by using the discriminative nature of quality scoring, high-quality path trajectories are used as positive samples and low-quality path trajectories as negative samples. This enables the trajectory generation model to learn the characteristics and patterns of high-quality path trajectories, identify and correct problems in low-quality path trajectories, and allow the trajectory generation model to continuously optimize its trajectory generation logic. This gradually improves the ability to generate high-quality trajectories under different working conditions, reduces the output of unqualified trajectories, and achieves self-optimization and long-term accumulation and evolution of path quality.

[0023] Secondly, this application provides a three-axis linkage trajectory generation device for intelligent tower cranes with a RAG-like structure, comprising: The conversion module is used to convert a 3D environmental map of the tower crane construction site into a semantic embedding vector; the 3D environmental map includes the spatial features of the tower crane and obstacle features; The acquisition module is used to acquire the lifting task description input by the user; the lifting task description is used to describe the lifting task objectives and the tower crane's three-axis linkage path planning requirements; The retrieval module is used to retrieve the target historical tower crane three-axis linkage path trajectory from the trajectory knowledge base based on the hoisting task description and the semantic embedding vector; the trajectory knowledge base stores the historical tower crane three-axis linkage path trajectory during the execution of historical tower crane tasks, the corresponding historical hoisting task description, and the corresponding historical semantic embedding vector of the tower crane construction site environment. The generation module is used to reconstruct the target historical tower crane three-axis linkage path trajectory based on the tower crane's equipment parameters, the semantic embedding vector, and the hoisting task description, and generate the target tower crane three-axis linkage path trajectory corresponding to the hoisting task description.

[0024] According to the RAG-like intelligent tower crane three-axis linkage trajectory generation device of this application, the three-dimensional environment map of the tower crane construction site environment is converted into a semantic embedding vector; the three-dimensional environment map includes the spatial features of the tower crane and obstacle features; a lifting task description input by the user is obtained; the lifting task description describes the lifting task objective and the tower crane three-axis linkage path planning requirements; the target historical tower crane three-axis linkage path trajectory is retrieved from the trajectory knowledge base according to the lifting task description and the semantic embedding vector; the trajectory knowledge base stores historical tower crane three-axis linkage path trajectories during the execution of historical tower crane tasks, the corresponding historical lifting task descriptions, and the corresponding historical semantic embedding vectors of the tower crane construction site environment; the target historical tower crane three-axis linkage path trajectory is reconstructed according to the tower crane equipment parameters, the semantic embedding vectors, and the lifting task description to generate the target tower crane three-axis linkage path trajectory corresponding to the lifting task description. This application embodiment converts a 3D map of the tower crane construction site environment into a semantic embedding vector, which captures the complex environmental features of the construction site. Combined with the user-input lifting task description, it can clarify the specific objectives and path planning requirements of the lifting task, thereby realizing personalized customization of the tower crane's three-axis linkage path trajectory. It retrieves the target's historical tower crane three-axis linkage path trajectory from the trajectory knowledge base, and reconstructs the target's historical tower crane three-axis linkage path trajectory by integrating tower crane equipment parameters, semantic embedding vectors, and lifting task descriptions, generating a target tower crane three-axis linkage path trajectory adapted to the current task. It integrates historical experience and user preferences, and optimizes the generation based on typical paths in similar environments. Compared with traditional tower crane three-axis linkage path planning methods based on fixed optimization objectives, it integrates the "semantic retrieval + path generation" mechanism in the RAG-like architecture, which can generate personalized paths that are closer to user needs, improve the flexibility of tower crane three-axis linkage path planning, and adapt to the operational needs of multiple tasks and multiple scenarios.

[0025] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the method for generating a three-axis linkage trajectory of a smart tower crane with a RAG-like structure as described in the first aspect above.

[0026] Fourthly, this application provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method for generating a three-axis linkage trajectory of an intelligent tower crane with a RAG-like structure as described in the first aspect above.

[0027] Fifthly, this application provides a chip, which includes a processor and a communication interface, the communication interface being coupled to the processor, and the processor being used to run programs or instructions to implement the method for generating three-axis linkage trajectory of a smart tower crane with a RAG-like structure as described in the first aspect above.

[0028] Sixthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method for generating a three-axis linkage trajectory of an intelligent tower crane with a RAG-like structure as described in the first aspect above.

[0029] The above-described one or more technical solutions in the embodiments of this application have at least one of the following technical effects: According to the RAG-like intelligent tower crane three-axis linkage trajectory generation method of this application, the method converts a three-dimensional environment map of the tower crane construction site into a semantic embedding vector; the three-dimensional environment map includes the spatial features of the tower crane and obstacle features; obtains a lifting task description input by the user; the lifting task description describes the lifting task objective and the tower crane three-axis linkage path planning requirements; retrieves the target historical tower crane three-axis linkage path trajectory from the trajectory knowledge base based on the lifting task description and the semantic embedding vector; the trajectory knowledge base stores historical tower crane three-axis linkage path trajectories during the execution of historical tower crane tasks, corresponding historical lifting task descriptions, and corresponding historical semantic embedding vectors of the tower crane construction site environment; reconstructs the target historical tower crane three-axis linkage path trajectory based on the tower crane's equipment parameters, the semantic embedding vectors, and the lifting task description to generate the target tower crane three-axis linkage path trajectory corresponding to the lifting task description. This application embodiment converts a 3D map of the tower crane construction site environment into a semantic embedding vector, which captures the complex environmental features of the construction site. Combined with the user-input lifting task description, it can clarify the specific objectives and path planning requirements of the lifting task, thereby realizing personalized customization of the tower crane's three-axis linkage path trajectory. It retrieves the target's historical tower crane three-axis linkage path trajectory from the trajectory knowledge base, and reconstructs the target's historical tower crane three-axis linkage path trajectory by integrating tower crane equipment parameters, semantic embedding vectors, and lifting task descriptions, generating a target tower crane three-axis linkage path trajectory adapted to the current task. It integrates historical experience and user preferences, and optimizes the generation based on typical paths in similar environments. Compared with traditional tower crane three-axis linkage path planning methods based on fixed optimization objectives, it integrates the "semantic retrieval + path generation" mechanism in the RAG-like architecture, which can generate personalized paths that are closer to user needs, improve the flexibility of tower crane three-axis linkage path planning, and adapt to the operational needs of multiple tasks and multiple scenarios.

[0030] Furthermore, in some embodiments, by constructing a Prompt based on the hoisting task description and inputting it into a large language model, structured information containing path planning objectives, path optimization indicators, constraints, and environmental characteristics is obtained. This aligns the task description with scene features and path features, thereby maximizing "relevance" and "executability" during the retrieval and generation process. As a result, it is possible to match the target historical tower crane three-axis linkage path trajectory that matches the current task requirements, thus improving the accuracy of retrieval.

[0031] Furthermore, in some embodiments, by using octrees to process 3D point cloud data, the construction site space can be divided and managed in layers, and massive point cloud data can be mapped into voxel nodes in an orderly manner, preserving the 3D spatial features of the environment. Moreover, the data structure is simplified through voxelization, reducing the complexity of subsequent data processing and reducing the consumption of computing resources. It also enables the subsequent path trajectory generation process to perceive the high-rise features in the environmental space, adapt to different construction site conditions, and achieve stronger environmental perception and path strategy transfer capabilities.

[0032] Furthermore, in some embodiments, by dividing local sub-regions and selectively extracting multi-dimensional features, it is possible to capture the unique environmental attributes of different regions. Voxel density can reflect the density of obstacle distribution, the traversable direction of space and vertical / horizontal openness can reflect the traversability of the region, and the changing gradient can show the abrupt changes in the environment. The extraction of multi-dimensional features gives the local semantic embedding vector rich environmental semantic information. By aggregating the local vectors into the overall semantic embedding vector, the detailed features of the local sub-regions are preserved, and the expressive power of the semantic embedding vector is enhanced.

[0033] Furthermore, in some embodiments, by extracting the feature vectors of voxel nodes to capture the basic attributes of each voxel node, an adjacency graph is constructed to represent the spatial relationships between voxel nodes in the form of a graph. This can effectively reflect the spatial topological information of the local sub-region. By processing the adjacency graph through a graph neural network, the powerful modeling capabilities of the graph neural network can be fully utilized to explore the attributes of the nodes themselves in the local sub-region and their spatial connections with surrounding nodes, so that the generated local semantic embedding vector can more accurately represent the environmental semantics of the local sub-region.

[0034] Furthermore, in some embodiments, by first filtering historical cases with similar environmental features based on environmental semantic similarity, the search scope is narrowed and the search efficiency is improved. Then, the description of the hoisting task is further matched from the filtering results, so that the environment corresponding to the selected historical tower crane three-axis linkage path trajectory is similar to the current work site environment and matches the current task, thereby improving the relevance of the search results.

[0035] Furthermore, in some embodiments, by integrating multi-dimensional information such as tower crane equipment parameters, semantic embedding vectors, hoisting task descriptions, and historical tower crane three-axis linkage path trajectories, the path trajectory generation model can fully draw on the experience of historical tower crane three-axis linkage path trajectories when reconstructing the trajectory. It can also combine the tower crane's own equipment parameters to ensure that the trajectory conforms to the equipment performance. Moreover, it can adapt the semantic embedding vectors to the characteristics of the tower crane construction site environment and generate a target tower crane three-axis linkage path trajectory that meets user preferences. Thus, it can provide diverse and strategy-rich path results while meeting safety constraints, adapting to the operational needs of multiple tasks and scenarios.

[0036] Furthermore, in some embodiments, by conducting structural rationality assessment, safety assessment, and execution stability assessment of the target tower crane's three-axis linkage path trajectory, the feasibility and reliability of the path trajectory in practical applications can be ensured from different perspectives. By calculating the difference index between the target tower crane's three-axis linkage path trajectory and the target historical tower crane three-axis linkage path trajectories in the trajectory knowledge base, the innovativeness and adaptability of the path can be further analyzed, helping to assess whether the path can better adapt to the specific needs of the current task. The quality score generated by combining the first and second assessment results provides a quantitative indicator for the quality of the path trajectory, making the advantages and disadvantages of the path intuitively apparent. When the quality score exceeds the target score threshold, the target tower crane's three-axis linkage path trajectory and its related information are stored in the trajectory knowledge base, thereby continuously optimizing the content of the trajectory knowledge base and providing a better historical reference for subsequent path planning.

[0037] Furthermore, in some embodiments, by using the discriminative nature of quality scoring, high-quality path trajectories are used as positive samples and low-quality path trajectories as negative samples. This enables the trajectory generation model to learn the characteristics and patterns of high-quality path trajectories, identify and correct problems in low-quality path trajectories, and allow the trajectory generation model to continuously optimize its trajectory generation logic, gradually improve its ability to generate high-quality trajectories under different working conditions, reduce unqualified trajectory outputs, and achieve self-optimization and long-term accumulation and evolution of path quality.

[0038] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0039] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart illustrating the method for generating a three-axis linkage trajectory of a smart tower crane with a RAG-like structure provided in this application embodiment; Figure 2 This is a schematic diagram of the structure of the intelligent tower crane three-axis linkage trajectory generation device with a RAG-like structure provided in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0041] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0042] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0043] The following description, in conjunction with the accompanying drawings, details the method, apparatus, and equipment for generating three-axis linkage trajectories of a smart tower crane with a RAG-like structure provided in this application, through specific embodiments and application scenarios.

[0044] Among them, the method for generating the three-axis linkage trajectory of a smart tower crane with a RAG-like structure can be applied to a terminal, specifically executed by the hardware or software in the terminal.

[0045] The terminal includes, but is not limited to, portable communication devices such as mobile phones or tablets with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that, in some embodiments, the terminal may not be a portable communication device, but rather a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads).

[0046] The following embodiments describe a terminal including a display and a touch-sensitive surface. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, mouse, and joystick.

[0047] The method for generating a three-axis linkage trajectory of a smart tower crane with a RAG-like structure provided in this application embodiment can be executed by an electronic device or a functional module or entity within an electronic device that can implement the method. The electronic devices mentioned in this application embodiment include, but are not limited to, mobile phones, tablets, computers, cameras, and wearable devices. The following description uses an electronic device as the execution subject to illustrate the method for generating a three-axis linkage trajectory of a smart tower crane with a RAG-like structure provided in this application embodiment.

[0048] like Figure 1 As shown, the method for generating the three-axis linkage trajectory of the intelligent tower crane with this type of RAG structure includes steps 110, 120, 130 and 140.

[0049] Step 110: Convert the 3D environment map of the tower crane construction site into a semantic embedding vector; the 3D environment map includes the spatial features of the tower crane and the features of obstacles.

[0050] In the construction industry, the tower crane construction site environment refers to the physical spatial environment in which the tower crane operates during actual construction. It is a complex and dynamically changing space, typically encompassing various elements such as the building's frame structure, construction equipment (e.g., tower cranes, hoists), temporary scaffolding, stacked building materials (e.g., steel, concrete blocks), and the activity area for construction workers. These elements not only intertwine spatially but also change in position and form over time (e.g., as construction progresses). For example, as the building's floors rise, the tower crane's operating range and the distribution of surrounding obstacles will change accordingly. Furthermore, dynamic factors may exist on the construction site, such as the movement of transport vehicles and the movement of construction workers, which can influence the tower crane's path planning.

[0051] A 3D environment map is a digital model that describes the various elements of a construction site and their interrelationships using precise geometric and spatial information. A 3D environment map can include the spatial characteristics of tower cranes, such as their boom length, boom height, and rotation angle range, as well as obstacle features, such as their location, shape, and size. For example, the frame structure of a building can be represented using 3D coordinate points and connections, recording the location, size, and shape of each structural component. The location and operational range of construction equipment can also be marked on the 3D environment map, including the base location of tower cranes, the length of their booms, and their rotation range. The 3D environment map can also include obstacle distribution information, such as dense areas of scaffolding and areas where materials are stacked.

[0052] In some embodiments, a three-dimensional environmental map of the tower crane construction site is constructed by multi-source sensor fusion. For example, LiDAR is used to scan and acquire three-dimensional point cloud data of the construction site, high-definition cameras are used to collect image information, and GPS (Global Positioning System) and IMU (Inertial Measurement Unit) are used to assist in locating the sensor positions, thereby generating a three-dimensional environmental map containing information such as location, size, shape, and category.

[0053] While 3D environment maps contain rich 3D data, this data is discrete physical information and cannot be directly understood by algorithms for semantic relationships; for example, buildings under construction are obstacles to be avoided. Semantic embedding vectors are a mathematical representation that can capture environmental features and semantic information. Converting 3D environment maps into semantic embedding vectors transforms complex 3D spatial data into low-dimensional, semantically meaningful vectors, thereby representing deep features such as the category, attributes, and spatial relationships of elements in the environment.

[0054] In some embodiments, features can be extracted from a 3D environment map, such as building types (e.g., frame structures, walls, etc.), obstacle types (e.g., scaffolding, material piles, etc.), and equipment locations and types (e.g., tower cranes, hoists, etc.). For example, on a construction site, the building frame structure can be one feature, and the corresponding feature vector can contain information such as height, width, and location; scaffolding can be identified as another feature, and the feature vector can contain information such as density and distribution range. After feature extraction, the extracted features can be semantically labeled, assigning each feature a specific semantic meaning. For example, the building frame can be labeled as a "structural obstacle," the scaffolding as a "temporary obstacle," and the tower crane as a "mobile device." Using machine learning algorithms, such as convolutional neural networks or graph neural networks in deep learning, the labeled features and their semantic information are mapped into a low-dimensional embedding space, generating semantic embedding vectors. Each dimension in the semantic embedding vector can implicitly contain specific semantic information; for example, one dimension might correspond to "distance from the tower crane," another dimension might correspond to "obstacle density," and the distance between vectors can represent the similarity of the environment.

[0055] Semantic embedding vectors can preserve the spatial relationships and semantic information in the original 3D environment map, facilitating subsequent path planning algorithms for processing and analysis. For example, for a construction site containing multiple building frames and scaffolding, after feature extraction and semantic annotation, a convolutional neural network can compress the environmental features and semantic information into a fixed-length vector. This vector can then be used as a semantic embedding vector for subsequent path planning.

[0056] Step 120: Obtain the lifting task description input by the user; the lifting task description is used to describe the lifting task objectives and the tower crane's three-axis linkage path planning requirements.

[0057] In this embodiment, the lifting task description input by the user is a specific requirement related to the current lifting operation, expressed by tower crane operators or construction managers using natural language (text, speech-to-text, etc.). The lifting task description may include lifting task objectives and path planning requirements.

[0058] The lifting task objective describes the specific lifting task that the tower crane needs to complete, such as the starting position, target position, type of object to be lifted (e.g., type and size of building materials), and weight to be lifted. The lifting task objective determines the basic direction and range of the tower crane's movement. For example, if the task is to lift a large precast concrete slab from the ground to the fifth floor of a building under construction, the task objective clearly states that the tower crane needs to move from a specific location on the ground to a designated location on the fifth floor, and the impact of the size and weight of the object being lifted on path planning must be considered.

[0059] Three-axis linkage of a tower crane refers to the synchronous movement of the hook in three-dimensional space during lifting operations through the coordinated action of three independent but controllable motion axes: the slewing axis, the luffing axis, and the hoisting axis. Specifically, the slewing axis controls the rotation of the boom around the tower's centerline on the horizontal plane, with motion parameters including the slewing angle and angular velocity; the luffing axis controls the longitudinal movement of the luffing trolley along the boom, with motion parameters including the trolley's distance from the slewing center and the luffing speed; and the hoisting axis controls the vertical lifting and lowering of the hook, with motion parameters including the hook height and hoisting speed. During the execution of the three-axis linkage path, the motion commands of each axis need to be calculated in real time to ensure that the hook moves along the target trajectory at a preset speed and acceleration, achieving smooth, efficient, and safe hook movement.

[0060] The three-axis linkage path of a tower crane is the trajectory of the hook moving in three-dimensional space during the lifting operation, as well as the trajectory formed by the coordinated movement of the three axes of motion of the tower crane: slewing axis, luffing axis, and hoisting axis.

[0061] The requirements for tower crane three-axis linkage path planning describe the user's specific requirements for path planning, reflecting the user's personalized preferences in actual operation. For example, a user may require "avoiding detours as much as possible," indicating that the tower crane three-axis linkage path planning should choose a straight or shortest path as much as possible while ensuring safety; a user may also require "prioritizing open areas," indicating that the tower crane three-axis linkage path planning should prioritize areas with fewer obstacles; and a user may require "attempting to cross over rather than avoid obstacles," indicating the need to assess the height and shape of obstacles to determine whether they can be crossed over, rather than simply avoiding them.

[0062] In some embodiments, users can input a description of the hoisting task in a graphical user interface (GUI). For example, users can select the starting and target points of the hoisting by clicking with the mouse, input the type and weight of the object to be hoisted, and select route planning preferences via drop-down menus, such as "avoid detours" or "prioritize open areas." Of course, users can also input the hoisting task description via voice or text input, and this embodiment of the application does not limit this method.

[0063] Step 130: Retrieve the target historical tower crane three-axis linkage path trajectory from the trajectory knowledge base based on the hoisting task description and semantic embedding vector; the trajectory knowledge base stores the historical tower crane three-axis linkage path trajectory during the execution of historical tower crane tasks, the corresponding historical hoisting task description, and the corresponding historical semantic embedding vector of the tower crane construction site environment.

[0064] In this embodiment of the application, the trajectory knowledge base is a carrier for storing the past operation data of the tower crane, which is equivalent to an accumulated historical experience base. It is a structured data set with "task-environment-trajectory" as the correlation dimension, which can provide similar scenario solutions that can be referenced for the current task.

[0065] In some embodiments, a trajectory knowledge base can be constructed based on the records of daily tower crane operations. Upon completion of a lifting task, the historical three-axis linkage path trajectory of the tower crane (i.e., the three-dimensional coordinate sequence of the tower crane hook movement, the three-dimensional coordinate sequence of the coordinated movements of the tower crane's slewing axis, luffing axis, and hoisting axis, etc.), the corresponding historical lifting task description (the lifting task description input by the user), and the historical semantic embedding vector of the tower crane construction site environment at that time (converted from the three-dimensional environment map at that time) can be stored. These data are linked through timestamps, task numbers, and other information to form a one-to-one correspondence between "environmental characteristics - lifting task description - path trajectory".

[0066] In some embodiments, semantic similarity matching can be performed between the hoisting task description and historical hoisting task descriptions in the trajectory knowledge base. The hoisting task description input by the current user, such as "hoisting from rebar pile A to the 5th floor of Building 3, prioritizing straight lines," is compared semantically with historical hoisting task descriptions stored in the trajectory knowledge base using natural language processing techniques. For example, if a historical hoisting task description is "hoisting from rebar pile B to the 4th floor of Building 2, minimizing turns," the two have a high degree of semantic overlap in terms of "hoisting object (rebar)," "operating floor (mid-to-high floors)," and "path preference (straight lines / few turns)," and can be determined to be similar.

[0067] The process of retrieving the target historical three-axis linkage path trajectory of the tower crane from the trajectory knowledge base also requires combining semantic embedding vectors. Semantic similarity matching can be performed between the semantic embedding vectors and historical semantic embedding vectors in the trajectory knowledge base. The similarity between the current tower crane construction site environment and the historical semantic embedding vectors in the trajectory knowledge base is calculated to determine whether the tower crane construction site environments are similar. For example, if the semantic embedding vector of the current tower crane construction site environment shows "a building under construction on the east side, a pile of materials on the north side, and an open area in the center," while the semantic embedding vector of a historical record shows "a wall under construction on the east side, a pile of steel bars on the north side, and an unobstructed passage in the center," the two are highly consistent in terms of element type and spatial distribution, and can be judged as having similar environments.

[0068] Specifically, firstly, a natural language model, such as BERT (Bidirectional Encoder Representations from Transformers), can be used to convert the current user-inputted hoisting task description into a task semantic embedding vector. For each historical record in the trajectory knowledge base, the cosine similarity between the current task semantic embedding vector and the semantic embedding vector of the historical hoisting task description is calculated; the cosine similarity between the current tower crane site environment semantic embedding vector and the historical semantic embedding vector is also calculated. Subsequently, a weighted average similarity score is obtained (e.g., task similarity weight 0.6, environment similarity weight 0.4). A higher score indicates a higher match between the historical record and the current hoisting task description. The historical tower crane three-axis linkage path trajectory from one or more historical records with the highest scores and highest matching degrees can be selected as the target historical tower crane three-axis linkage path trajectory.

[0069] Step 140: Reconstruct the target historical tower crane three-axis linkage path trajectory based on the tower crane's equipment parameters, semantic embedding vector, and hoisting task description, and generate the target tower crane three-axis linkage path trajectory corresponding to the hoisting task description.

[0070] While the target historical tower crane three-axis linkage path trajectory retrieved from the trajectory knowledge base is generated under similar environments and similar lifting task descriptions, direct reuse will result in trajectory inapplicability due to differences in the current tower crane construction site environment details, tower crane equipment status, and specific task parameters. For example, a historical tower crane three-axis linkage path trajectory may be generated based on another tower crane with a larger lifting capacity, or a certain obstacle in the corresponding tower crane construction site environment may have been removed. Direct use may lead to collision risks or equipment overload. Therefore, the reconstruction process uses the historical tower crane three-axis linkage path trajectory as a basic framework, making targeted adjustments based on current constraints to generate a target tower crane three-axis linkage path trajectory that meets user needs and adapts to equipment capabilities and the tower crane construction site environment.

[0071] In this embodiment, the tower crane's equipment parameters are the fundamental constraints for path planning, determining the tower crane's capability range and movement limitations during operation. These parameters can include motion parameters such as the maximum lifting height, working radius range, and speed limits for each mechanism; performance parameters such as maximum lifting capacity and rated lifting capacity at different radius levels; and structural parameters such as tower height, boom length, and minimum hook position. These equipment parameters constitute the range constraints of the path trajectory. For example, if the current maximum lifting height of the tower crane is 40 meters, and a segment of the historical three-axis linkage path reaches a height of 45 meters, then the height of that segment needs to be reduced through reconstruction.

[0072] The semantic embedding vector contains detailed semantic features of the current tower crane construction site environment, such as obstacle locations, open area distribution, and dynamic element states. During reconstruction, it is necessary to verify whether the historical tower crane three-axis linkage path trajectory conflicts with the current tower crane construction site environment based on these features, and make targeted adjustments accordingly. For example, if the semantic embedding vector shows "an 8-meter-high scaffold has now been added to the area traversed by the historical trajectory," then the trajectory path needs to be adjusted to avoid or cross it; if it shows "the area bypassed by the historical trajectory has now become open space," the path can be shortened based on the user's need to "avoid detours."

[0073] The task objectives and personalized requirements in the lifting task description determine the direction of path trajectory adjustments. For example, if the starting and / or ending positions in the task objectives have changed compared to the historical three-axis linkage path trajectory of the tower crane, then the starting and / or ending positions need to be readjusted. If the weight of the object being lifted has changed, then the movement range of the path trajectory needs to be adjusted appropriately. When the user requests "attempt to cross rather than avoid obstacles," if the historical three-axis linkage path trajectory of the tower crane chooses to detour around a low obstacle, the reconstruction needs to consider the lifting height parameters of the tower crane to determine whether the hook height can be increased to cross the obstacle in order to reduce the path length. If the user requests "prioritize open areas," then the path adjustment needs to strengthen the path bias towards open areas.

[0074] In some embodiments, reconstructing the historical three-axis linkage path trajectory of a tower crane is a dynamic adjustment and optimization process. First, the target historical three-axis linkage path trajectory retrieved from the trajectory knowledge base can be used as the initial path. This initial path is adjusted according to the tower crane's equipment parameters to ensure that each point on the path is within the tower crane's movement range and meets the tower crane's movement accuracy and speed limits. Next, the path is further optimized based on the semantic embedding vector of the tower crane construction site environment and the user-input lifting task description. For example, if the lifting task description requires "avoiding detours as much as possible," optimization algorithms, such as genetic algorithms or particle swarm optimization algorithms, can be used to adjust the path by combining the semantic features in the semantic embedding vector of the tower crane construction site environment, thereby reducing detours.

[0075] In some embodiments, the reconstructed path trajectory can also be verified for safety and feasibility, checking whether the path collides with obstacles and whether it meets the motion constraints of the tower crane. If a safety hazard or infeasibility is found in the path, the path can be readjusted and optimized until a target tower crane three-axis linkage path trajectory that fully meets the requirements is generated.

[0076] According to the RAG-like intelligent tower crane three-axis linkage trajectory generation method of this application, the following steps are taken: 1) Convert a three-dimensional environment map of the tower crane construction site into a semantic embedding vector; the three-dimensional environment map includes the spatial features of the tower crane and obstacle features; 2) Obtain the user-inputted hoisting task description; the hoisting task description describes the hoisting task objective and the tower crane three-axis linkage path planning requirements; 3) Retrieve the target historical tower crane three-axis linkage path trajectory from the trajectory knowledge base based on the hoisting task description and the semantic embedding vector; the trajectory knowledge base stores historical tower crane three-axis linkage path trajectories during the execution of historical tower crane tasks, the corresponding historical hoisting task descriptions, and the corresponding historical semantic embedding vectors of the tower crane construction site environment; 4) Reconstruct the target historical tower crane three-axis linkage path trajectory based on the tower crane's equipment parameters, semantic embedding vectors, and hoisting task description to generate the target tower crane three-axis linkage path trajectory corresponding to the hoisting task description. This application embodiment converts a 3D map of the tower crane construction site environment into a semantic embedding vector, which captures the complex environmental features of the construction site. Combined with the user-input lifting task description, it can clarify the specific objectives and path planning requirements of the lifting task, thereby realizing personalized customization of the tower crane's three-axis linkage path trajectory. It retrieves the target's historical tower crane three-axis linkage path trajectory from the trajectory knowledge base, and reconstructs the target's historical tower crane three-axis linkage path trajectory by integrating tower crane equipment parameters, semantic embedding vectors, and lifting task descriptions, generating a target tower crane three-axis linkage path trajectory adapted to the current task. It integrates historical experience and user preferences, and optimizes the generation based on typical paths in similar environments. Compared with traditional tower crane three-axis linkage path planning methods based on fixed optimization objectives, it integrates the "semantic retrieval + path generation" mechanism in the RAG-like architecture, which can generate personalized paths that are closer to user needs, improve the flexibility of tower crane three-axis linkage path planning, and adapt to the operational needs of multiple tasks and multiple scenarios.

[0077] In some embodiments, the target historical three-axis linkage path trajectory of the tower crane is retrieved from the trajectory knowledge base based on the hoisting task description and semantic embedding vector, including: Aligning the hoisting task description with the semantic embedding vector; including: constructing a Prompt based on the hoisting task description and inputting it into a large language model to obtain the structured information of the hoisting task description; the structured information includes path planning objectives, path optimization indicators, constraints and environmental characteristics; The Prompt is rewritten based on structured information to obtain optimized knowledge base retrieval query information; the knowledge base retrieval query information includes environmental characteristics and path optimization indicators; The target historical tower crane three-axis linkage path trajectory is retrieved from the trajectory knowledge base by retrieving query information and semantic embedding vectors from the knowledge base.

[0078] In this embodiment, to improve the accuracy of retrieval in the trajectory knowledge base, the unstructured hoisting task description input by the user can be aligned with the semantic embedding vector representing the current work site. Therefore, a Prompt is constructed, the core tasks of which are: aligning with the user's input requirements (goals, constraints, scenarios); combining the characteristics of the site environment (maps, obstacles, dynamic information); and mapping to path planning characteristics (algorithms, path quality, execution constraints); thereby maximizing "relevance" and "executability" in the retrieval and generation process. For example: Input Prompt for Large Language Model (LLM): You are a path planning requirement parser. The user will provide natural language input, i.e., a description of the hoisting task, such as "Please plan a safe path to the target point, avoiding obstacles." Please parse the input into the following structured information: Planning objectives: Describe the target location or area that the user hopes to reach.

[0079] Path optimization metrics: shortest distance / fastest time / minimum energy consumption / maximum safety / smoothness.

[0080] Constraints: obstacle avoidance requirements, speed limits, joint range, and dynamic constraints.

[0081] Environmental characteristics: map dimension (2D / 3D), obstacle type (static / dynamic), terrain features.

[0082] RAG (Retrieval-Augmented Generation) retrieval rewriting prompt process: Based on the above analysis results, i.e., structured information, an optimized knowledge base retrieval query is generated, emphasizing technical knowledge related to "path planning algorithms, environmental constraints, and planning cases." Please ensure: The query contains keywords related to environmental characteristics.

[0083] The query contains keywords related to path optimization metrics.

[0084] If the user does not explicitly specify their planning preferences, the system will automatically add a "comparison of commonly used path planning algorithms".

[0085] Output format: A search query string.

[0086] Finally, the user-inputted path planning requirements are transformed into specific goals and constraints. Combined with semantic embedding vectors, the historical three-axis linkage path trajectory of the target tower crane is retrieved from the trajectory knowledge base. For example, the knowledge base retrieval query information and semantic embedding vectors can be input into a large language model to obtain the historical three-axis linkage path trajectory of the target tower crane.

[0087] In this embodiment, a Prompt is constructed based on the hoisting task description and input into a large language model to obtain structured information containing path planning objectives, path optimization indicators, constraints, and environmental characteristics. This aligns the task description with scene and path features, thereby maximizing "relevance" and "executability" during retrieval and generation. As a result, the target historical tower crane three-axis linkage path trajectory that matches the current task requirements can be matched, improving the accuracy of retrieval.

[0088] In some embodiments, before converting a 3D environmental map of the tower crane construction site environment into a semantic embedding vector, the following is included: Acquire 3D point cloud data of the tower crane construction site environment; An octree is initialized with the spatial range of the tower crane construction site environment as the root node. The nodes of the octree are recursively divided, and the 3D point cloud data is mapped to the voxel nodes of the octree to obtain a global octree map. The global octree map is used as a 3D environment map of the tower crane construction site.

[0089] In this embodiment, sensors can scan the physical space of the tower crane construction site environment, converting various spatial points on the site into a digital point set containing three-dimensional coordinates, i.e., three-dimensional point cloud data. This three-dimensional point cloud data can capture the geometry and spatial distribution of the tower crane construction site environment with high precision, including the location of building frames, scaffolding, material storage areas, and other construction equipment such as tower cranes and hoists. The three-dimensional point cloud data can be acquired using laser scanners or stereo vision systems. For example, on a construction site, a laser scanner can quickly scan the entire construction area, generating three-dimensional point cloud data containing millions of points.

[0090] The dynamic nature of the construction site environment increases the complexity of data collection. For moving elements such as mixer trucks and workers, temporal point cloud analysis can be used to eliminate transient interference (such as the location where workers briefly stop) and retain relatively stable static features (such as piles of steel bars and scaffolding). For areas that change with the construction progress (such as newly added walls each day), the 3D point cloud data can be updated regularly.

[0091] While 3D point cloud data is detailed, its sheer volume (often millions or even tens of millions of points) and lack of hierarchical structure make it difficult to use directly for path planning and semantic analysis. Therefore, an octree algorithm can be used to structure the 3D point cloud data, generating a global octree map, which can then be used as a 3D environmental map of the tower crane construction site.

[0092] Octree mapping is a sparse 3D voxel map structure. Compared to 3D point cloud data, it retains the overall 3D structural information while effectively reducing information redundancy, thus significantly reducing the computational load of the subsequent overall system. The core idea of ​​octree mapping is to decompose a complex environment into a series of variable-sized cubic units, i.e., voxel nodes, through recursive partitioning of 3D space. It records the attributes of these voxel nodes, such as their 3D spatial coordinates, occupancy status (whether they are occupied by objects, object type, etc.), voxel size (resolution), partitioning depth, and topological adjacency relationships between voxels.

[0093] The construction of an octree map begins with the definition of an initial bounding box for the entire target space. This initial bounding box needs to completely cover the maximum spatial range of the construction site environment, for example, a cube with dimensions of 0-100 meters east-west, 0-80 meters north-south, and 0-50 meters vertically, serving as the root node of the octree. Specifically, the spatial range of the tower crane construction site environment can be determined based on 3D point cloud data. This spatial range is then used as the root node to initialize the octree, with all 3D point cloud data residing within the root node of the octree.

[0094] The root node is refined through recursive partitioning: Based on information from the 3D point cloud data, the root node (cube) is uniformly divided into 8 equal-sized sub-cubes along the midpoints of the X, Y, and Z coordinate axes, i.e., 8 child nodes. The partitioning process dynamically stops based on environmental complexity, and each child node repeats this process until the number of points in the node is below a threshold or a preset maximum partitioning depth is reached (e.g., 10 layers, corresponding to a minimum voxel size of approximately 0.1 meters). For example, a dense point cloud (500 points) containing a pile of rebar within the root node will be divided into 8 child nodes. The child nodes covering the rebar pile will continue to be partitioned due to the density of the point cloud, while the child nodes in the open area will stop being partitioned due to the sparse point cloud. Each node that is no longer partitioned is a voxel node.

[0095] Global octree maps eliminate redundant point cloud information through voxelization. For example, in a 10-cubic-meter space, the original 3D point cloud data may contain 1,000 points, while the octree map only needs to use multiple voxels and their attributes to represent them, reducing the amount of data by more than 90% and significantly reducing the computational load during subsequent semantic embedding vector conversion.

[0096] In this embodiment, by using octrees to process 3D point cloud data, the construction site space can be divided and managed in layers. Massive point cloud data is mapped into voxel nodes in an orderly manner, preserving the 3D spatial features of the environment. Furthermore, voxelization simplifies the data structure, reduces the complexity of subsequent data processing, and reduces the consumption of computing resources. Moreover, it enables the subsequent path trajectory generation process to perceive high-rise features in the environmental space, adapt to different construction site conditions, and achieve stronger environmental perception and path strategy transfer capabilities.

[0097] In some embodiments, converting a 3D environmental map of the tower crane construction site into a semantic embedding vector includes: Divide the global octree map into multiple local sub-regions; Multi-dimensional features are extracted from local sub-regions to obtain local semantic embedding vectors for the local sub-regions; the multi-dimensional features include at least one of the following: density distribution of voxel nodes, spatial traversable direction, vertical / horizontal openness, and gradient of voxel nodes. The semantic embedding vectors of each local sub-region are aggregated to obtain the semantic embedding vector.

[0098] In this embodiment, although the global octree map can cover the tower crane construction site environment, directly extracting semantic features from the global octree map can easily lead to the dilution of detailed features due to its large scope. For example, a global map containing material areas, construction areas, and passageways may not accurately capture local features such as "narrow passageways at the edges of the material area" or "open platforms inside the construction area" if processed as a whole. Therefore, the global octree map can be divided into multiple local sub-regions for feature extraction. In this embodiment, different partitioning methods can be used. One approach is uniform partitioning based on spatial scale, for example, dividing the global map into 5m×5m×5m cube grids, with each grid serving as a local sub-region, suitable for scenarios with relatively uniform distribution of environmental features. Another approach is adaptive partitioning based on the octree node hierarchy, selecting nodes containing complete semantic units from the middle-level nodes of the global octree map, such as layers 4-6, as local sub-regions.

[0099] After dividing the local sub-regions, multi-dimensional features can be extracted for each local sub-region. Multi-dimensional features may include, but are not limited to, the density distribution of voxel nodes, the traversable direction of space, the vertical / horizontal openness, and the gradient of voxel node changes. These features can describe the environmental characteristics of the local sub-regions from different perspectives.

[0100] The density distribution of voxel nodes describes the density of point cloud data in a local sub-region. The object density of a local sub-region can be determined by statistically analyzing the number, density, and distribution pattern of voxels. For example, the density distribution of voxel nodes can be low density, extremely low height, or uniform distribution, corresponding to a flat channel semantically.

[0101] The traversable directions in a space reflect which directions are traversable and which directions are obstructed within a local sub-region. This allows analysis of the feasibility of permitted crane hook movement directions within the local sub-region, such as the X, Y, and Z axes. For example, a traversable direction could be defined as traversable direction = vertically upwards, which semantically means vertical lifting is permitted only.

[0102] Vertical / horizontal openness measures the openness of a local sub-region in the vertical and horizontal directions. Vertical openness refers to the maximum unobstructed height (distance from the ground to the highest obstacle) in the Z-axis direction within the sub-region, while horizontal openness refers to the maximum continuous range of unobstructed area in the XY plane. For example, vertical / horizontal openness can be high vertical openness and medium horizontal openness, which corresponds to the semantics of being suitable for horizontal movement of tower cranes.

[0103] The gradient of a voxel node reflects the rate of change of point cloud data in a local sub-region. Regions with large gradients may indicate rapid changes in terrain or structure. The boundary sharpness of objects within a local sub-region, i.e., the gradient, can be determined by calculating the differences in attributes between adjacent voxels (such as height difference, density difference, etc.). For example, the gradient of a voxel node can be low, with blurred boundaries, corresponding to a continuous flat region.

[0104] In some embodiments, a pre-trained feature encoding model, such as a 3D feature extraction network based on CNN (Convolutional Neural Network), can be used to extract multi-dimensional features from local sub-regions to obtain a fixed-dimensional vector (e.g., 64-dimensional), which is the local semantic embedding vector.

[0105] In this embodiment, the local semantic embedding vectors of each local sub-region can be aggregated by weighted averaging, max pooling, or deep learning-based aggregation networks to obtain semantic embedding vectors that can describe the on-site environment of the tower crane construction site.

[0106] In this embodiment, by dividing local sub-regions and extracting multi-dimensional features in a targeted manner, the unique environmental attributes of different regions can be captured. Voxel density can reflect the density of obstacle distribution, the passable direction of space and vertical / horizontal openness can reflect the passability of the region, and the changing gradient can show the abrupt changes in the environment. The extraction of multi-dimensional features gives the local semantic embedding vector rich environmental semantic information. By aggregating the local vectors into the overall semantic embedding vector, the detailed features of the local sub-regions are preserved, and the expressive power of the semantic embedding vector is enhanced.

[0107] In some embodiments, multi-dimensional feature extraction is performed on local sub-regions to obtain local semantic embedding vectors for the local sub-regions, including: Extract the feature vectors of voxel nodes in a local sub-region; Construct an adjacency graph of the local sub-region based on the voxel nodes in the local sub-region; the adjacency graph includes nodes and edges, nodes correspond to voxel nodes of the local sub-region, edges indicate that the voxel nodes corresponding to two nodes are adjacent in physical space, and the node attributes of the nodes are the feature vectors of the voxel nodes. By using a graph neural network to extract multi-dimensional features from the adjacency graph, local semantic embedding vectors of local sub-regions are obtained.

[0108] In an octree map, each voxel node represents a 3D spatial unit within a local subregion. Voxel nodes contain rich geometric and spatial information, such as the density of the point cloud data, the location of the voxel node, and the environmental features surrounding the voxel node. For example, the information contained in each voxel node can be encoded as a feature vector:

[0109] in, Represents voxel nodes eigenvectors, Represents voxel nodes The three-dimensional spatial coordinates, Represents voxel nodes The probability of occupancy. Represents voxel nodes Size (resolution) Represents voxel nodes The depth of division, Represents voxel nodes The occupancy status is adopted. Encoding representation.

[0110] The feature vector of a single voxel can only reflect its own attributes, while the semantics of a local sub-region also require connecting the spatial relationships between voxels. Constructing an adjacency graph can connect these scattered voxel nodes to form a structured network that includes spatial relationships.

[0111] An adjacency graph is a graph structure used to represent the relationships between nodes, capturing the spatial adjacency relationships between voxel nodes. An adjacency graph consists of nodes and edges, where nodes correspond to voxel nodes in a local subregion, and edges indicate that two nodes are physically adjacent.

[0112] When constructing the adjacency graph G=(V,E), each voxel node in a local subregion can be considered as a node v in the adjacency graph. i ∈V, and the edges between nodes are determined based on the adjacency relationship of the voxel nodes in physical space. If two voxel nodes are spatially adjacent, for example, they share a face, an edge, or a vertex in three-dimensional space, then an edge e is established between these two nodes. ij ∈E, representing node v i With node v j Adjacency. The attributes of each node are the feature vectors of the corresponding voxel node. By constructing an adjacency graph, we not only preserve the feature information of the voxel nodes, but also introduce the spatial relationships between nodes.

[0113] After constructing the adjacency graph, a graph neural network (GNN) can be used to extract multi-dimensional features from the adjacency graph. A graph neural network is a deep learning model used to process graph-structured data, capable of capturing the relationships between nodes and extracting global semantic information.

[0114] Graph neural networks (Graph Neural Networks) generate updated feature vectors for each node by aggregating the feature information of nodes and their neighbors. During the processing of a Graph Neural Network, the network automatically learns features across multiple dimensions, including the density distribution of voxel nodes, spatial traversability, vertical / horizontal openness, and the gradient changes of voxel nodes. Through multiple layers of the Graph Neural Network, these features are progressively extracted and fused to generate local semantic embedding vectors for local sub-regions.

[0115] Specifically, in each layer of the GNN, each node collects the feature vectors of its neighboring nodes and merges them with its own features for updates. After multiple layers of message passing, the features of each node contain spatial association information of its surrounding neighbors and even the entire sub-region. Through global pooling operations, such as taking the average or maximum value of the features of all nodes, the information of the entire adjacency graph can be condensed into a fixed-length vector, namely the local semantic embedding vector.

[0116] In this embodiment, the basic attributes of each voxel node are captured by extracting the feature vectors of the voxel nodes, and an adjacency graph is constructed to represent the spatial relationship between voxel nodes in the form of a graph. This can effectively reflect the spatial topological information of the local sub-region. By processing the adjacency graph through a graph neural network, the powerful modeling capability of the graph neural network can be fully utilized to explore the attributes of the nodes in the local sub-region and their spatial connections with the surrounding nodes, so that the generated local semantic embedding vector can more accurately represent the environmental semantics of the local sub-region.

[0117] In some embodiments, the target historical tower crane three-axis linkage path trajectory is retrieved from the trajectory knowledge base based on the hoisting task description and semantic embedding vector, including: Calculate the similarity between the semantic embedding vector and each historical semantic embedding vector in the trajectory knowledge base, and determine the target historical semantic embedding vector whose similarity is greater than the target similarity. Calculate the similarity between the hoisting task description and the target historical semantic embedding vector for each historical hoisting task description, and determine the historical tower crane three-axis linkage path trajectory corresponding to the historical hoisting task description with the highest similarity as the target historical tower crane three-axis linkage path trajectory.

[0118] In this embodiment, all historical semantic embedding vectors stored in the trajectory knowledge base can be traversed. Similarity measurement methods, such as cosine similarity and Euclidean distance, are used to calculate the similarity between the current semantic embedding vector and historical semantic embedding vectors. To filter out the historical records most similar to the current tower crane construction site environment, a target similarity threshold can be set. When the similarity between a historical semantic embedding vector and the current semantic embedding vector is greater than the target similarity threshold, the historical semantic embedding vector is determined as the target historical semantic embedding vector. For example, if the target similarity threshold is 0.8, and the current semantic embedding vector has a similarity of 0.85 with historical semantic embedding vector A and 0.75 with historical semantic embedding vector B, then historical semantic embedding vector A can be determined as the target historical semantic embedding vector. The historical records corresponding to the target historical semantic embedding vector in the trajectory knowledge block are then retrieved.

[0119] Since the target historical semantic embedding vector may include multiple historical records, that is, multiple different historical hoisting task descriptions, it is possible to further determine the fit between the current hoisting task description and the historical hoisting task description, and find the historical tower crane three-axis linkage path trajectory that is most similar to the tower crane construction site environment and best fits the current task intent.

[0120] Specifically, it is possible to traverse all historical hoisting task descriptions corresponding to the target historical semantic embedding vector, and calculate the similarity between the current hoisting task description and the historical hoisting task description using similarity measurement methods such as cosine similarity and Euclidean distance. The historical tower crane three-axis linkage path trajectory corresponding to the historical hoisting task description with the highest similarity is determined as the target historical tower crane three-axis linkage path trajectory.

[0121] In this embodiment, by first filtering historical cases with similar environmental features based on environmental semantic similarity, the search scope is narrowed and the search efficiency is improved. Then, the description of the hoisting task is further matched from the filtering results, so that the environment corresponding to the selected historical tower crane three-axis linkage path trajectory is similar to the current work site environment and matches the current task, thereby improving the relevance of the search results.

[0122] In some embodiments, the target historical tower crane three-axis linkage path trajectory is reconstructed based on the tower crane's equipment parameters, semantic embedding vector, and hoisting task description to generate the target tower crane three-axis linkage path trajectory corresponding to the hoisting task description, including: The equipment parameters, semantic embedding vectors, lifting task description, and target historical tower crane three-axis linkage path trajectory of the tower crane are input into the path trajectory generation model to obtain the target tower crane three-axis linkage path trajectory corresponding to the lifting task description output by the generation model.

[0123] In this embodiment, the path trajectory generation model is a deep learning model trained on a large amount of historical data. Its core function is to simulate the decision-making process of human operators adjusting the path based on the environment, equipment, and needs, and to transform scattered input elements into a coherent path trajectory.

[0124] For example, a large amount of training data can be collected, including tower crane equipment parameters, semantic embedding vectors of the tower crane construction site environment, lifting task descriptions, and historical tower crane three-axis linkage path trajectories. This training data can be cleaned, normalized, and format converted, and the historical tower crane three-axis linkage path trajectories can be used as labels to construct a training dataset.

[0125] The path trajectory generation model can employ an encoder-decoder architecture. The encoder portion can be designed with an environment encoder (processing semantic embedding vectors), an equipment encoder (processing the tower crane's physical parameters), and a task encoder (processing the lifting task description), fusing multi-source information through an attention mechanism. The decoder portion, based on the fused features and combined with the structural information of historical tower crane three-axis linkage path trajectories, progressively generates new path trajectory points and outputs motion parameters such as velocity and acceleration. The path trajectory generation model can also include a constraint module, using built-in physical constraints (such as maximum lifting height) and safety constraints (such as obstacle avoidance distance) to ensure that the generated trajectory conforms to actual operational requirements.

[0126] The training process of the path trajectory generation model can combine supervised learning and reinforcement learning. In the supervised learning phase, historical three-axis linkage path trajectories of tower cranes are used as labels. The goal is to minimize the error between the path trajectory generated by the model and the corresponding real path trajectory, allowing the model to learn path generation patterns from historical experience. In the reinforcement learning phase, a reward function can be designed, comprehensively considering indicators such as path trajectory safety (e.g., distance to obstacles), efficiency (e.g., path length), and stability (e.g., acceleration changes). The parameters of the path trajectory generation model are optimized using a policy gradient algorithm.

[0127] The input to the path trajectory generation model is the tower crane's equipment parameters, semantic embedding vectors, lifting task description, and historical tower crane three-axis linkage path trajectories. The output of the path trajectory generation model can be a specific tower crane three-axis linkage path trajectory, including information such as the coordinates, direction of movement, and speed of each point on the tower crane's three-axis linkage path trajectory.

[0128] In this embodiment, by integrating multi-dimensional information such as tower crane equipment parameters, semantic embedding vectors, hoisting task descriptions, and historical tower crane three-axis linkage path trajectories, the path trajectory generation model can fully draw on the experience of historical tower crane three-axis linkage path trajectories when reconstructing the trajectory. It can also combine the tower crane's own equipment parameters to ensure that the trajectory conforms to the equipment performance. Furthermore, it can adapt the semantic embedding vectors to the characteristics of the tower crane construction site environment, generating a target tower crane three-axis linkage path trajectory that meets user preferences. Thus, it can provide diverse and strategy-rich path results while meeting safety constraints, adapting to the operational needs of multiple tasks and scenarios.

[0129] In some embodiments, the method further includes: A multi-dimensional evaluation of the target tower crane's three-axis linkage path trajectory is conducted to obtain the first evaluation result; the multi-dimensional evaluation includes at least one of structural rationality evaluation, safety evaluation, and execution stability evaluation; The difference index between the target tower crane's three-axis linkage path trajectory and the target tower crane's historical three-axis linkage path trajectory in the trajectory knowledge base is calculated to obtain the second evaluation result; the difference index includes at least one of the following: spatial difference between trajectory points, velocity curve difference, and embedding vector difference. A quality score for the target tower crane's three-axis linkage path trajectory is generated based on the first and second evaluation results. If the quality score is greater than the target score threshold, the target tower crane three-axis linkage path trajectory, the corresponding hoisting task description, and the semantic embedding vector are associated and stored in the trajectory knowledge base.

[0130] In this embodiment, after generating the target tower crane's three-axis linkage path trajectory, the reliability of the target tower crane's three-axis linkage path trajectory can be verified through multi-dimensional evaluation. Then, based on the evaluation results, it can be decided whether to include the target tower crane's three-axis linkage path trajectory in the trajectory knowledge base, forming a closed-loop system of "generation-evaluation-iteration".

[0131] In this embodiment, the multi-dimensional assessment may include structural rationality assessment, safety assessment, and execution stability assessment. The structural rationality assessment judges the feasibility of the trajectory from the kinematics perspective of the tower crane and verifies whether the trajectory conforms to the mechanical characteristics of the tower crane's three-axis linkage, namely hoisting, luffing, and slewing.

[0132] Safety assessments can be performed on a 3D environment reconstructed from semantically embedded vectors to verify whether the target tower crane's three-axis linkage path trajectory avoids all obstacles and maintains a safe distance. Stability assessments can analyze the smoothness of the trajectory's velocity and acceleration curves, reducing the risk of swaying of the hoisted object or equipment damage caused by violent movements.

[0133] Through multi-dimensional evaluation, a quantitative first evaluation result can be generated, such as the weighted sum of the scores of each item. For example, if the structural rationality score of the target tower crane's three-axis linkage path trajectory is 90 points, the safety score is 95 points, and the stability score is 85 points, the first evaluation result after weighted calculation is 90 points (after weighted calculation), indicating that the intrinsic quality of the target tower crane's three-axis linkage path trajectory is excellent.

[0134] In this embodiment, the difference index between the target tower crane's three-axis linkage path trajectory and the target tower crane's historical three-axis linkage path trajectory in the trajectory knowledge base can also be calculated to evaluate the similarity and difference between the newly generated path trajectory and the historical tower crane's three-axis linkage path trajectory, thereby determining the innovation and practicality of the target tower crane's three-axis linkage path trajectory.

[0135] Difference indicators can include spatial differences between trajectory points, velocity curve differences, and embedding vector differences. Spatial differences between trajectory points measure the spatial similarity between the target tower crane's three-axis linkage path trajectory and historical tower crane three-axis linkage path trajectories. This is obtained by calculating the spatial distance between corresponding points on the two paths. Smaller spatial differences indicate that the two paths are spatially close, while larger spatial differences indicate significant differences between the paths.

[0136] The speed curve difference measures the similarity in speed changes between the target tower crane's three-axis linkage path trajectory and the historical tower crane's three-axis linkage path trajectory. By comparing the speed curves of the two paths, it can be assessed whether the speed changes during the motion are consistent.

[0137] The embedding vector difference index measures the similarity between the target tower crane's three-axis linkage path trajectory and the historical tower crane's three-axis linkage path trajectory in the semantic embedding space. By calculating the similarity between the semantic embedding vectors of the two paths, the semantic similarity between the two paths can be assessed. Based on the calculation results of the difference index, a quantitative second evaluation result can be generated, such as the weighted sum of the scores of each item.

[0138] The quality score comprehensively considers the first assessment results from multi-dimensional evaluation and the second assessment results from the difference indicators. By setting different weights, the importance of each assessment dimension can be adjusted according to actual needs. For example, if safety is the most important consideration, a higher weight can be assigned to the safety assessment. The quality score can be calculated using a weighted average, a comprehensive scoring formula, or other statistical methods. A higher quality score indicates that the target tower crane's three-axis linkage path trajectory performs well in terms of structural rationality, safety, and operational stability, and shows appropriate differences from historical tower crane three-axis linkage path trajectories, demonstrating a certain degree of innovation and practicality.

[0139] After generating a quality score for the target tower crane's three-axis linkage path trajectory, if the quality score is greater than a preset target score threshold, it indicates that the target tower crane's three-axis linkage path trajectory has high quality. The target tower crane's three-axis linkage path trajectory and its related information can be stored in a trajectory knowledge base for future reference. The target score threshold is a preset threshold used to determine whether the quality of the target tower crane's three-axis linkage path trajectory meets acceptable standards.

[0140] The target tower crane's three-axis linkage path trajectory and related information are stored in the trajectory knowledge base. The specific storage content may include: the complete coordinate sequence and speed curve of the target tower crane's three-axis linkage path trajectory; the corresponding hoisting task description; and the semantic embedding vector when generating the target tower crane's three-axis linkage path trajectory. This information is stored in the trajectory knowledge base in association. The trajectory knowledge base continuously accumulates high-quality data, enriching its content.

[0141] In this embodiment, by conducting structural rationality assessment, safety assessment, and execution stability assessment on the target tower crane's three-axis linkage path trajectory, the feasibility and reliability of the path trajectory in practical applications can be ensured from different perspectives. By calculating the difference index between the target tower crane's three-axis linkage path trajectory and the target tower crane's historical three-axis linkage path trajectories in the trajectory knowledge base, the innovativeness and adaptability of the path are further analyzed, helping to assess whether the path can better adapt to the specific needs of the current task. The quality score generated by combining the first and second assessment results provides a quantitative indicator for the quality of the path trajectory, making the advantages and disadvantages of the path intuitively apparent. When the quality score exceeds the target score threshold, the target tower crane's three-axis linkage path trajectory and its related information are stored in the trajectory knowledge base, thereby continuously optimizing the content of the trajectory knowledge base and providing a better historical reference for subsequent path planning.

[0142] In some embodiments, the method further includes: Obtain the three-axis linkage path trajectory of the target tower crane with a quality score greater than the target score threshold and the three-axis linkage path trajectory of the target tower crane with a quality score less than or equal to the target score threshold; Construct positive and negative sample sets based on the target tower crane three-axis linkage path trajectory with a quality score greater than the target score threshold and the target tower crane three-axis linkage path trajectory with a quality score less than or equal to the target score threshold; The path trajectory generation model is fine-tuned and trained using positive and negative sample sets.

[0143] Specifically, a target score threshold can be used to collect positive and negative sample sets. The positive sample set can include target tower crane three-axis linkage path trajectories with quality scores greater than the target score threshold. These target tower crane three-axis linkage path trajectories can be considered high-quality examples that can provide excellent path trajectory examples for the path trajectory generation model, helping the model learn how to generate high-quality path trajectories.

[0144] The negative sample set can include target tower crane three-axis linkage path trajectories with quality scores less than or equal to the target score threshold. These target tower crane three-axis linkage path trajectories can be considered to have problems in some aspects, such as insufficient safety, unstable execution, or unreasonable structure. The negative sample set provides examples of path trajectories that need improvement for the path trajectory generation model, helping the model learn how to avoid generating low-quality path trajectories.

[0145] In this embodiment, the semantic embedding vector corresponding to the three-axis linkage path trajectory of the target tower crane and the description of the hoisting task can be used as a sample. When constructing the positive and negative sample set, each sample can also be labeled to clarify whether the sample is a positive sample or a negative sample.

[0146] To achieve the goal of fine-tuning training, a suitable loss function needs to be designed. This loss function can include classification loss and feature matching loss. Classification loss can employ cross-entropy loss, which measures the difference between the probability of the path trajectory generation model generating a high-quality path trajectory and the true label. Feature matching loss can be the similarity between the features of the path trajectory generated by the path trajectory generation model (such as velocity curves and embedding vectors) and the features of positive samples. By minimizing the difference, it guides the path trajectory generation model to learn patterns of high-quality path trajectories. For example, if the velocity curve of a positive sample is smooth and without abrupt changes, while the trajectory generated by the path trajectory generation model exhibits drastic velocity fluctuations, the feature matching loss will prompt the path trajectory generation model to optimize its velocity planning strategy.

[0147] It should be noted that the fine-tuning training process is an incremental learning of the path trajectory generation model. The fine-tuning training process starts from the currently used path trajectory generation model, rather than re-initializing the parameters of the path trajectory generation model.

[0148] In this embodiment, by using the discriminative nature of quality scoring, high-quality path trajectories are used as positive samples and low-quality path trajectories as negative samples. This enables the trajectory generation model to learn the characteristics and patterns of high-quality path trajectories, identify and correct problems in low-quality path trajectories, and allow the trajectory generation model to continuously optimize its trajectory generation logic. This gradually improves the ability to generate high-quality trajectories under different working conditions, reduces the output of unqualified trajectories, and achieves self-optimization and long-term accumulation and evolution of path quality.

[0149] The method for generating a three-axis linkage trajectory for a smart tower crane with a RAG-like structure provided in this application can be executed by a device for generating a three-axis linkage trajectory for a smart tower crane with a RAG-like structure. This application uses the execution of the method by the device for generating a three-axis linkage trajectory for a smart tower crane with a RAG-like structure as an example to illustrate the device for generating a three-axis linkage trajectory for a smart tower crane with a RAG-like structure provided in this application.

[0150] This application also provides a smart tower crane three-axis linkage trajectory generation device with a RAG-like structure.

[0151] like Figure 2 As shown, the intelligent tower crane three-axis linkage trajectory generation device of this type of RAG structure includes: The conversion module 210 is used to convert a 3D environment map of the tower crane construction site into a semantic embedding vector; the 3D environment map includes the spatial features of the tower crane and obstacle features; The acquisition module 220 is used to acquire the lifting task description input by the user; the lifting task description is used to describe the lifting task objectives and the tower crane's three-axis linkage path planning requirements; The retrieval module 230 is used to retrieve the target historical tower crane three-axis linkage path trajectory from the trajectory knowledge base based on the hoisting task description and semantic embedding vector; the trajectory knowledge base stores the historical tower crane three-axis linkage path trajectory during the execution of historical tower crane tasks, the corresponding historical hoisting task description, and the corresponding historical semantic embedding vector of the tower crane construction site environment. The generation module 240 is used to reconstruct the target historical tower crane three-axis linkage path trajectory based on the tower crane's equipment parameters, semantic embedding vector, and hoisting task description, and generate the target tower crane three-axis linkage path trajectory corresponding to the hoisting task description.

[0152] According to the RAG-like intelligent tower crane three-axis linkage trajectory generation device of this application, the following steps are taken: 1. Convert a three-dimensional environment map of the tower crane construction site environment into a semantic embedding vector; the three-dimensional environment map includes the spatial features of the tower crane and obstacle features; 2. Obtain a user-inputted hoisting task description; the hoisting task description describes the hoisting task objective and the tower crane three-axis linkage path planning requirements; 3. Retrieve the target historical tower crane three-axis linkage path trajectory from the trajectory knowledge base based on the hoisting task description and the semantic embedding vector; the trajectory knowledge base stores historical tower crane three-axis linkage path trajectories during the execution of historical tower crane tasks, the corresponding historical hoisting task descriptions, and the corresponding historical semantic embedding vectors of the tower crane construction site environment; 4. Reconstruct the target historical tower crane three-axis linkage path trajectory based on the tower crane's equipment parameters, semantic embedding vectors, and hoisting task description to generate the target tower crane three-axis linkage path trajectory corresponding to the hoisting task description. This application embodiment converts a 3D map of the tower crane construction site environment into a semantic embedding vector, which captures the complex environmental features of the construction site. Combined with the user-input lifting task description, it can clarify the specific objectives and path planning requirements of the lifting task, thereby realizing personalized customization of the tower crane's three-axis linkage path trajectory. It retrieves the target's historical tower crane three-axis linkage path trajectory from the trajectory knowledge base, and reconstructs the target's historical tower crane three-axis linkage path trajectory by integrating tower crane equipment parameters, semantic embedding vectors, and lifting task descriptions, generating a target tower crane three-axis linkage path trajectory adapted to the current task. It integrates historical experience and user preferences, and optimizes the generation based on typical paths in similar environments. Compared with traditional tower crane three-axis linkage path planning methods based on fixed optimization objectives, it integrates the "semantic retrieval + path generation" mechanism in the RAG-like architecture, which can generate personalized paths that are closer to user needs, improve the flexibility of tower crane three-axis linkage path planning, and adapt to the operational needs of multiple tasks and multiple scenarios.

[0153] In some embodiments, the retrieval module 230 is further configured to: Aligning the hoisting task description with the semantic embedding vector; including: constructing a Prompt based on the hoisting task description and inputting it into a large language model to obtain the structured information of the hoisting task description; the structured information includes path planning objectives, path optimization indicators, constraints and environmental characteristics; The Prompt is rewritten based on structured information to obtain optimized knowledge base retrieval query information; the knowledge base retrieval query information includes environmental characteristics and path optimization indicators; The target historical tower crane three-axis linkage path trajectory is retrieved from the trajectory knowledge base by retrieving query information and semantic embedding vectors from the knowledge base.

[0154] In some embodiments, the conversion module 210 is further configured to: Acquire 3D point cloud data of the tower crane construction site environment; An octree is initialized with the spatial range of the tower crane construction site environment as the root node. The nodes of the octree are recursively divided, and the 3D point cloud data is mapped to the voxel nodes of the octree to obtain a global octree map. The global octree map is used as a 3D environment map of the tower crane construction site.

[0155] In some embodiments, the conversion module 210 is further configured to: Divide the global octree map into multiple local sub-regions; Multi-dimensional features are extracted from local sub-regions to obtain local semantic embedding vectors for the local sub-regions; the multi-dimensional features include at least one of the following: density distribution of voxel nodes, spatial traversable direction, vertical / horizontal openness, and gradient of voxel nodes. The semantic embedding vectors of each local sub-region are aggregated to obtain the semantic embedding vector.

[0156] In some embodiments, the conversion module 210 is further configured to: Extract the feature vectors of voxel nodes in a local sub-region; Construct an adjacency graph of the local sub-region based on the voxel nodes in the local sub-region; the adjacency graph includes nodes and edges, nodes correspond to voxel nodes of the local sub-region, edges indicate that the voxel nodes corresponding to two nodes are adjacent in physical space, and the node attributes of the nodes are the feature vectors of the voxel nodes. By using a graph neural network to extract multi-dimensional features from the adjacency graph, local semantic embedding vectors of local sub-regions are obtained.

[0157] In some embodiments, the retrieval module 230 is further configured to: Calculate the similarity between the semantic embedding vector and each historical semantic embedding vector in the trajectory knowledge base, and determine the target historical semantic embedding vector whose similarity is greater than the target similarity. Calculate the similarity between the hoisting task description and the target historical semantic embedding vector for each historical hoisting task description, and determine the historical tower crane three-axis linkage path trajectory corresponding to the historical hoisting task description with the highest similarity as the target historical tower crane three-axis linkage path trajectory.

[0158] In some embodiments, the generation module 240 is further configured to: The equipment parameters, semantic embedding vectors, lifting task description, and target historical tower crane three-axis linkage path trajectory of the tower crane are input into the path trajectory generation model to obtain the target tower crane three-axis linkage path trajectory corresponding to the lifting task description output by the generation model.

[0159] In some embodiments, the intelligent tower crane three-axis linkage trajectory generation device with a RAG-like structure may further include a quality assessment module for: A multi-dimensional evaluation of the target tower crane's three-axis linkage path trajectory is conducted to obtain the first evaluation result; the multi-dimensional evaluation includes at least one of structural rationality evaluation, safety evaluation, and execution stability evaluation; The difference index between the target tower crane's three-axis linkage path trajectory and the target tower crane's historical three-axis linkage path trajectory in the trajectory knowledge base is calculated to obtain the second evaluation result; the difference index includes at least one of the following: spatial difference between trajectory points, velocity curve difference, and embedding vector difference. A quality score for the target tower crane's three-axis linkage path trajectory is generated based on the first and second evaluation results. If the quality score is greater than the target score threshold, the target tower crane three-axis linkage path trajectory, the corresponding hoisting task description, and the semantic embedding vector are associated and stored in the trajectory knowledge base.

[0160] In some embodiments, the quality assessment module is further configured to: Obtain the three-axis linkage path trajectory of the target tower crane with a quality score greater than the target score threshold and the three-axis linkage path trajectory of the target tower crane with a quality score less than or equal to the target score threshold; Construct positive and negative sample sets based on the target tower crane three-axis linkage path trajectory with a quality score greater than the target score threshold and the target tower crane three-axis linkage path trajectory with a quality score less than or equal to the target score threshold; The path trajectory generation model is fine-tuned and trained using positive and negative sample sets.

[0161] The intelligent tower crane three-axis linkage trajectory generation device with a RAG-like structure in this application embodiment can be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the specific type of device.

[0162] The intelligent tower crane three-axis linkage trajectory generation device with a RAG-like structure in this application embodiment can be a device with an operating system. This operating system can be a Microsoft (Windows) operating system, an Android operating system, an iOS operating system, or other possible operating systems; this application embodiment does not specifically limit it.

[0163] In some embodiments, such as Figure 3 As shown, this application embodiment also provides an electronic device 300, including a processor 301, a memory 302, and a computer program stored in the memory 302 and executable on the processor 301. When the program is executed by the processor 301, it implements the various processes of the above-described embodiment of the intelligent tower crane three-axis linkage trajectory generation method for RAG-like structures and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0164] It should be noted that the electronic devices in the embodiments of this application include the aforementioned mobile electronic devices and non-mobile electronic devices.

[0165] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described embodiment of the intelligent tower crane three-axis linkage trajectory generation method for RAG-like structures and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0166] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0167] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method for generating the three-axis linkage trajectory of an intelligent tower crane with a RAG-like structure.

[0168] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0169] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described embodiment of the intelligent tower crane three-axis linkage trajectory generation method for RAG-like structures, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0170] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0171] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0172] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0173] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0174] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0175] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for generating a three-axis linkage trajectory of a smart tower crane with a RAG-like structure, characterized in that, include: Convert the 3D environmental map of the tower crane construction site into a semantic embedding vector; The three-dimensional environment map includes the spatial features of the tower crane and the features of obstacles; Obtain the lifting task description input by the user; the lifting task description is used to describe the lifting task objectives and the tower crane's three-axis linkage path planning requirements; The target historical tower crane three-axis linkage path trajectory is retrieved from the trajectory knowledge base based on the hoisting task description and the semantic embedding vector; the trajectory knowledge base stores the historical tower crane three-axis linkage path trajectory during the execution of historical tower crane tasks, the corresponding historical hoisting task description, and the corresponding historical semantic embedding vector of the tower crane construction site environment. Based on the tower crane's equipment parameters, the semantic embedding vector, and the hoisting task description, the target historical tower crane's three-axis linkage path trajectory is reconstructed to generate the target tower crane's three-axis linkage path trajectory corresponding to the hoisting task description.

2. The method according to claim 1, characterized in that, The step of retrieving the target historical three-axis linkage path trajectory of the tower crane from the trajectory knowledge base based on the hoisting task description and the semantic embedding vector includes: Aligning the hoisting task description with the semantic embedding vector includes: constructing a Prompt based on the hoisting task description and inputting it into a large language model to obtain structured information of the hoisting task description; the structured information includes path planning objectives, path optimization indicators, constraints, and environmental characteristics. The Prompt is rewritten based on the structured information to obtain optimized knowledge base retrieval query information; the knowledge base retrieval query information includes the environmental characteristics and the path optimization indicators. Based on the query information and semantic embedding vector retrieved from the knowledge base, the target historical tower crane three-axis linkage path trajectory is obtained from the trajectory knowledge base.

3. The method according to claim 1, characterized in that, Before converting the 3D environment map of the tower crane construction site into semantic embedding vectors, the following steps are included: Obtain the three-dimensional point cloud data of the tower crane construction site environment; An octree is initialized with the spatial range of the tower crane construction site environment as the root node. The nodes of the octree are recursively divided, and the three-dimensional point cloud data is mapped to the voxel nodes of the octree to obtain a global octree map. The global octree map is used as a three-dimensional environment map of the tower crane construction site.

4. The method according to claim 3, characterized in that, The process of converting a 3D environmental map of the tower crane construction site into a semantic embedding vector includes: The global octree map is divided into multiple local sub-regions; Multi-dimensional feature extraction is performed on the local sub-region to obtain the local semantic embedding vector of the local sub-region; the multi-dimensional features include at least one of the following: voxel node density distribution, spatial traversable direction, vertical / horizontal openness, and voxel node variation gradient. The semantic embedding vector is obtained by aggregating the local semantic embedding vectors of each local sub-region.

5. The method according to claim 4, characterized in that, The step of extracting multi-dimensional features from the local sub-region to obtain the local semantic embedding vector of the local sub-region includes: Extract the feature vectors of voxel nodes in the local sub-region; The adjacency graph of the local sub-region is constructed based on the voxel nodes in the local sub-region. The adjacency graph includes nodes and edges. The nodes correspond to the voxel nodes of the local sub-region, and the edges indicate that the voxel nodes corresponding to two nodes are adjacent in physical space. The node attributes of the nodes are the feature vectors of the voxel nodes. The local semantic embedding vector of the local sub-region is obtained by extracting multi-dimensional features from the adjacency graph using a graph neural network.

6. The method according to claim 1, characterized in that, The step of retrieving the target historical tower crane three-axis linkage path trajectory from the trajectory knowledge base based on the hoisting task description and the semantic embedding vector includes: Calculate the similarity between the semantic embedding vector and each historical semantic embedding vector in the trajectory knowledge base, and determine the target historical semantic embedding vector whose similarity is greater than the target similarity; Calculate the similarity between the hoisting task description and the target historical semantic embedding vector for each historical hoisting task description, and determine the historical tower crane three-axis linkage path trajectory corresponding to the historical hoisting task description with the highest similarity as the target historical tower crane three-axis linkage path trajectory.

7. The method according to claim 1, characterized in that, The process of reconstructing the target historical tower crane's three-axis linkage path trajectory based on the tower crane's equipment parameters, the semantic embedding vector, and the lifting task description, to generate the target tower crane's three-axis linkage path trajectory corresponding to the lifting task description, includes: The equipment parameters of the tower crane, the semantic embedding vector, the hoisting task description, and the target historical tower crane three-axis linkage path trajectory are input into the path trajectory generation model to obtain the target tower crane three-axis linkage path trajectory corresponding to the hoisting task description output by the generation model.

8. The method according to claim 1, characterized in that, The method further includes: A multi-dimensional evaluation is performed on the three-axis linkage path trajectory of the target tower crane to obtain a first evaluation result; the multi-dimensional evaluation includes at least one of structural rationality evaluation, safety evaluation, and execution stability evaluation; The difference index between the target tower crane's three-axis linkage path trajectory and the target tower crane's historical three-axis linkage path trajectory in the trajectory knowledge base is calculated to obtain a second evaluation result; the difference index includes at least one of the following: spatial difference between trajectory points, velocity curve difference, and embedding vector difference. A quality score for the target tower crane's three-axis linkage path trajectory is generated based on the first evaluation result and the second evaluation result. If the quality score is greater than the target score threshold, the target tower crane three-axis linkage path trajectory, the hoisting task description corresponding to the target tower crane three-axis linkage path trajectory, and the semantic embedding vector are associated and stored in the trajectory knowledge base.

9. The method according to claim 8, characterized in that, The method further includes: Obtain the three-axis linkage path trajectory of the target tower crane with a quality score greater than the target score threshold and the three-axis linkage path trajectory of the target tower crane with a quality score less than or equal to the target score threshold; Construct positive and negative sample sets based on the target tower crane three-axis linkage path trajectory with a quality score greater than the target score threshold and the target tower crane three-axis linkage path trajectory with a quality score less than or equal to the target score threshold; The path trajectory generation model is fine-tuned and trained using the positive and negative sample sets.

10. A smart tower crane three-axis linkage trajectory generation device with a RAG-like structure, characterized in that, include: The conversion module is used to convert the 3D environment map of the tower crane construction site into semantic embedding vectors; The three-dimensional environment map includes the spatial features of the tower crane and the features of obstacles; The acquisition module is used to acquire the lifting task description input by the user; the lifting task description is used to describe the lifting task objectives and the tower crane's three-axis linkage path planning requirements; The retrieval module is used to retrieve the target historical tower crane three-axis linkage path trajectory from the trajectory knowledge base based on the hoisting task description and the semantic embedding vector; the trajectory knowledge base stores the historical tower crane three-axis linkage path trajectory during the execution of historical tower crane tasks, the corresponding historical hoisting task description, and the corresponding historical semantic embedding vector of the tower crane construction site environment. The generation module is used to reconstruct the target historical tower crane three-axis linkage path trajectory based on the tower crane's equipment parameters, the semantic embedding vector, and the hoisting task description, and generate the target tower crane three-axis linkage path trajectory corresponding to the hoisting task description.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1-9.

Images