An artificial intelligence-based method and system for sensing and optimally scheduling decisions in complex engineering environments

Abstract

The application provides a kind of based on artificial intelligence and realizes the perception and optimal decision scheduling method and system of complex engineering environment, it is related to computer vision technical field, by constructing the comprehensive decision model based on image processing, environment perception and prior knowledge, realize the autonomous optimization and decision of scheduling task in engineering project environment.The high based on artificial intelligence and realizes the perception and optimal decision scheduling method of complex engineering environment of the application specifically includes: collecting the panoramic image of engineering project site and obtaining project BIM data;Based on panoramic image and BIM data, the starting position candidate area weight graph in material scheduling process, material end position candidate area weight graph and space scheduling weight graph are constructed;According to starting position candidate area weight graph, material end position candidate area weight graph and space scheduling weight graph, the optimal scheduling decision is obtained;The optimal scheduling decision is sent to tower crane control room and ground control personnel.Through the above steps, the global, dynamic planning of engineering project material distribution task is realized.

Description

A method and system for perception and optimal decision-making scheduling in complex engineering environments based on artificial intelligence. Technical Field

[0001] This invention relates to the field of computer vision technology. Specifically, it relates to a method and system for perception and optimal decision-making scheduling in complex engineering environments based on artificial intelligence. By constructing a comprehensive decision-making model based on image processing, environmental perception, and prior knowledge, it achieves autonomous optimization and intelligent decision-making for scheduling tasks in complex and dynamic engineering project environments. Background Technology

[0002] Traditional construction projects, especially in the vertical transportation phase of large buildings, rely heavily on human experience and real-time on-site communication and coordination. Typically, on-site material dispatching is handled by the site foreman or signalman, who, based on the construction schedule and their personal experience, determines the temporary material storage areas on the ground and directs the tower crane operator to perform lifting operations via walkie-talkie or hand signals. The tower crane operator, in a high-altitude control room, relies on limited visibility and communication with ground personnel to complete a series of complex operations such as grabbing, lifting, rotating, translating, and placing materials. This process has many inherent drawbacks. As modern buildings become taller, larger, and more complex, and construction deadlines become increasingly tight, the traditional extensive management model relying on manual experience is no longer suitable for the refined, efficient, and safe requirements of modern construction, becoming a bottleneck restricting the overall efficiency of projects.

[0003] In recent years, artificial intelligence technologies, represented by deep learning, especially machine vision and image processing technologies, have demonstrated enormous application potential across various industries and successfully driven the automation and intelligent transformation of many fields. In the industrial manufacturing sector, vision technology is widely used for the automatic detection of product defects, achieving accuracy and efficiency far exceeding that of the human eye. In the security sector, facial recognition and behavior analysis technologies have built intelligent urban security systems. In the transportation sector, image recognition is one of the core technologies for achieving autonomous driving and intelligent traffic flow control. However, despite the enormous potential of this technology, its application in construction project management remains relatively rudimentary and superficial. Currently, the application of image technology on construction sites is mostly concentrated on the monitoring and identification of point-like, static, or simple behaviors. For example, deploying cameras at construction site entrances and exits to achieve facial recognition and identity verification of construction workers to strengthen access control management; or using video surveillance within the work area to automatically identify whether workers are wearing safety helmets, reflective vests, etc., as required, to conduct compliance supervision of safe production. While these applications have improved the safety management level of construction sites to some extent, they are essentially still in the category of post-event retrospective or passive supervision. They have not penetrated into the core processes of engineering project production, nor have they touched on how to use these advanced technologies to proactively optimize and make decisions on complex production scheduling issues.

[0004] In the entire construction project, the efficient, accurate, and safe delivery of materials is the lifeline for ensuring the project progresses according to plan. However, the selection of starting and ending points and the determination of delivery routes in material delivery scheduling are currently complex problems that rely on human estimation. A scientific method that can integrate multi-source information, perform quantitative analysis, and provide a globally optimal solution is lacking. Therefore, how to apply advanced sensing and intelligent decision-making technologies to this core aspect to achieve intelligent and optimized material scheduling throughout the entire process is a pressing technical challenge in this field.

[0005] In view of this, in order to overcome the shortcomings of the above-mentioned background technology, the present invention aims to propose an engineering project scheduling method and system that can deeply integrate machine vision, three-dimensional spatial modeling and decision-making algorithms, so as to realize global, dynamic and intelligent planning of tower crane material delivery tasks at construction sites. Summary of the Invention

[0006] This invention provides a method for perception and optimal decision-making scheduling in complex engineering environments based on artificial intelligence. The method specifically includes the following steps:

[0007] Collect panoramic images of the project site and obtain the project's BIM data;

[0008] Based on panoramic images and BIM data, a weighted map of candidate areas for starting position, a weighted map of candidate areas for ending position, and a weighted map of spatial scheduling are constructed during the material scheduling process.

[0009] The optimal scheduling decision is obtained based on the weight map of candidate regions for starting position, the weight map of candidate regions for material ending position, and the weight map of spatial scheduling.

[0010] The optimal scheduling decision is sent to the tower crane control room and ground control personnel.

[0011] This specification also proposes an artificial intelligence-based perception and optimal decision-making scheduling system for complex engineering environments, the device comprising:

[0012] Acquisition Module: The acquisition module acquires panoramic images of the project site and obtains the project's BIM data;

[0013] Weighted map generation module: Based on panoramic images and BIM data, constructs a weighted map of candidate areas for starting position, candidate areas for ending position, and spatial scheduling weighted map during the material scheduling process;

[0014] Scheduling Decision Module: The scheduling decision module obtains the optimal scheduling decision based on the weight map of candidate regions for starting position, candidate regions for material destination position, and spatial scheduling weight map.

[0015] The communication module sends the optimal scheduling decision to the tower crane control room and ground control personnel.

[0016] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the aforementioned artificial intelligence-based method for perception and optimal decision-making scheduling in complex engineering environments.

[0017] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned method for perception and optimal decision-making scheduling in complex engineering environments based on artificial intelligence.

[0018] This invention proposes an artificial intelligence-based method and system for perception and optimal decision-making scheduling in complex engineering environments. This invention achieves flexible, globally dynamic scheduling with variable start and end points, greatly enhancing adaptability to complex and changing construction site environments. Traditional scheduling heavily relies on fixed material storage and unloading areas; if these areas are occupied or have unfavorable conditions, scheduling becomes stagnant or inefficient. This invention completely breaks this constraint, treating both the start and end points as variables to be optimized. It can optimize the parking positions of material transport vehicles, dynamically and instantly finding the globally optimal start and end point positions by evaluating the weighted graph of the entire ground and floor plans.

[0019] Secondly, this invention elevates path planning from two-dimensional to three-dimensional optimization by constructing a three-dimensional voxel-based spatial risk map, and introduces quantitative analysis of environmental factors, achieving proactive safety assurance. By establishing a three-dimensional voxel weight map, this invention quantifies the risk of each voxel in the tower crane's workspace, particularly by extending the building structure and ground safety zone upwards to the no-fly zone, constructing an integrated air-ground three-dimensional protection network. Simultaneously, it transforms solar glare, a long-standing but difficult-to-quantify environmental factor that plagues operators, into predictable and avoidable three-dimensional weights. This enables the path decision-making system to proactively plan paths that avoid glare zones.

[0020] This invention abandons the traditional cost function with subjectively set parameters and adopts a purely data-driven decision model based on multi-objective evaluation, ensuring the robustness and global optimality of the final solution. Traditional optimization algorithms rely on manually set weighted cost functions with subjective preferences, resulting in fragile decision results and difficulty in proving their optimality. This invention employs a decision framework of feasible solution set generation + multi-objective performance vector evaluation + ranking optimization. It does not presuppose which is more important, safety or efficiency, but objectively calculates the performance of each feasible solution across five independent dimensions: starting point quality, ending point quality, path safety, path length, and scheduling complexity. Ultimately, by finding the solution closest to the virtual ideal optimal solution, this process is entirely data-driven, making its decision logic more transparent and robust. It can uncover the solution that achieves the best balance among multiple conflicting objectives, ensuring that the final output scheduling instruction is the globally optimal solution with the best overall performance across all dimensions. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 is a flowchart of the invention for sensing and optimal decision-making scheduling in complex engineering environments based on artificial intelligence. Detailed Implementation

[0023] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0024] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. This application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this application, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number and aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0026] Additionally, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that practice can be carried out without these specific details.

[0027] This specification presents an embodiment of a method for perception and optimal decision-making scheduling in complex engineering environments based on artificial intelligence. The method specifically includes the following steps:

[0028] Collect panoramic images of the project site and obtain the project's BIM data;

[0029] Based on panoramic images and BIM data, a weighted map of candidate areas for starting position, a weighted map of candidate areas for ending position, and a weighted map of spatial scheduling are constructed during the material scheduling process.

[0030] The optimal scheduling decision is obtained based on the weight map of candidate regions for starting position, the weight map of candidate regions for material ending position, the spatial scheduling weight map, and the comprehensive cost function.

[0031] The optimal scheduling decision is sent to the tower crane control room and ground control personnel.

[0032] In a preferred embodiment of the present invention, the artificial intelligence-based method for perception and optimal decision-making scheduling in complex engineering environments is not continuously running, but is triggered by a specific scheduling task instruction. This instruction marks the beginning of a specific material hoisting and delivery request.

[0033] Specifically, this dispatching task instruction can be manually initiated by authorized site management personnel (such as the construction team leader, material handler, or tower crane operator) through a dedicated user interface terminal (such as a tablet or industrial touchscreen) deployed at the construction site or tower crane control room. When initiating the instruction, core information for the task can be entered, such as the type of material to be dispatched and the target delivery floor. Upon receiving the instruction, the system will treat it as the starting signal for the process flow and immediately execute the first step, namely activating the image acquisition system to capture real-time panoramic images of the site and retrieving the corresponding BIM data, to ensure that all subsequent analyses and decisions are based on the latest and most accurate site conditions.

[0034] The panoramic images collected at the project site include:

[0035] At least one image acquisition unit is deployed on a tower crane, and the image acquisition unit is set to face the ground at a preset downward angle;

[0036] The slewing mechanism of the tower crane is controlled to rotate at a preset angular velocity, and the image acquisition unit is synchronously driven to acquire multiple image frames covering different azimuth angles during the rotation process.

[0037] Image registration is performed on the multiple image frames to stitch them together to generate a panoramic image.

[0038] Specifically, this embodiment deploys an image acquisition system integrated onto a tower crane, which serves as the primary dispatching equipment. Specifically, the image acquisition system preferably consists of 2 to 4 industrial-grade wide-angle cameras with 4K resolution and high dynamic range imaging capabilities. These cameras are mounted on a circular array bracket, which is fixed to the base of the tower crane's jib (lifting boom) near the tower body, or directly below the crane operator's control room. The circular array bracket rotates with the tower crane, ensuring that the camera's line of sight is not obstructed by the tower body or the jib itself. All cameras are oriented with their lenses pointing downwards at a preset viewing angle of 45 to 75 degrees to the horizontal plane, covering a broad annular area from near the tower crane's base to the edge of its maximum working radius.

[0039] In performing image acquisition tasks, this invention does not rely on a single shot, but generates a complete panoramic image through a dynamic, video stream-based image stitching process. After the acquisition process begins, the scheduling system sends instructions to the tower crane's control system, driving the crane's slewing mechanism to perform a complete 360-degree rotation at an extremely low and constant angular velocity, ranging from 0.5 to 1.0 degrees per second. During the rotation, all cameras mounted on the ring-shaped support are simultaneously activated, continuously recording video streams at a high frame rate. These video streams are transmitted in real-time to a central server at the construction site connected to the image acquisition system. Upon receiving the video streams, the server processes them, extracting keyframe images from the continuous video stream at fixed time intervals. The system performs preprocessing and registration stitching operations on the extracted keyframe image sequence. The preprocessing includes image correction, color and brightness correction, and white balance and exposure normalization for all keyframes. After preprocessing, this invention uses a Scale Invariant Feature Transform (SIFT) algorithm to detect and extract stable feature points in adjacent keyframe images. By comparing the descriptors of these feature points, feature point matching pairs are established between adjacent images. Based on the coordinate information of these matching pairs, the system uses random sampling consistency to calculate and determine an optimal homography matrix. By projecting all keyframes into a unified coordinate system in a chain according to the calculated transformation matrix, the system can achieve precise alignment of all images. A ring-shaped panoramic image is generated. This panoramic image, centered on the rotation center of the tower crane, fully displays all scene information within the working area, providing a high-quality, global data source for subsequent recognition work. This panoramic image includes the road layout of the project site, the planned static and dynamic safety zone boundaries, and the overall outline of the construction target building and the layout details of its currently under-construction top floor, such as open areas and occupied equipment or material storage areas.

[0040] In addition to acquiring real-time visual information from the site through an image acquisition system, this invention also includes in-depth utilization of the project's BIM (Building Information Modeling) data. In this embodiment, the central server of the scheduling method retrieves the Building Information Modeling file corresponding to the current construction progress from the project's local database. The file adopts the Industry Basic Class (IFC) standardized format. After obtaining the BIM data file, the data parsing and information extraction module processes it. This module parses the hierarchical structure of the BIM data file to obtain the current building geometry information, traverses the building component objects contained in the model, and, based on a set of preset filtering rules, filters out component categories that play a decisive role in structural stability. Preferably, it filters out structural component objects such as "IfcWall" (wall), "IfcBeam" (beam), "IfcColumn" (column), and "IfcSlab" (floor slab) as defined in the international classification standard. After filtering out structural components, the system further queries and extracts the attribute information of these components to accurately identify the load-bearing parts. Specifically, the system retrieves the attribute set of each structural component to determine whether the component has a specific label or parameter that identifies it as a load-bearing structure. For example, the system identifies components with an "isLoadBearing" attribute value of "1", or walls, beams, or columns whose "StructuralRole" attribute is explicitly marked as "LoadBearing".

[0041] After logically identifying the load-bearing components, the system extracts their precise 3D geometric information, including their spatial location, shape, dimensions, and boundary vertex coordinates. Using at least three calibrated common reference points, the system calculates the homogeneous transformation matrix between the BIM design coordinate system and the Cartesian coordinate system. This matrix is ​​then applied to uniformly transform the coordinates of all identified load-bearing components' vertices and the architectural geometric information in the hierarchical structure of the BIM data file to the Cartesian coordinate system. The system outputs a load-bearing information dataset and a building geometry dataset, which define the coordinates of the supporting structures and the overall building location within a unified site coordinate system.

[0042] Establish a Cartesian coordinate system covering the entire construction site, with its origin located on the ground at the center of the tower crane's rotation, and the Z-axis pointing vertically upwards. Discretize the two-dimensional area of ​​the ground into a raster map. Each grid cell in a raster map For a given area on the construction site, the three-dimensional spatial region is discretized into a three-dimensional voxel grid V. Each voxel v is uniquely identified by its three-dimensional integer index (i,j,k) and represents a cubic region in space, where i, j, and k are indices.

[0043] In one specific embodiment, the present invention constructs a weighted map of candidate starting positions during material scheduling based on panoramic images and BIM data. ;

[0044] The weight map of the candidate region at the starting position From the visibility weight graph and glare weight at starting position constitute;

[0045] ;

[0046] ;

[0047] This is a distance weight, used to represent the change in operational difficulty due to distance from the center of the tower crane. Operation is difficult in the immediate vicinity directly below the tower crane, with a weight of 0. As distance increases, visibility and operational convenience gradually improve, and the weight increases linearly. Once an optimal operational distance is reached, convenience peaks and remains constant, with a weight of 1. This weight value... It is a grid Distance from center point to tower crane center Piecewise functions:

[0048]

[0049] in, , These are the minimum and maximum operating radii of the tower crane, respectively. The optimal operating radius is the radius of the area directly beneath the tower crane where safe operation is impossible due to severely limited visibility. To ensure the driver's field of vision and operating experience are optimal, and Between these values, the weights smoothly and linearly increase from 0 to 1. This is the furthest working distance that the tower crane design itself can achieve. and Between these values, the weight remains at the highest value of 1. Beyond this range... The region has a weight of 0.

[0050] Occlusion weight is used to represent the impact of visual occlusion caused by the target building itself.

[0051] Occlusion weight construction includes: using a pre-trained semantic segmentation model to process panoramic images. Perform image segmentation to generate a binarized architectural mask image. ;

[0052] Suzuki contour tracking algorithm was used to... Extract the contour point set of the building in the panoramic image ;

[0053] The contour point set is transformed to obtain the polygonal region boundary in Cartesian coordinates. ;

[0054] Occlusion weights are generated based on the positional relationship between the polygon region boundary and each grid cell in the raster map G.

[0055]

[0056] The conversion of a contour point set to a polygonal region boundary is based on a camera-based pinhole imaging model, achieved through reverse ray projection. This projection originates from the camera's optical center and travels along points passing through the panoramic image. The direction of a ray can be used to determine a ray in three-dimensional space. The direction vector of this ray... It can be calculated using the following formula:

[0057] in, This is the intrinsic parameter matrix of the camera. Let be the rotation matrix of the camera pose. This is a transpose. The intersection of this 3D ray and the ground plane is the projection position of the contour point on the ground. After projecting all contour points onto the ground, the ground polygon of the building-occupied area is obtained. .

[0058] and These are the static safety zone weights and the dynamic safety zone weights, respectively. The static safety zone includes boundary data of fixed restricted areas (offices, hazardous materials warehouses, fixed machinery facilities, and temporary power grids), and the dynamic safety zone includes boundary data of pre-defined personnel and vehicle access areas at the construction site. The boundary data is then mapped to a raster map. Above. Any grid cell that falls within these areas. ,That or The value is assigned to 0, while those outside the region are assigned 1.

[0059] Distance weight With three hard safety constraint weights ( , , By performing point-by-point multiplication, this embodiment ultimately generates a visibility weight graph. This weighted map not only excludes all areas that are unusable due to obstruction and safety regulations, but also provides a quantitative assessment of the operational convenience of all available areas.

[0060] The method for constructing a weighted map of candidate starting positions disclosed in this invention improves the safety of material starting point selection compared to existing methods that rely on subjective judgment based on human experience. In complex construction sites, blind spots for tower crane operators and negligence on the part of ground personnel are major causes of accidents. This invention, by constructing occlusion weights, can accurately identify and quantify blind spots caused by the building itself from a "global perspective" based on image segmentation, eliminating the possibility of placing materials in these dangerous areas at the source. By introducing static and dynamic safety zone weights, this scheme incorporates fixed hazard sources (such as high-voltage lines and hazardous materials warehouses) and moving risk points (such as pedestrians and vehicles) into the decision model, forming an active electronic fence. Furthermore, this invention significantly optimizes the operational efficiency of tower cranes. Distance weights no longer simply determine "visibility," but introduce a continuous assessment of ease of operation. By modeling the ease of operation for tower crane operators under different operating radii using piecewise functions, we can proactively avoid areas with distorted vision and difficult operation caused by close proximity, while also favoring areas where the trolley does not need to be moved to its extreme position. This not only greatly reduces the operator's workload and minimizes errors that may be caused by high-difficulty operations, but also directly shortens the fine-tuning and positioning time in a single lifting cycle, thereby improving overall operational efficiency.

[0061] Spatial scheduling weight graph Glare Voxel Weighting Map 3D solid mask and three-dimensional restricted area mask constitute;

[0062]

[0063] This embodiment constructs a glare voxel weight map. This is used to quantify the impact of sunlight on the safety of tower crane operators' observation during high-altitude operations, with weight values ​​in... Between, among It is the preset minimum weight value, indicating that it is severely affected but not completely invisible.

[0064] The construction of the glare voxel weight map includes: acquiring solar position information and calculating the solar azimuth angle. and elevation angle Specifically, the system obtains the precise date and time through its built-in clock, and combines this with the geographical coordinates of the construction site (latitude and longitude), then calls upon standard astronomical libraries such as Pyephem or Skyfield to calculate the azimuth of the sun in the sky. and elevation angle .

[0065] Establish a glare cone model and calculate the unit direction vector from the control room to the sun. The glare cone model also includes a preset half-apex angle. Specifically, the observation point from the crane operator's cab is taken as the vertex of the glare cone model, and the vertex's position in the Cartesian frame is... In fact, since the origin of the Cartesian coordinate system is located on the ground where the center of rotation of the tower crane is located, and the Z-axis is vertically upward, therefore... It can be simplified to , The height of the control room. The unit direction vector from the vertex towards the sun. It can be calculated from the sun's azimuth and altitude angles:

[0066]

[0067] That is to As the vertex, Define a central axis with a preset semi-vertical angle. A cone, used as a glare cone model, in which the semi-apex angle It can be set to 15 degrees.

[0068] Calculate the weight value for each voxel v in space and construct a glare voxel weight map. Specifically, for each voxel in space... Its center point physical coordinates are Calculate from vertex Direction vector pointing to this voxel ,calculate With unit direction vector The angle between :

[0069] ;

[0070] included angle This indicates the proximity of the driver's line of sight to sunlight at that point. The smaller the angle, the more severe the glare. A Gaussian decay-smoothed function is used to calculate the glare voxel weight map. :

[0071] in, It is the half-apex angle of the glare cone. The standard deviation is used to control the drasticness of the weight decrease; preferably, it is defined as follows: That is, when the line of sight is directly facing the sun ( The weight reaches its minimum value. As the line of sight moves away from the sun, the weight smoothly returns to 1.

[0072] Each grid cell on the ground Set the height to 0 (i.e.) For each voxel, calculate the direction vector from the vertex to that voxel, calculate the angle between the direction vector and the unit direction vector, and then input the angle into a Gaussian decay smoothing function to obtain the glare weight at the starting position. .

[0073] Constructing a 3D solid mask based on BIM data And use static and dynamic safe zones to set up three-dimensional restricted area masks. ;

[0074] Among them, three-dimensional solid mask Used to identify areas in space that are absolutely inaccessible due to the presence of physical entities, it is a binary value (0 or 1). Based on the acquired coordinate-aligned BIM building's overall position coordinates, the system can obtain a 3D mesh model of the building. For each voxel... The coordinates of the center point The coordinates are determined by testing a point within a 3D mesh using an inside-3D mesh algorithm. Is it located inside the building structure? If Inside the building structure, 1 indicates that the location is an obstacle; otherwise, it is 1.

[0075] 3D forbidden zone mask This involves extending the ground-based safety zone into three-dimensional space. The boundaries of the acquired static safety zones (such as offices) and dynamic safety zones (such as personnel activity areas) are defined as impassable virtual walls. These two-dimensional safety zones are then stretched along the Z-axis across the entire workspace height to form columnar no-fly zones.

[0076] For voxels center point Take the projection point on the ground and determine whether the projection point is located within any ground safety zone. If so, then... Otherwise, it is 1.

[0077] This invention enables proactive avoidance of dynamic environmental risks, particularly the impact of solar glare, significantly improving operational safety. In traditional hoisting operations, solar glare is a common but easily overlooked major safety hazard. For example, on a sunny afternoon, when a tower crane needs to move materials from the east to the west, its path may need to traverse the area of ​​the sky where the sun is located. In traditional operations, the operator only experiences sudden glare and a sharp decline in visual ability when the boom rotates to a specific angle and the line of sight is close to the sunlight. At this point, the operator has to slow down, stop, or rely entirely on ground signalmen, which not only severely impacts operational efficiency but also greatly increases the risk of collisions due to the inability to accurately judge the distance between the load and the building. This invention fundamentally solves this problem by constructing a glare voxel weight map. This invention draws a weight map in three-dimensional space to describe a glare hazard zone. Therefore, during path planning, the scheduling system already knows where visual risks exist and automatically plans a slightly detour but safe path with clear visibility throughout. Secondly, this invention constructs a three-dimensional protection system, achieving a comprehensive integration of "hard" obstacles and "soft" rules in space. In traditional operations, the driver's safety awareness mainly focuses on avoiding hard collisions between the suspended load and the building structure, while being completely powerless regarding ground-level safety rules, such as whether the suspended load is passing under pedestrian walkways or temporary work areas. This invention, by integrating two types of three-dimensional masks, uses a three-dimensional solid mask based on BIM data to cover the entire building with a series of passable entities. The path planning algorithm ensures that the suspended load will never collide with any part of the building. On the other hand, by stretching the ground safety zone upwards to form a three-dimensional restricted area mask, it creates multiple safe space zones throughout the entire space, making path planning not only consider obstacles in the air but also comply with ground-level safety rules. For example, even if a path is unobstructed in the air, if it passes directly above a ground-level personnel rest area, the system will determine it as an illegal path and automatically detour. This integrated air-ground three-dimensional protection logic is impossible to achieve from the perspective of a single human operator.

[0078] Traditional path planning is conservative. For safety, drivers often choose the simplest and most redundant path, such as "vertical ascent to the highest point - horizontal movement - vertical descent," which is a huge waste of time and energy. The three-dimensional weighted graph constructed in this invention provides a detailed hazard map for the path planning algorithm. On this map, the algorithm can clearly distinguish where there are absolute no-go zones and high-risk areas (such as glare zones). This allows the algorithm to boldly seek better paths while ensuring safety, such as finding a sufficiently safe but shorter diagonal shuttle path between two obstacles. By quantifying the risks in the entire three-dimensional space, this invention transforms path selection from a fuzzy, intuition-based decision into a precise, computable optimization problem.

[0079] This invention also requires the construction of a weighted map of candidate regions for the endpoint location;

[0080] Based on the current construction floor height From the 3D voxel raster V, select all voxel center point coordinates with the height values ​​that are closest. Multiple voxels form a subset of target floor voxels. ,for For each voxel, calculate an endpoint position weight. Its range is .

[0081] Endpoint Candidate Region Weight Map Weighted by spatial openness and structural support weight constitute;

[0082] ;

[0083] Spatial openness weight Used to assess the accessible activity space around the unloading point. Based on panoramic images and generated binarized building mask images. Acquire building area images Using region growing on building area images Perform image segmentation to obtain the top-floor image of the current building;

[0084] A multi-object detection model was applied to the current building top-floor image to obtain a set of multiple obstacle detection bounding boxes. ,in These represent the center coordinates of the nth obstacle, as well as the width and height of the bounding box, respectively, where N is the number of obstacles;

[0085] Construct a binary free space map, which means creating a temporary two-dimensional raster map that is the same size as and aligned with the floor plan and initializes all of them to 1 (representing free space).

[0086] Traverse the set of target boxes The rectangular area covered by each target box is marked as 0 in the binary free space map. For any grid in FreeSpaceMap... Its value is: ;

[0087] in It is the first A target bounding box is the pixel area covered by a raster map.

[0088] Building a distance map based on FreeSpaceMap Define obstacle grid set Distance from map The value of each zone point is its distance from the point on the map. Euclidean distance to the nearest grid center: ;

[0089] Distance Map Normalization to The interval is used to obtain a temporary two-dimensional openness weight map. . ;

[0090] For the target floor voxel subset Each voxel in the array, based on the horizontal projection coordinates of its center point, in The search is performed, and the found value is assigned to the voxel as its openness weight. ;

[0091] Structural support weight Used to assess the structural bearing capacity below the unloading point, for a subset of target floor voxels. For each voxel in the dataset, the system performs the following operations: obtains the physical coordinates of the center point of each voxel, queries whether there exists a weighing structure in the load-bearing information dataset where the planar distance and height difference between the physical coordinates of the center point and the coordinates of the weighing structure are both less than a preset threshold, and generates a temporary load-bearing structure distribution map based on the query results, where the positions that meet the preset threshold conditions are marked as 1, and the other positions are marked as 0.

[0092] Apply a distance transformation to the temporary load-bearing structure distribution map to obtain a distance map from each point to the nearest load-bearing structure. Distance transformation and The construction method is the same;

[0093] Obtain structural support weights :

[0094] ;

[0095] in and These are the basic safety weight and the preset attenuation radius, respectively.

[0096] The optimal scheduling decision is obtained based on the weighted map of candidate regions for starting positions, the weighted map of candidate regions for material destination positions, the spatial scheduling weighted map, and the comprehensive cost function, including:

[0097] Based on the weight map of candidate regions at the starting position and endpoint location candidate region weights Select candidate starting grid sets that are above their respective thresholds. ={ | } and candidate endpoint grid set ,in, and These represent the starting and ending raster numbers that satisfy the thresholds, respectively. and The elements that are the start and end raster sets;

[0098] For each candidate pairing In three-dimensional voxel space Perform path connectivity analysis using breadth-first search (BFS); Starting from, As the endpoint, in three-dimensional voxel space The BFS algorithm is run in the middle. BFS can guarantee that if a path exists, a path with the fewest voxels can be found;

[0099] If BFS successfully finds a path Then the triplet Define a feasible scheduling scheme and store it in the feasible solution set. middle;

[0100] By iterating through all candidate pairings, a set containing multiple feasible scheduling schemes is finally obtained. M is the number of feasible scheduling schemes, where each scheme ; These represent the starting point, ending point, and scheduling path of the t-th scheme, respectively.

[0101] For each scheme Calculate multi-dimensional performance vectors This vector is used to objectively describe the advantages and disadvantages of the solution from different perspectives. In this embodiment, the present invention defines a five-dimensional performance vector:

[0102]

[0103] in - Each is a separate performance metric:

[0104] The starting point quality is determined by the weight of the candidate region corresponding to the starting position in the weight map of the proposed solution.

[0105] ;

[0106] The endpoint quality is determined by the weight of the candidate region in the endpoint location weight map corresponding to the endpoint quality.

[0107] ;

[0108] and These represent the index transformations in the candidate region weight graphs for the starting and ending positions, respectively.

[0109] For path safety, the lowest weight of all voxels on the path in the spatial scheduling weight map is used. This indicator represents the safety weakness of the scheme; a higher value indicates that the entire path does not pass through particularly dangerous areas.

[0110]

[0111] The path length, i.e. the total geometric length of the path, represents the basic time and space costs.

[0112] ;

[0113] Length() represents the number of voxels passed through;

[0114] Scheduling complexity is used to measure the total amount of movement required for a tower crane to complete this path, and is an indirect assessment of energy consumption and operational complexity.

[0115]

[0116] in These are the total vertical, radial, and angular displacements along the path. Each represents its own coefficient.

[0117] After obtaining the five-dimensional performance vectors of all feasible scheduling schemes, the ideal optimal solution is calculated. And the worst solution

[0118] ;

[0119] ;

[0120] For each scheme Calculate separately Normalized performance vector and ideal optimal solution And the worst solution Euclidean distance between and ;

[0121] For each scheme Calculate an exponent :

[0122]

[0123] The value in between. The closer the value is to 1, the closer the solution is to the ideal optimal solution, and the further away it is from the ideal worst solution, meaning the better the solution is.

[0124] The system supports all schemes. Sort the values ​​in descending order, and find the triplet of the feasible solution that ranks first. This is determined as the optimal decision for this scheduling.

[0125] The optimal scheduling decision method disclosed in this invention solves the problem of decision bias and fragility caused by the reliance on a single cost function in traditional scheduling methods by constructing a framework that includes feasible solution generation, multi-objective performance evaluation and ranking-based optimization decision-making.

[0126] First, traditional path planning algorithms, whether A* or others, often combine multiple factors such as safety, efficiency, and energy consumption into a single cost value through a manually set cost function. This weighting is highly subjective; even minor adjustments can lead to significant differences in the final solution, lacking scientific basis. This invention constructs a multi-dimensional performance vector for each feasible solution, fully preserving the solution's performance on each independent objective. It employs optimal similarity for decision-making, focusing on finding the solution closest to a virtual "ideal optimal solution." This process is purely data-driven, requiring no human intervention in weighting, thus ensuring the objectivity and stability of the decision results and making the final optimal choice more convincing.

[0127] Secondly, this invention achieves a comprehensive, multi-faceted evaluation of scheduling schemes, uncovering hidden optimal solutions overlooked by single-objective optimization methods. In a complex hoisting task, the best solution is often a clever balance among multiple conflicting objectives. For example, a scheme with the shortest path (k4 optimal) may have the worst safety (k3 lowest); while a scheme with extremely high origin and destination quality (k1, k2 optimal) may have a very long and complex path (k4, k5 worst). Traditional single-cost function methods, due to their weight settings, tend to choose one of these extreme solutions, ignoring more balanced intermediate solutions. The decision framework of this invention, however, comprehensively considers all five performance indicators. It can identify and ultimately select a scheme that may not have the shortest path, but has very high safety, origin and destination quality, and moderate scheduling complexity. This ability to make intelligent trade-offs and discover and select the scheme with the most balanced overall performance aligns with the choices of experienced tower crane operators, representing a significant advantage of this invention compared to existing technologies, ensuring that the final output decision is truly usable and safe in practical applications.

[0128] This specification also proposes an artificial intelligence-based perception and optimal decision-making scheduling system for complex engineering environments, the device comprising:

[0129] Acquisition Module: The acquisition module acquires panoramic images of the project site and obtains the project's BIM data;

[0130] Weighted map generation module: Based on panoramic images and BIM data, constructs a weighted map of candidate areas for starting position, candidate areas for ending position, and spatial scheduling weighted map during the material scheduling process;

[0131] Scheduling Decision Module: The scheduling decision module obtains the optimal scheduling decision based on the weight map of candidate regions for starting position, candidate regions for material destination position, and spatial scheduling weight map.

[0132] The communication module sends the optimal scheduling decision to the tower crane control room and ground control personnel.

[0133] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the aforementioned artificial intelligence-based method for perception and optimal decision-making scheduling in complex engineering environments.

[0134] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned method for perception and optimal decision-making scheduling in complex engineering environments based on artificial intelligence.

[0135] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0136] In this specification, the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the descriptions of the embodiments described later are relatively simple, and relevant parts can be referred to the descriptions of the foregoing embodiments.

[0137] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for perception and optimal decision-making scheduling in complex engineering environments based on artificial intelligence, characterized in that, The method includes: acquiring panoramic images of the project site and obtaining BIM data; constructing a weighted map of candidate areas for starting positions, candidate areas for ending positions, and a spatial scheduling weighted map based on the panoramic images and BIM data; obtaining the optimal scheduling decision based on the candidate areas for starting positions, candidate areas for ending positions, and spatial scheduling weighted maps; and sending the optimal scheduling decision to the tower crane control room and ground control personnel. The method also includes: establishing a Cartesian coordinate system covering the entire construction site, with its origin located on the ground where the tower crane's rotation center is located, and the Z-axis pointing vertically upwards; and discretizing the two-dimensional area of ​​the ground into a grid map. Each grid cell in a raster map For a given area on the construction site, the three-dimensional spatial region is discretized into a three-dimensional voxel grid V. Each voxel v is uniquely identified by its three-dimensional integer index (i,j,k) and represents a cubic region in space, where i, j, and k are indices; Spatial scheduling weight graph. Glare Voxel Weight Map 3D solid mask and three-dimensional restricted area mask The glare voxel weight map construction includes: acquiring solar position information and calculating the solar azimuth angle. and elevation angle Establish a glare cone model and calculate the unit direction vector from the control room to the sun. ： The glare cone model also includes a preset half-apex angle. For each voxel in space Its center point physical coordinates are Calculate from vertex Direction vector pointing to this voxel ,calculate With unit direction vector The angle between ： A Gaussian decay smoothing function is used to calculate the glare voxel weight map. ： ;in, 。 2. The method for perception and optimal decision-making scheduling of complex engineering environments based on artificial intelligence according to claim 1, characterized in that, The weight map of the candidate region at the starting position Visibility weight graph and glare weight at starting position constitute.

3. The method for perception and optimal decision-making scheduling in complex engineering environments based on artificial intelligence according to claim 2, characterized in that: Endpoint Candidate Region Weight Map Weighted by spatial openness and structural support weight constitute; This represents the current height of the floor under construction.

4. The method for perception and optimal decision-making scheduling in complex engineering environments based on artificial intelligence according to claim 3, characterized in that: Based on the weight map of candidate regions at the starting position and endpoint location candidate region weights Select candidate starting raster sets that are above their respective thresholds. ={ | } and candidate endpoint grid set ,in, and These represent the starting and ending raster numbers that satisfy the thresholds, respectively. and These are the elements in the starting and ending raster sets.

5. The method for perception and optimal decision-making scheduling in complex engineering environments based on artificial intelligence according to claim 4, characterized in that: For each candidate pairing In three-dimensional voxel space Perform path connectivity analysis using BFS; if BFS successfully finds a path... Then the triplet Define a feasible scheduling scheme and store it in the feasible solution set. In the middle; traverse all candidate pairings to obtain a set containing multiple feasible scheduling schemes. M is the number of feasible scheduling schemes, where each scheme ； These represent the starting point, ending point, and scheduling path of the t-th scheme, respectively.

6. The method for perception and optimal decision-making scheduling in complex engineering environments based on artificial intelligence according to claim 5, characterized in that: For each scheme Calculate multi-dimensional performance vectors ： in, The starting point quality is determined by the weight of the candidate region corresponding to the starting position in the weight map of the proposed solution. ; The endpoint quality is determined by the weight of the candidate region in the endpoint location weight map corresponding to the endpoint quality. ;in, and These represent the index transformations in the candidate region weight graphs for the starting and ending positions, respectively. For path safety, the path is represented by the lowest weight in the spatial scheduling weight graph of all voxels. ； The path length is expressed as the total geometric length of the path. ;Length() represents the number of voxels passed through; For scheduling complexity; ;in, These are the total vertical, radial, and angular displacements along the path. Each coefficient is assigned separately; the ideal optimal solution is calculated. And the worst solution ： ； For each solution Calculate separately Normalized performance vector and ideal optimal solution And the worst solution Euclidean distance between and For each scheme Calculate an exponent ： ， The value in Between, the system for all schemes The values ​​are sorted in descending order, and the first-ranked solution is the triplet of feasible solutions. It was determined to be the optimal decision for this scheduling.

7. A perception and optimal decision-making scheduling system for complex engineering environments based on artificial intelligence, the system being used to execute the perception and optimal decision-making scheduling method for complex engineering environments based on artificial intelligence as described in any one of claims 1-6, characterized in that, The system includes: a data acquisition module, which acquires panoramic images of the project site and obtains BIM data; a weighted map generation module, which constructs a weighted map of candidate areas for starting positions, candidate areas for ending positions, and a spatial scheduling weighted map based on the panoramic images and BIM data; a scheduling decision module, which obtains the optimal scheduling decision based on the candidate areas for starting positions, candidate areas for ending positions, and the spatial scheduling weighted map; and a communication module, which sends the optimal scheduling decision to the tower crane control room and ground control personnel.

Images