An automatic construction simulation model building method and device based on video images

Abstract

The embodiments of the present application provide a method and device for automatically constructing a construction simulation model based on video images. The method specifically includes: establishing a construction simulation model for construction operations; the construction simulation model includes: a construction process simulation model corresponding to a construction unit intelligent agent object; the construction process simulation model includes a construction operation simulation model; the construction operation simulation model includes: queue elements and delay elements; the queue elements are used to simulate the queuing and waiting process of construction resource intelligent agent objects in construction operations; the delay elements are used to simulate the duration of construction operations; according to the video images of the construction operation site, using object detection technology, object tracking technology and clustering technology to determine the construction resource distribution information corresponding to the construction operation surface of the construction operation site; according to the construction resource distribution information, instantiate the queue elements and delay elements in the corresponding construction operation simulation model. The embodiments of the present application can improve the construction efficiency of the construction simulation model.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of construction simulation, and in particular to a method and device for automatically constructing a construction simulation model based on video images. Background Art

[0002] Construction simulation technology is a new technical means for engineering construction progress control. Its purpose is to simulate the engineering construction process by computer, provide support for formulating construction progress plans, allocating construction resources, and discovering possible problems in construction, so as to provide guidance for engineering construction decisions. An important basis for its realization is to establish a construction simulation model that reflects the characteristics of engineering construction.

[0003] In the related art, usually, first, a static construction simulation model is established for construction operations; then, according to the situation at the construction operation site, the model parameters included in the static construction simulation model are instantiated. Among them, the model parameters may include: construction resource parameters, construction object parameters, etc. The instantiation method of the model parameters is usually the manual input method, and the user manually inputs the model parameters into the construction simulation model in the computer system.

[0004] However, the method of manually inputting the model parameter values into the construction simulation model in the computer system is both time-consuming and laborious and lacks real-time performance, which affects the construction efficiency of the construction simulation model. Summary of the Invention

[0005] The embodiments of the present application provide a method for automatically constructing a construction simulation model based on video images, which can improve the construction efficiency of the construction simulation model.

[0006] Correspondingly, the embodiments of the present application further provide a device for automatically constructing a construction simulation model based on video images, an electronic device, and a machine-readable medium to ensure the implementation and application of the above method.

[0007] To solve the above problems, the embodiments of the present application disclose a method for automatically constructing a construction simulation model based on video images, including:

[0008] For construction operations, a construction simulation model is established; the construction simulation model includes: construction unit intelligent agent objects, construction resource scheduling intelligent agent objects, construction resource intelligent agent objects, and construction process simulation models respectively corresponding to multiple construction unit intelligent agent objects; the construction process simulation model includes: one or more construction operation simulation models; the construction operation simulation model includes: queue elements and delay elements; the queue elements are used to simulate the queuing and waiting process of construction resource intelligent agent objects in the construction operation; the delay elements are used to simulate the duration of the construction operation;

[0009] The simulation process of the construction simulation model includes: the construction unit agent object sends a resource request message to the construction resource scheduling agent object according to the corresponding construction process simulation model; the construction resource scheduling agent object sends a task assignment message to at least one selected target construction resource agent object according to the resource request message; after receiving the task assignment message, at least one target construction resource agent object moves to the target construction work surface where the construction unit agent object is located; the construction resource agent objects that meet the set conditions at the target construction work surface are put into the queue represented by the queue element for queuing and waiting; the construction resource agent objects that do not need to queue and wait or are taken out from the queue element at the target construction work surface are used to perform the construction simulation of the construction process; after the simulation of the construction process of the construction unit agent object is completed, the corresponding delay element ends;

[0010] According to the video image of the construction operation site, the construction resource distribution information corresponding to the construction work surface of the construction operation site is determined by using object detection technology, object tracking technology and clustering technology;

[0011] According to the construction resource distribution information, the queue element and the delay element in the corresponding construction process simulation model are instantiated.

[0012] The embodiment of the present application also discloses an automatic construction simulation model building device based on video images. The device includes:

[0013] A model building module, which is used to establish a construction simulation model for the construction operation. The construction simulation model includes: a construction unit agent object, a construction resource scheduling agent object, a construction resource agent object, and construction process simulation models respectively corresponding to a plurality of construction unit agent objects. The construction process simulation model includes: one or more construction process simulation models. The construction process simulation model includes: a queue element and a delay element. The queue element is used to simulate the queuing and waiting process of the construction resource agent object in the construction process. The delay element is used to simulate the duration of the construction process;

[0014] The simulation process of the construction simulation model includes: the construction unit agent object sends a resource request message to the construction resource scheduling agent object according to the corresponding construction process simulation model; the construction resource scheduling agent object sends a task assignment message to at least one selected target construction resource agent object according to the resource request message; after receiving the task assignment message, at least one target construction resource agent object moves to the target construction work surface where the construction unit agent object is located; the construction resource agent objects that meet the set conditions at the target construction work surface are put into the queue represented by the queue element for queuing; the construction simulation of the construction process is performed by using the construction resource agent objects that do not need to queue or are taken out from the queue element at the target construction work surface; after the simulation of the construction process of the construction unit agent object is completed, the corresponding delay element ends;

[0015] A video analysis module, configured to determine the construction resource distribution information corresponding to the construction work surface at the construction operation site according to the video image of the construction operation site by using object detection technology, object tracking technology and clustering technology;

[0016] An instantiation module, configured to instantiate the queue element and the delay element in the corresponding construction process simulation model according to the construction resource distribution information.

[0017] An embodiment of the present application also discloses an electronic device, including: a processor; and a memory, on which executable code is stored, and when the executable code is executed, the processor is caused to execute the method as described in the embodiment of the present application.

[0018] An embodiment of the present application also discloses a machine-readable medium, on which executable code is stored, and when the executable code is executed, a processor is caused to execute the method as described in the embodiment of the present application.

[0019] An embodiment of the present application also discloses a computer program product, including computer programs / instructions, and when the computer programs / instructions are executed by a processor, the foregoing method is implemented.

[0020] The embodiment of the present application has the following advantages:

[0021] The technical solution of the embodiment of the present application specifically includes: a static model construction link, a resource distribution information acquisition link and an instantiation link.

[0022] Among them, in the static model construction link, the construction simulation model is divided into multiple parts, including construction unit intelligent agent objects, construction resource scheduling intelligent agent objects, construction resource intelligent agent objects, and construction process simulation models. The construction process simulation model is further divided into multiple construction process simulation models. The construction process simulation model consists of queue elements and delay elements, which are used to simulate the queuing waiting and process duration of construction resources, respectively, and provides a simulation process of the construction simulation model.

[0023] In the resource distribution information acquisition stage, target detection technology, target tracking technology and clustering technology are used to automatically determine the construction resource distribution information corresponding to the construction work surface.

[0024] In the model instantiation stage, the queue elements and delay elements in the corresponding construction process simulation model are instantiated according to the automatically determined construction resource distribution information.

[0025] The embodiment of the present application adopts target detection technology, target tracking technology and clustering technology to analyze the video image of the construction site, so as to quickly and automatically determine the construction resource distribution information, and use the automatically determined construction resource distribution information for the instantiation of queue elements and delay elements. On the one hand, because the embodiment of the present application does not need to manually count the construction resource distribution information one by one, the collection time of the construction resource distribution information can be shortened. On the other hand, in view of the fact that the embodiment of the present application eliminates the link of manually inputting the construction resource distribution information, it is possible to save manual input time. In summary, the embodiment of the present application can improve the construction efficiency of the construction simulation model. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic diagram of the steps of a method for automatically building a construction simulation model based on video images according to an embodiment of the present application;

[0027] Figure 2 It is a schematic diagram of the state transition of a construction resource intelligent agent object according to an embodiment of the present application;

[0028] Figure 3 is a schematic diagram of the state transition of a transportation equipment agent object according to an embodiment of the present application;

[0029] Figure 4 This is a schematic diagram of the state transition of a construction resource scheduling agent object according to an embodiment of the present application;

[0030] FIG5( a ) and FIG5( b ) are schematic diagrams of modeling of a construction process according to an embodiment of the present application;

[0031] Figure 6 It is a schematic diagram of a construction process simulation model of excavation blocks in an embodiment of the present application;

[0032] Figure 7 It is a schematic diagram of a construction simulation model according to an embodiment of the present application;

[0033] Figure 8 It is a schematic structural diagram of an automatic construction simulation model building device based on video images according to an embodiment of the present application;

[0034] Figure 9 It is a schematic structural diagram of a device provided by an embodiment of the present application. Specific Embodiments

[0035] To make the above objects, features, and advantages of the present application more obvious and understandable, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0036] Embodiments of the present application can be applied to the hydropower engineering industry, and are used to simulate the construction process of hydropower projects based on a computer, providing support for formulating a construction schedule, allocating construction resources, and discovering possible problems in construction, so as to provide guidance for engineering construction decisions.

[0037] In the related art, usually, first, a static construction simulation model is established for construction operations; then, according to the situation at the construction operation site, the model parameters included in the static construction simulation model are instantiated. Among them, the model parameters may include: construction resource parameters, construction object parameters, etc. The instantiation method of the model parameters is usually a manual input method, and the user manually inputs the model parameters into the construction simulation model in the computer system.

[0038] However, the method of manually inputting the model parameter values into the construction simulation model in the computer system is both time-consuming and laborious and lacks real-time performance, affecting the construction efficiency of the construction simulation model.

[0039] To solve the technical problem of the low construction efficiency of the construction simulation model in the related art, embodiments of the present application provide an automatic construction simulation model building method based on video images. The method specifically includes the following steps:

[0040] For construction operations, a construction simulation model is established; the construction simulation model includes: construction unit intelligent agent objects, construction resource scheduling intelligent agent objects, construction resource intelligent agent objects, and construction process simulation models respectively corresponding to multiple construction unit intelligent agent objects; the construction process simulation model includes: one or more construction operation simulation models; the construction operation simulation model includes: queue elements and delay elements; the queue elements are used to simulate the queuing and waiting process of construction resource intelligent agent objects in the construction operation; the delay elements are used to simulate the duration of the construction operation;

[0041] The simulation process of the construction simulation model includes: the construction unit agent object sends a resource request message to the construction resource scheduling agent object according to the corresponding construction process simulation model; the construction resource scheduling agent object sends a task assignment message to at least one selected target construction resource agent object according to the resource request message; after receiving the task assignment message, at least one target construction resource agent object moves to the target construction work surface where the construction unit agent object is located; the construction resource agent objects that meet the set conditions at the target construction work surface are put into the queue represented by the queue element for queuing; the construction simulation of the construction process is performed by using the construction resource agent objects that do not need to queue or are taken out from the queue element at the target construction work surface; after the simulation of the construction process of the construction unit agent object is completed, the corresponding delay element ends;

[0042] According to the video images of the construction site, the construction resource distribution information corresponding to the construction work surface of the construction site is determined by using object detection technology, object tracking technology and clustering technology;

[0043] According to the construction resource distribution information, the queue element and the delay element in the corresponding construction process simulation model are instantiated.

[0044] The technical solution of the embodiment of the present application specifically includes: a static model construction link, a resource distribution information acquisition link and an instantiation link.

[0045] Among them, in the static model construction link, the construction simulation model is divided into multiple parts such as construction unit agent objects, construction resource scheduling agent objects, construction resource agent objects, and construction process simulation models, and the construction process simulation model is further divided into multiple construction process simulation models. The construction process simulation model is composed of a queue element and a delay element, which are respectively used to simulate the queuing of construction resources and the process duration, and provide the simulation process of the construction simulation model.

[0046] In the resource distribution information acquisition link, object detection technology, object tracking technology and clustering technology are used to automatically determine the construction resource distribution information corresponding to the construction work surface.

[0047] In the model instantiation link, according to the automatically determined construction resource distribution information, the queue element and the delay element in the corresponding construction process simulation model are instantiated.

[0048] The embodiment of the present application adopts target detection technology, target tracking technology and clustering technology to analyze the video image of the construction site, so as to quickly and automatically determine the construction resource distribution information, and use the automatically determined construction resource distribution information for the instantiation of queue elements and delay elements. On the one hand, because the embodiment of the present application does not need to manually count the construction resource distribution information one by one, the collection time of the construction resource distribution information can be shortened. On the other hand, in view of the fact that the embodiment of the present application eliminates the link of manually inputting the construction resource distribution information, it is possible to save manual input time. In summary, the embodiment of the present application can improve the construction efficiency of the construction simulation model.

[0049] In addition, the embodiment of the present application divides the construction simulation model into multiple parts such as construction unit intelligent body object, construction resource scheduling intelligent body object, construction resource intelligent body object and construction process simulation model. The multiple parts of the construction simulation model complement each other and work closely together from different angles, showing the whole process of simulation of construction operations from resource scheduling to construction process. Among them, the construction unit intelligent body object is used to represent the construction object; the construction resource intelligent body object is used to simulate the state information of the construction resources of each resource category; the construction resource scheduling intelligent body object realizes the reasonable allocation of the construction resource intelligent body object; the construction process simulation model clarifies the simulation method of multiple construction processes; the queue element simulates the queuing waiting situation of the construction resource intelligent body object on the construction operation surface; the delay element controls the time of each construction process and simulates the time dimension of each link in the construction process. In summary, the construction simulation model of the embodiment of the present application can realize the accurate simulation of the construction process in the construction operation, can provide support for the early discovery of potential problems such as unreasonable resource allocation, unsmooth process connection, and delayed construction progress in the construction operation, and can also provide strong support for optimizing the construction operation plan, thereby helping the construction project to be promoted efficiently and with high quality.

[0050] Method Embodiment

[0051] refer to Figure 1 , shows a schematic flow chart of the steps of a method for automatically building a construction simulation model based on video images according to an embodiment of the present application, the method specifically comprising the following steps:

[0052] Step 101, establishing a construction simulation model for construction operations; the construction simulation model specifically includes: a construction unit intelligent agent object, a construction resource scheduling intelligent agent object, a construction resource intelligent agent object, and a construction process simulation model corresponding to a plurality of construction unit intelligent agent objects; the construction process simulation model specifically includes: one or more construction process simulation models; the construction process simulation model specifically includes: a queue element and a delay element; the queue element is used to simulate the queuing and waiting process of the construction resource intelligent agent objects in the construction process; the delay element is used to simulate the duration of the construction process;

[0053] The simulation process of the above construction simulation model specifically includes: the construction unit agent object sends a resource request message to the construction resource scheduling agent object according to the corresponding construction process simulation model; the construction resource scheduling agent object sends a task assignment message to at least one selected target construction resource agent object according to the above resource request message; after receiving the task assignment message, at least one target construction resource agent object moves to the target construction work surface where the construction unit agent object is located; the construction resource agent objects that meet the set conditions at the target construction work surface are put into the queue represented by the queue element for queuing and waiting; the construction simulation of the construction process is performed by using the construction resource agent objects that do not need to queue and wait or are taken out from the queue element at the target construction work surface; after the simulation of the construction process of the construction unit agent object is completed, the corresponding delay element ends;

[0054] Step 102: According to the video image of the construction operation site, use object detection technology, object tracking technology, and clustering technology to determine the construction resource distribution information corresponding to the construction work surface of the construction operation site;

[0055] Step 103: According to the construction resource distribution information, instantiate the queue element and the delay element in the corresponding construction process simulation model.

[0056] Figure 1 The method shown can be used to construct a construction simulation model for construction operations.

[0057] In the engineering field, construction operations refer to a series of organized and planned production activities carried out on the construction site of a project to meet the engineering design requirements by arranging various construction resources (such as labor, materials, construction machinery, etc.) according to specific construction techniques, processes, and specifications.

[0058] For example, in the scenario of a hydropower project, examples of construction operations can include: slope excavation operations, dam pouring operations, or tunnel excavation operations, etc. It can be understood that the embodiments of the present application do not limit specific construction operations.

[0059] Taking the slope excavation operation as an example, the construction objects of the slope excavation operation can be divided into three levels: excavation areas, excavation benches, and excavation layers according to the geometric structure subordination relationship of the excavation parts.

[0060] Among them, an excavation area may include: several excavation benches. Specifically, the entire construction object can be divided into different excavation areas such as the front block excavation area and the top surface excavation area of the protective layer according to the vertical partition line. Subsequently, each excavation area is further divided into multiple excavation benches according to the designed bench height. For example, the elevation of 50m - 55m is designated as the first excavation bench, and 55m - 60m is designated as the second excavation bench. Due to the different heights, the construction conditions of different excavation benches vary.

[0061] A specific excavation sub-bench includes: several excavation sub-blocks. Specifically, each excavation bench can be divided into multiple excavation sub-blocks along the horizontal direction. Each excavation sub-block is a basic construction unit, and subsequent specific construction processes are carried out in these excavation sub-blocks.

[0062] In the field of construction simulation technology, an agent refers to an entity that can perceive the environment, make decisions, and take actions. It has autonomy and can decide what actions to execute based on its own goals and perception of the environment to achieve the predetermined goals.

[0063] An agent object is the instantiation of an agent in a specific application scenario (such as a construction simulation model). It endows the abstract concept of an agent with specific attributes, states, and behavior rules, enabling it to represent an entity with autonomous behavior capabilities in a specific construction simulation model. Behavior rules refer to a series of guidelines that specify how an agent should act in a specific environment (such as a construction scenario). It clarifies the response method of the agent to internal state changes and external stimuli (such as instructions, events, etc.), and determines the behavior logic and execution order of the agent.

[0064] For example, in a construction simulation model, construction unit agent objects, construction resource scheduling agent objects, and construction resource agent objects all have their own specific attributes (such as the geometric dimensions of construction units, the types and quantities of construction resources, etc.), states (such as idle, busy, etc.), and behavior rules (such as resource scheduling algorithms, the state transition methods of construction resources).

[0065] The following will separately describe the construction unit agent object, the construction resource scheduling agent object, and the construction resource agent object.

[0066] The construction unit agent object is used to describe construction units such as excavation sub-blocks. The collection of multiple construction unit agent objects can be used to describe the construction object corresponding to the entire construction operation. The construction unit agent object usually contains attribute parameters related to the construction unit, and these attribute parameters can be information such as construction unit size, location, and geological conditions, and each construction unit's characteristics are characterized by these attribute parameters.

[0067] The embodiments of the present application can instantiate construction units to obtain construction unit intelligent agent objects. For example, the instantiation process of a construction unit specifically includes: First, obtain the partition number of the construction unit from the project documents; then, for the partition number of the construction unit, obtain attribute information such as the geometric shape, size data, position data, and first behavior rules of the construction unit from the project documents; bind the partition number of the construction unit with the attribute information to obtain a construction unit intelligent agent object.

[0068] Taking the slope excavation operation as an example, the construction unit intelligent agent object specifically includes: an excavation partition intelligent agent object, an excavation bench intelligent agent object, and an excavation block intelligent agent object.

[0069] Among them, the attribute information of the excavation partition intelligent agent object includes the partition number.

[0070] The attribute information of the excavation bench intelligent agent object includes: the partition number of the affiliated excavation partition, the bench number, the bench height, and the start and end elevations.

[0071] The attribute information of the excavation block intelligent agent object includes: the affiliated gradient number, the block number, and the engineering quantity information. The engineering quantity information can include the excavation block area, the length along the reference line, the width along the reference line, the excavation block height, the number of deep / shallow hole drillings, etc. The slope excavation operation can be described as a set composed of all excavation block intelligent agent objects. The construction simulation of the slope excavation operation can be understood as: performing simulation calculations on the set of excavation block intelligent agent objects. Optionally, the set of excavation partition intelligent agent objects and the set of excavation bench intelligent agent objects are only used for imposing construction constraint conditions, engineering quantity information statistics, etc., and are not used as specific construction simulation objects.

[0072] The construction resource intelligent agent object represents various construction resources, such as labor (construction personnel), materials (construction equipment, materials, etc.). In a construction scenario, each construction resource can be regarded as a construction resource intelligent agent object. The construction resource intelligent agent object can receive task instructions from the construction resource scheduling intelligent agent object and play a role in the construction process corresponding to the construction unit intelligent agent object. For example, the excavation equipment performs excavation operations in the specified excavation block according to the instructions, and the transport vehicle transports the excavated soil away, etc. The behavior and state of the construction resource intelligent agent object will be affected by the construction process, and at the same time, it will also feedback information to the construction resource scheduling intelligent agent object for better resource allocation.

[0073] The embodiments of the present application can instantiate construction resources to obtain construction resource agent objects. Specifically, by analyzing the types of construction resources and the requirements for construction resource scheduling involved in construction operations, and defining the possible states of construction resource agents, as well as the state transition methods, triggering conditions, and second behavior rules between different states, the instantiation of construction resources is completed. The information of the construction resource scheduling agent object specifically includes: resource category, resource number, performance parameters, status, state transition method, triggering condition, etc.

[0074] The states of the construction resource agent object can include but are not limited to: initial state, idle state, working state, maintenance state, etc. The state transition methods can include: time-triggered, condition-triggered, and message-triggered. Among them, time-triggered means that after a specific time interval is met, it switches from one set state to another set state; condition-triggered means that it switches from one set state to another set state after a specific condition is met; message-triggered means that when the construction resource agent object receives a specific message, it switches from one set state to another set state.

[0075] Referring to Figure 2 , a schematic diagram of the state transition of the construction resource agent object in an embodiment of the present application is shown. The corresponding state transition process can be described as follows: at the beginning of the simulation, the state of the construction resource agent object changes from the initial state to the idle state. When it receives a task assignment message (message-triggered), the state of the construction resource agent object changes to the working state and goes to the corresponding construction operation surface. After completing the corresponding construction process, its state changes to the idle state (condition-triggered). When it receives an operation and maintenance message (message-triggered), the state of the construction resource agent object changes to the maintenance state and goes to the maintenance site or conducts maintenance operations on-site. The construction operation surface can refer to the place where construction activities are actually carried out. Areas such as the site for excavation operations, the material yard for storing materials, and the area where the dam is filled are all construction operation surfaces.

[0076] Taking the slope excavation operation as an example, the resource categories of the construction resources that may be involved in the excavation construction process specifically include: drilling equipment, bulldozing equipment, excavation and loading equipment, and transportation equipment. Then, the drilling equipment agent object, bulldozing equipment agent object, excavation and loading equipment agent object, and transportation equipment agent object can be defined accordingly.

[0077] Taking the transportation equipment agent object as an example, its state and state transition can be defined as Figure 3As shown in the figure. At the beginning of the simulation, the state of the transportation equipment agent object changes from the initial state to the idle state. When receiving the task assignment message, the state of the transportation equipment agent object changes to moving to the excavation working face. After arriving at the excavation working face, the state of the transportation equipment agent object changes to waiting for loading. After completing construction processes such as loading, heavy hauling, and unloading, the state of the transportation equipment agent object changes to idle. When receiving the start of daily maintenance message, the state of the transportation equipment agent object changes to daily maintenance.

[0078] The construction resource scheduling agent object is used to schedule the construction resource agent objects. It can perceive the resource requirements during the construction process of the construction unit agent objects. For example, in slope excavation operations, when a certain excavation block (construction unit agent object) needs construction resources such as excavation equipment and transport vehicles, it can send a resource request message to the construction resource scheduling agent object. The construction resource scheduling agent object will make scheduling decisions based on factors such as the availability of the construction resource agent objects and the priority of the construction units, and allocate appropriate construction resource agent objects to the construction unit agent objects in need.

[0079] The construction resource scheduling agent object of the embodiment of the present application may include the following information: scheduling scope (responsible construction working face), scheduling strategy (such as allocating resources according to the priority of the construction unit or resource utilization efficiency, etc.), state, state transition method, trigger condition, third behavior rule, etc.

[0080] The state of the construction resource scheduling agent object may include but is not limited to: initial state, construction task existence judgment state, waiting for construction task state, service construction simulation object selection state, idle construction resource judgment state, waiting for construction resource idle state, construction resource scheduling state, task assignment, etc.

[0081] Refer to Figure 4 , a state transition diagram of the construction resource scheduling agent object of an embodiment of the present application is shown, where the state of the construction resource scheduling agent object includes: initial state, construction task existence judgment state, waiting for construction task state, service construction simulation object selection state, idle construction resource judgment state, waiting for construction resource idle state, construction resource scheduling state, task assignment, and other states;

[0082] After the construction simulation model starts the simulation, the state of the construction resource scheduling agent changes from the initial state to the construction task existence judgment state. If it receives a resource request message sent by any construction unit agent object, it is considered that there is a construction task, and the state changes to the service construction simulation object selection state; if it does not receive a resource request message, the state changes to the waiting for construction task state, and it loops to judge until it receives a resource request message;

[0083] When the state of the construction resource scheduling agent changes to the service construction simulation object selection state, according to the first preset rule, select a target construction unit agent object from multiple construction unit agent objects that send requests, and the state changes to the idle construction resource judgment state; if there is a construction resource agent object in the idle state, the state changes to the construction resource scheduling state; if not, the state changes to the waiting for construction resource idle state, and loop to judge until a construction resource agent state changes to the idle state.

[0084] When the state of the construction resource scheduling agent changes to the construction resource scheduling state, according to the second preset rule, select one or more target construction resource agent objects from multiple construction resource agent objects in the idle state, the state changes to the task assignment state, and send a task assignment message to the target construction resource agent object; the task assignment message is used to prompt the target construction resource agent object to go to the target construction operation surface corresponding to the target construction unit agent object to perform the corresponding construction process operation.

[0085] The first preset rule is used to optimize the resource allocation order, reasonably allocate the construction resource agent objects to specific construction unit agent objects in multiple resource request messages, so as to speed up the construction progress and balance the resource supply and demand.

[0086] The first preset rule specifically includes: based on the priority of the construction unit, giving priority to the construction units with critical paths or construction processes ahead to obtain construction resources; or, based on the urgency of resource requirements, giving priority to the construction units with urgent time and great impact on resource shortage; or, based on the resource allocation efficiency, giving priority to allocating construction resources to the construction units that are closer to the construction resource agent objects. It can be understood that the embodiments of the present application do not limit the specific first preset rule.

[0087] The second preset rule aims to accurately and efficiently select construction resource agent objects to meet the resource requirements of construction unit agent objects.

[0088] The second preset rule specifically includes:

[0089] Based on resource adaptability: preferentially select construction resource agent objects that are highly adaptable to the construction processes of the target construction unit agent objects. For example, if the target construction unit is performing fine welding operations, preferentially select welding equipment agents with high welding accuracy and that can meet the process requirements of the operation.

[0090] Based on performance parameters: preferentially select those with better performance parameters from the construction resource agent objects in the idle state.

[0091] Based on resource scheduling cost: Considering the resource scheduling cost comprehensively, preferentially select the intelligent agent object of construction resources with a short scheduling path and low energy consumption. For example, if there are multiple idle intelligent agent objects of construction resources, preferentially select the intelligent agent object of construction resources that is close to the target construction operation surface and has good traffic conditions on the route to the operation surface, so as to reduce the transportation cost and time cost.

[0092] The following describes the construction process simulation model corresponding to the intelligent agent object of the construction unit.

[0093] In the specific implementation, for the intelligent agent object of the construction unit, analyze the construction processes involved in its construction process and the dependency relationships between the construction processes, and based on the analysis results, simulate and model the construction process of the construction unit to obtain the construction process simulation model corresponding to the intelligent agent object of the construction unit.

[0094] The dependency relationship of the construction process refers to the mutual connection and influence between different construction processes. The above-mentioned dependency relationships specifically include: sequential dependency, parallel dependency, and end dependency. Among them, sequential dependency means that the next construction process can only start after the previous construction process is completed. Parallel dependency means that two construction processes can start simultaneously, but one of the construction processes needs to wait for the construction situation of the other construction process to meet the set conditions. End dependency means that when one construction process is completed, the other construction process must also be completed.

[0095] The above-mentioned construction process simulation model specifically includes: one or more construction process simulation models; the above-mentioned construction process simulation model includes: queue elements and delay elements; the above-mentioned queue elements are used to simulate the queuing and waiting process of the intelligent agent object of construction resources in the construction process; the above-mentioned delay elements are used to simulate the duration of the construction process. When both queue elements and delay elements are used, the queue elements and delay elements can be connected in series.

[0096] The embodiment of the present application can put the intelligent agent object of construction resources that meets the set conditions at the target construction operation surface into the queue represented by the queue elements for queuing and waiting.

[0097] The above-mentioned set conditions specifically include:

[0098] Set condition 1: For the first intelligent agent object of construction resources at the target construction operation surface, the first intelligent agent object of construction resources corresponds to the first resource category, and the number of the first type of intelligent agent object of construction resources at the target construction operation surface exceeds the maximum number of simultaneous work; and / or

[0099] Set condition 2. For the first construction resource agent object at the target construction operation surface, the first construction resource agent object corresponds to the first resource category, and the construction resource agent objects of the first resource category cooperate with the construction resource agent objects of the second resource category to jointly simulate the construction process. The target construction operation surface may refer to the construction operation surface where the construction unit agent object that sends the resource request message is located.

[0100] For Set condition 1, when a construction resource agent object arrives at the target construction operation surface, if there are construction resource agent objects of the same resource category and the number of construction resource agent objects of this resource category exceeds the maximum number that can work simultaneously, the arriving construction resource agent object is placed in the queue represented by the queue element for waiting until a construction resource agent object of the same resource category leaves the current construction operation surface.

[0101] For Set condition 2, when a construction resource agent object arrives at the target construction operation surface, if it is necessary to cooperate with the construction resource agent objects of other resource categories to jointly carry out the construction of the current construction process, the arriving construction resource agent object is placed in the queue represented by the queue element for waiting until the construction resource agent objects of the other resource categories required for cooperation arrive at the target construction operation surface.

[0102] Based on the simulation modeling of a single construction process, by analyzing the dependency relationships of the construction processes involved in the construction flow, in a series or parallel manner, the simulation modeling of the construction flow can be completed.

[0103] Referring to FIG. 5(a), a schematic diagram of the modeling of the construction flow according to an embodiment of the present application is shown. Among them, for the sequential dependency relationship between construction processes, the construction process simulation models between different construction processes can be connected to complete the modeling of the construction flow. For example, the queue element 1 and the delay element 1 included in construction process 1, and the queue element 2 and the delay element 2 included in construction process 2 are connected in series.

[0104] Referring to FIG. 5(b), a schematic diagram of the modeling of the construction flow according to an embodiment of the present application is shown. Among them, for the parallel dependency and end dependency relationships, the construction processes can be independently modeled, and functions are used to implement the modeling of the dependency relationships in a conditional application manner. For example, the queue element 3 and the delay element 3 included in construction process 3 are independently modeled, and the queue element 4 and the delay element 4 included in construction process 4 are independently modeled. Construction process 3 and construction process 4 present a parallel relationship.

[0105] The model parameters of the construction flow simulation model may include: the number of queue elements, the number of delay elements, the maximum allowable capacity of the queue, and the queue queuing rules, etc. The embodiments of the present application can perform the following parameter settings on the construction flow simulation model:

[0106] 1) The maximum allowable capacity of the queue represents the maximum number of intelligent agent objects of construction resources that the queue elements can accommodate. If the queue is full, newly arrived intelligent agent objects of construction resources will become idle and return to wait or be rescheduled to a new construction work surface.

[0107] 2) The queue queuing rule: represents the queuing rule for intelligent agent objects of construction resources of the same resource category, such as first in first out, last in first out, etc.

[0108] Taking the slope excavation operation constructed by the drill and blast method as an example, for the excavation block, that is, the construction unit of the foundation, its construction process involves the following construction operations:

[0109] 1) Surveying and setting out (construction operation A)

[0110] 2) Drilling (construction operation B)

[0111] 3) Charging and wiring (construction operation C)

[0112] 4) Blasting (construction operation D)

[0113] 5) Mucking and loading (construction operation E)

[0114] 6) Slag transportation (construction operation F)

[0115] The dependency relationships between each construction operation are as follows:

[0116] 1) Sequential dependency: A → B (Drilling can only be carried out after surveying and setting out is completed)

[0117] 2) Sequential dependency: B → C (Charging and wiring can only be carried out after drilling is completed)

[0118] 3) Sequential dependency: C → D (Blasting can only be carried out after charging and wiring is completed)

[0119] 4) Sequential dependency: D → E (Mucking and loading can only be carried out after blasting is completed)

[0120] 5) Sequential dependency: E → F (Slag transportation can only be carried out after mucking and loading is completed)

[0121] Then the construction process simulation model of an excavation block is as Figure 6As shown, it specifically includes: construction process simulation models for 6 construction processes. The construction process simulation models for 6 construction processes specifically include: the construction process simulation model for construction process A, the construction process simulation model for construction process B, the construction process simulation model for construction process C, the construction process simulation model for construction process D, the construction process simulation model for construction process E, and the construction process simulation model for construction process F. The construction process simulation model for construction process A specifically includes: queue element A and delay element A. The construction process simulation model for construction process B specifically includes: queue element B and delay element B. The construction process simulation model for construction process C specifically includes: queue element C and delay element C. The construction process simulation model for construction process D specifically includes: queue element D and delay element D. The construction process simulation model for construction process E specifically includes: queue element E and delay element E. The construction process simulation model for construction process F specifically includes: queue element E and delay element F.

[0122] In addition to implementing the modeling of the construction process simulation model, the embodiments of the present application also provide the following simulation process for the construction simulation model: the construction unit agent object sends a resource request message to the construction resource scheduling agent object according to the corresponding construction process simulation model; the construction resource scheduling agent object sends a task assignment message to at least one selected target construction resource agent object according to the resource request message; after receiving the task assignment message, at least one target construction resource agent object moves to the target construction operation surface where the construction unit agent object is located; the construction resource agent objects that meet the set conditions at the target construction operation surface are put into the queue represented by the queue element for queuing and waiting; the construction simulation of the construction process is executed by using the construction resource agent objects that do not need to queue and wait or are taken out from the queue element at the target construction operation surface; after the simulation of the construction process of the construction unit agent object is completed, the corresponding delay element ends.

[0123] This construction simulation model simulates a complete construction process, involving the interaction between multiple parts and the sequential execution of construction processes, and multiple parts work together to complete the simulation of construction processes. The above simulation process specifically includes the following steps:

[0124] Step A1, resource request;

[0125] The construction unit agent object sends a resource request message to the construction resource scheduling agent object according to the corresponding construction process simulation model. Step A1 simulates the situation in actual construction that before starting construction, the construction unit will request the required resources from the department or system responsible for resource scheduling. For example, before performing the mucking and loading construction process, the construction unit agent object needs to request construction resource agent objects such as the excavation equipment agent object and the dump truck equipment agent object from the construction resource scheduling agent object.

[0126] Step A2, Resource Scheduling;

[0127] Based on the received resource request message, the construction resource scheduling agent object sends a task assignment message to at least one selected target construction resource agent object. The construction resource scheduling agent object will select appropriate construction resource agent objects according to factors such as the requested resource category, quantity, distance, etc., and send a task assignment message to notify them to prepare for work. For example, according to the location, status, etc. of the transportation equipment agent object, the transportation equipment agent object that is closest to the construction unit agent object and idle is selected, and a task assignment message is sent to it.

[0128] Step A3, Resource Movement:

[0129] After receiving the task assignment message, at least one target construction resource agent object moves to the target construction operation surface where the construction unit agent object is located. This simulates the process of allocating construction resources to the job site in reality. For example, the selected transportation equipment agent object will drive from its current location to the target construction operation surface.

[0130] The process by which at least one target construction resource agent object moves to the target construction operation surface where the construction unit agent object is located after receiving the task assignment message specifically includes:

[0131] Step B1, Generate a road network model based on the traffic data of the construction operation area;

[0132] Taking the construction scenario of a hydropower project as an example, relying on traffic data such as the general layout plan and the in-site traffic layout CAD (Computer-Aided Design) drawings, the road network information of the area to be modeled is obtained, and a road network model described in the form of path points is generated according to the control point coordinate information in the plane rectangular coordinate system.

[0133] The road network model is a model constructed based on the traffic data of the area to be modeled, aiming to digitally describe the structure and characteristics of the road network within the area to be modeled.

[0134] The road network model specifically includes the following elements:

[0135] Path points: Represent specific positions on the road, and their exact positions are determined by coordinates in the plane rectangular coordinate system. These path points are arranged in sequence along the road's direction, and when connected, they form the path of the road.

[0136] Path segments: Formed by connecting adjacent path points, representing a section of the road. Each path segment has its specific attributes, such as length, direction, road type (such as main road, secondary road, etc.).

[0137] Topological relationship: Describes the connection relationship between path points and path segments, as well as the relationships such as intersections and branches between roads. Through the topological relationship, the structure of the entire road network can be clearly represented, facilitating operations such as path planning and traffic flow analysis.

[0138] Step B2: Set road attribute parameters in the road network model; the road attribute parameters include: road grade, allowable maximum vehicle speed, allowable maximum traffic density, number of lanes, and road surface width;

[0139] Step B3: Set markers for the specified area in the road network model; the specified area is the starting point or ending point of transportation; the specified area includes at least one of: construction operation surface, material storage area, and equipment parking area;

[0140] Among them, the material storage area can refer to the area for material entry and management. The equipment parking area can refer to the area for storing construction resources.

[0141] Step B4: Generate a path file corresponding to the target construction resource agent object according to the road network model, and the path file includes: starting point, ending point, passing points, and path length of transportation;

[0142] Step B5: After receiving the task assignment message, the at least one target construction resource agent object moves from the starting point to the target construction operation surface where the construction unit agent object is located according to the path file.

[0143] The road network model and path file generated through steps B1 - B4 can describe the specific route of the construction resource agent object from the starting point to the target construction operation surface, including detailed information such as the starting point, ending point, and passing points, providing clear navigation for the movement of the construction resource agent object.

[0144] In step B2, road attribute parameters such as the allowable maximum vehicle speed, maximum traffic density, number of lanes, and road surface width are specified in detail, and these road attribute parameters reflect the road traffic capacity and limiting conditions.

[0145] Step B4 can calculate the driving time of each section using the allowable maximum vehicle speed and road length. Or, step B4 can predict the road congestion situation based on the maximum traffic density. Or, step B4 can judge the road bearing capacity by combining the number of lanes and road surface width.

[0146] By comprehensively analyzing the road attribute parameters, step B4 can accurately calculate the lengths and estimated times of different paths, thereby selecting the optimal driving path for the construction resource agent object, enabling the target construction resource agent object to reach the target construction operation surface efficiently and accurately, and ensuring the orderly progress of the construction process.

[0147] Step A4: Queue and wait;

[0148] Put the construction resource agent objects that meet the set conditions at the target construction operation surface into the queue represented by the queue element for queuing and waiting. For example, due to limited construction site, multiple construction equipment cannot construct simultaneously after arrival and can only wait to enter the operation surface in a certain order. The construction equipment that meets a certain set condition (such as first-come, first-served, sorted by equipment performance, etc.) enters the queuing sequence.

[0149] Step A5: Construction simulation;

[0150] Use the construction resource agent objects that do not need to queue and wait or are taken out from the queue elements at the target construction operation surface to perform the construction simulation of the construction process. For example, according to the construction technology and process standards, the excavation equipment agent object simulates the excavation action, controls the excavation depth, angle and range, and the self-unloading equipment agent object simulates the material loading, transportation and unloading process. Through the collaborative simulation of these construction resource agent objects, the actual operation process of the construction process is fully presented.

[0151] Step A6: Process completed.

[0152] After completing the simulation of the i-th construction process of the construction unit agent object, the corresponding delay element ends.

[0153] The delay element is used to simulate the duration of the construction process. Taking the mucking loading as an example, the delay element counts the duration of the entire construction process of mucking loading, specifically including the time for the self-unloading truck to move to the designated position, the operation time for the excavation equipment to excavate and load, and the waiting time of the self-unloading equipment in the queue, etc. These times together constitute the total time required to complete the entire mucking loading process, and the delay element comprehensively counts and accumulates these times to reflect the duration of the entire construction process.

[0154] The construction process simulation model can include: the dependency relationships between different construction processes. In this way, the embodiments of the present application can execute all the construction processes included in the construction process simulation model according to the above dependency relationships.

[0155] For example, Figure 6The construction sequence of the shown construction process simulation model is as follows: First, carry out construction process A (surveying and setting out), and after its completion, carry out construction process B (drilling). After construction process B is completed, then carry out construction process C (charging and connecting wires). After construction process C is completed, start construction process D (blasting). After construction process D is completed, start construction process E (mucking and loading). Finally, on the basis of the completion of construction process E, carry out construction process F (slag transportation). Each construction process is carried out strictly in such a sequential and progressive order. The completion of the previous process creates conditions for the subsequent process to start, and each process is simulated and presented by the corresponding construction process simulation model (queue element and delay element) for the queuing situation and duration in the construction process.

[0156] In specific implementation, the construction simulation model may further include: the connection relationship between different construction process simulation models. The above connection relationship may include: series or parallel, etc. In the embodiments of the present application, by analyzing the construction sequence between construction unit agent objects, the construction process simulation models of each construction unit agent object can be connected in series or parallel to obtain the construction simulation model of the construction operation, that is, a static construction simulation model for the construction operation is constructed. It should be noted that in the static construction simulation model, the number of parallel construction process models will be preset in the initial stage. Subsequently, as the construction operation progresses, the number of parallel construction process simulation models will be dynamically updated according to the analysis results of video images to achieve the accuracy and real-time performance of the construction simulation model.

[0157] In specific implementation, the process data structure can be generated according to the sequential relationship between the construction processes of different construction units; the process data structure is used to describe the sequential relationship between the construction processes of different construction units; according to the process data structure, the construction process simulation models corresponding to different construction units are connected; the connection relationship between different construction process simulation models includes: parallel or series.

[0158] Taking the slope excavation operation as an example, the excavation block numbering sequence can be used to describe the excavation operation sequence between each excavation block, and the construction process models of the excavation blocks are connected in series or parallel in sequence to obtain a static construction simulation model of a certain excavation area.

[0159] Refer to Figure 7, showing a schematic diagram of a construction simulation model according to an embodiment of the present application. Among them, the construction simulation model may include: construction process simulation models A, B, C, and D connected in parallel. Assume that a certain excavation area W is composed of 4 excavation blocks A, B, C, and D, and corresponding construction process simulation models A, B, C, and D are respectively established. If the 4 excavation blocks A, B, C, and D are excavated simultaneously, the simulation model of the excavation area W is Figure 7 As shown, it is composed of construction process simulation models A, B, C, and D connected in parallel, and the completion time is determined by the excavation block that finishes excavation last.

[0160] The embodiment of the present application can execute the construction operations corresponding to all construction unit intelligent agent objects according to the connection relationship between different construction process simulation models.

[0161] The embodiment of the present application can convert the connection relationship between different construction process simulation models in the construction simulation model into a data structure such as a directed graph. The nodes in the directed graph represent the construction process simulation models, and the edges in the directed graph represent the sequence relationship between the construction process simulation models.

[0162] For the series relationship, it is reflected as a linear sequence in the directed graph, that is, after the construction process simulation model corresponding to the previous node is completed, the construction process simulation model corresponding to the next node can start; for the parallel relationship, multiple nodes can simultaneously point to the subsequent same node, which means that the construction process simulation models corresponding to these nodes can be carried out in parallel.

[0163] In this way, the embodiment of the present application can perform topological sorting to clarify the sequence relationship between different construction unit intelligent agent objects, traverse using depth-first or breadth-first search algorithms, simulate parallel execution using multi-threading or processes when encountering parallel processes, and execute strictly in sequence when encountering series processes, while taking into account resource allocation and management, constraint condition handling, etc. throughout the process.

[0164] In step 102, target detection technology, target tracking technology, and clustering technology can be used to automatically determine the construction resource distribution information corresponding to the construction operation surface.

[0165] Among them, the process of using target detection technology, target tracking technology, and clustering technology to determine the construction resource distribution information corresponding to the construction operation surface at the construction operation site according to the video image of the construction operation site specifically includes:

[0166] Step C1: Perform target detection and target tracking on the video image to obtain the detection frames, resource categories, and identification serial numbers corresponding to multiple construction resources in the video image;

[0167] Step C2: Cluster multiple construction resources in the video image according to the center point coordinates corresponding to the detection frames to obtain several clusters; the detection frames within the same cluster belong to the same construction operation surface.

[0168] Step C3: Determine the construction resource distribution information corresponding to the construction operation surface at the construction site according to the resource categories corresponding to the detection frames within the same cluster.

[0169] In step C1, the object detection technology and object tracking technology aim to identify different objects from the input video image and continuously track them, determining the position, category, and identification number of the objects in the image. In the construction operation scenario, the construction resources are the objects to be detected and tracked. By using the pre-trained object detection model and object tracking model (such as a convolutional neural network model based on deep learning, etc.) to process the video image, the object detection model and object tracking model will scan and continuously locate the areas where multiple construction resources are located in the video image according to the learned characteristic patterns of various construction resources. This area is presented in the form of a detection frame, and at the same time, it can determine the category of the resources within each detection frame, such as a certain construction machinery, construction material, or construction worker, etc., and the identification number corresponding to the resources within each detection frame, such as 1, 2, 3, etc. The construction machinery can specifically include: excavation equipment or dump equipment, etc. The detection frame can be the circumscribed rectangle corresponding to the construction resource in the video image.

[0170] In step C2, since the spatial distribution of construction resources at the construction site is regular, the construction resources on the same construction operation surface often have relatively close spatial position characteristics in the input image. The clustering technology operates based on the center point coordinates corresponding to the detection frames, which are attributes that can reflect the spatial scale. Grouping the detection frames with relatively close spatial distances into one category, that is, clustering them into a cluster, means that these construction resources with similar spatial scales in the input image are likely to be on the same construction operation surface. Because in the actual construction operation process, the spatial distances presented by the construction resources that cooperate to complete the same construction process in the image are close. Through this clustering method, the numerous construction resources at the entire construction site can be initially divided according to the construction operation surfaces they belong to, realizing the distinction of construction resources on different construction operation surfaces.

[0171] Assume that the detection frame is represented by the center point coordinates (x, y), and form a two-dimensional data set with the center point coordinates of all detection frames, in the format of:

[0172] data = [[x1, y1], [x2, y2],...]

[0173] The Manhattan distance is used as the similarity metric, and according to the optimization goal of minimizing the intra-cluster distance, a clustering algorithm is used to divide all detected bounding boxes into several clusters. The detected bounding boxes within the same cluster are considered to belong to the same construction operation surface. The number of construction operation surfaces is obtained based on the number of clusters.

[0174] After each cluster (i.e., construction operation surface) is partitioned through clustering, each cluster contains multiple detected bounding boxes, and each detected bounding box corresponds to a specific resource category. By counting the resource categories corresponding to each detected bounding box within the same cluster and the quantities of each resource category, etc., step C3 can determine which specific construction resources are on each construction operation surface and the distribution of the construction resources.

[0175] The construction resource distribution information specifically includes: at least one resource category and the resource quantity corresponding to each resource category.

[0176] In a specific implementation, when meeting the preset invocation conditions, a call request is sent to the video image analysis module, and the video image analysis module determines the construction resource distribution information corresponding to the construction operation surface at the construction operation site based on the video image of the construction operation site.

[0177] The preset invocation conditions specifically include:

[0178] The start of construction operations; and / or

[0179] The update of construction processes; and / or

[0180] The change of construction resources; and / or

[0181] The arrival of the next time period.

[0182] During the construction modeling process, to accurately grasp the construction resource distribution information of each construction operation surface at the construction operation site, it is necessary to rely on the video image analysis module for analysis and processing. When the preset invocation conditions are met, a call request is sent to the video image analysis module. The purpose of this is to ensure timely acquisition of key information while avoiding unnecessary resource waste (such as computing resources, time costs, etc.), enabling the analysis work to be carried out on demand and accurately and efficiently obtaining the construction resource distribution.

[0183] Among them, when the construction operations start, a call request is sent to the video image analysis module to clarify the initial resource configuration of each construction operation surface, providing basic data for the instantiation of the construction simulation model.

[0184] When the construction process is updated, it means that the construction process has switched. Sending a call request to the video image analysis module can determine whether the resource configuration is reasonable under the new construction process, ensuring the smooth connection of construction and the on-demand allocation of resources.

[0185] When construction resources change, a call request is sent to the video image analysis module, enabling quick awareness of resource changes on the construction work surface.

[0186] When the next time period arrives, a call request is sent to the video image analysis module to regularly understand the dynamic changes of resources.

[0187] In a specific implementation, the construction resource distribution information includes: at least one resource category and the corresponding resource quantity for each resource category;

[0188] The process of instantiating queue elements and delay elements in the corresponding construction process simulation model according to the construction resource distribution information specifically includes:

[0189] Step C1: If the resource quantity of a resource category in the construction resource distribution information exceeds the maximum simultaneous working quantity, the construction resource agent objects exceeding the maximum simultaneous working quantity are sorted first according to the arrival time or priority to obtain a first sorting result;

[0190] Step C2: Save the first sorting result to the first queue corresponding to the queue elements in the corresponding construction process simulation model.

[0191] Among them, in the construction resource distribution information, when the quantity of a resource category exceeds the maximum simultaneous working quantity, in order to reasonably arrange these resources, a sorting mechanism needs to be introduced. Sorting according to the arrival time can make the construction resource agent objects that arrive first be considered for use first; sorting by priority can highlight important construction resource agent objects. In this way, the construction resource agent objects exceeding the quantity limit are arranged in an orderly manner.

[0192] On the one hand, the above first sorting can make the use of construction resources more reasonable and orderly, avoid disorderly competition and chaotic allocation among resource agent objects, and ensure the orderly progress of the construction process. On the other hand, the above first sorting creates good conditions for the delay element to accurately record the maintenance time, making the statistics of relevant data such as resource usage duration more logical and accurate, and thus better reflecting the time utilization of construction resources in the entire construction process.

[0193] Step C2 stores the first sorting result obtained in Step C1 into the first queue corresponding to the queue elements in the corresponding construction process simulation model. It realizes the orderly management of construction resource agent objects.

[0194] Optionally, the above method may further include:

[0195] When the simulation of the construction process of the construction resource agent object of a resource category is completed and the current working quantity of a resource category is less than the maximum simultaneous working quantity, the intelligent agent construction resource agent object corresponding to the resource category is taken out from the first queue for further simulation of the construction process.

[0196] In the construction process simulation, taking out the construction resource agent object corresponding to the resource category from the first queue needs to meet the following two conditions:

[0197] Condition 1: The simulation of the construction process of the construction resource agent object of a resource category is completed, that is, some of the intelligent agent objects in this type of resource have completed the current construction tasks assigned to them and are in an idle state.

[0198] Condition 2: The current working quantity of a resource category is less than the maximum simultaneous working quantity, which indicates that there is still remaining available space in this type of resource, allowing more resource intelligent agent objects to be put into work.

[0199] When these two conditions are met simultaneously, the construction resource agent object corresponding to the resource category is taken out from the first queue and put into the further simulation of the construction process to make full use of the resources and ensure the continuity and efficiency of the construction simulation.

[0200] In the specific implementation, the process of instantiating the queue elements and delay elements in the corresponding construction process simulation model according to the construction resource distribution information specifically includes:

[0201] Step D1: According to the collaboration relationship between the first resource category and the second resource category, match the first construction resource agent object and the second construction resource agent object corresponding to the construction resource distribution information to obtain several matching pairs; a matching pair includes: a first construction resource agent object and a second construction resource agent object; the number of the matching pairs does not exceed the maximum simultaneous working quantity;

[0202] Step D2: Perform a second sorting on the first construction resource agent objects not in the matching pairs according to the arrival time or priority to obtain a second sorting result;

[0203] Step D3: Perform a third sorting on the second construction resource agent objects not in the matching pairs according to the arrival time or priority to obtain a third sorting result;

[0204] Step D4: Save the second sorting result to the second queue corresponding to the queue element in the corresponding construction process simulation model, and save the third sorting result to the third queue corresponding to the queue element in the corresponding construction process simulation model.

[0205] Among them, step D1 considers the collaborative relationship between different resource categories, pairs relevant construction resource agent objects, and based on the collaboration, the role of resources can be better exerted, and the number of matching pairs is controlled not to exceed the maximum number of simultaneous operations to ensure the rational use of resources. Through matching, step D1 forms collaborative combinations, improves the resource utilization efficiency, conforms to the resource cooperation situation in actual construction, and helps the construction process simulation to be more accurate.

[0206] For the first construction resource agent objects that did not participate in the matching pairs, step D2 sorts them according to the arrival time or priority to determine their subsequent usage order.

[0207] For the second construction resource agent objects that are not in the matching pairs, step D3 also sorts them according to the arrival time or priority to standardize their usage order.

[0208] Step D4 stores the sorted results into the corresponding second queue and third queue respectively. Using the characteristics of the queue to store the resource order can achieve the orderly storage and management of different resources, facilitating the subsequent extraction of resources in sequence for the construction process simulation.

[0209] In summary, from step D1 to step D4, according to the collaborative relationship and quantity of different resource categories in the construction resource distribution information, the construction resource agent objects are reasonably matched and sorted, and the results are saved to the corresponding queues, providing an orderly and practical collaborative requirement-compliant resource allocation plan for the construction process simulation model, enabling the construction simulation to more accurately reflect the allocation and usage of resources in the actual construction process, thereby assisting construction management and decision-making, and improving construction efficiency and quality.

[0210] Optionally, the above method may further include: when the simulation of a construction process of a matching pair is completed and the number of working matching pairs is less than the maximum number of simultaneous operations, the construction resource agent objects of the corresponding resource categories are respectively taken out from the second and third queues for further simulation of the construction process.

[0211] During the construction process simulation, for the resource matching pairs formed according to the collaborative relationship before, when the simulation of a corresponding construction process of a certain matching pair is completed, the number of working matching pairs will be checked. If the number of working matching pairs is less than the pre-set maximum number of simultaneous operations, it means that there is still room to invest new resources to continue the construction simulation. At this time, the construction resource agent objects of the corresponding resource categories will be respectively taken out from the second queue and the third queue and added to ensure that the construction process simulation can continue and proceed orderly, so that the entire construction simulation is more in line with the situation of dynamic allocation and use of resources in actual construction. Taking out the construction resource agent objects of the corresponding resource categories from the second and third queues respectively can form new matching pairs.

[0212] The delay element is used to count the duration of construction processes, while the queue element stores the sorting information of construction resource agent objects. The state changes and operation sequences of the construction resource agent objects represented by the queue element during the construction process will affect the duration of the construction process, that is, the time counted by the delay element. For example, the waiting time of the construction resource agent objects in the queue is part of the time counted by the delay element.

[0213] In a specific implementation, a delay element instance can be created for each construction process. This delay element instance can be a data structure for storing and accumulating relevant time information. For example, an object containing attributes such as total duration, total waiting time, and total operation time can be created.

[0214] Moreover, calculations can be performed on the total duration, total waiting time, and total operation time.

[0215] Calculation of the total waiting time: Based on the enqueue time and dequeue time of the queue element, calculate the waiting time of each construction resource agent object in the queue and accumulate it into the total waiting time attribute of the delay element.

[0216] Calculation of the total operation time: Based on the start time and end time of each construction operation performed by the construction resource agent object, calculate the duration of each construction operation and accumulate it into the total operation time attribute of the delay element.

[0217] Calculation of the total duration: Add the total waiting time and the total operation time to obtain the total duration of the construction process, and store it in the total duration attribute of the delay element.

[0218] During the construction process, as the construction resource agent objects continuously enter and leave the queue and perform construction operations, the time statistical information of the delay element is continuously updated.

[0219] In an alternative implementation of this application, the construction simulation model may further include: the connection relationships between different construction process simulation models; specifically, the connection relationships include: parallel connection;

[0220] The method may further include: determining the number of construction work surfaces at the construction site based on the video images of the construction site; instantiating the number of parallel construction process models according to the number of construction work surfaces.

[0221] The construction simulation model covers the parallel connection relationship of different construction process simulation models. By analyzing the video images of the construction site and using image recognition and other technologies, the number of construction work surfaces can be determined based on visual features such as construction resource distribution and construction activity scope. Since each construction work surface is relatively independent and has specific construction tasks, the parallel construction process model is instantiated according to the number of construction work surfaces, so that the parallel process in the construction simulation model matches the parallel construction of multiple work surfaces in actual construction, so that the simulation is more in line with the actual construction status.

[0222] By instantiating the parallel construction process model according to the number of construction work surfaces, the fit between the construction simulation model and the actual construction can be significantly improved, and the scene of simultaneous construction of multiple construction work surfaces can be accurately simulated, providing a reliable reference for construction decision-making. At the same time, this method also helps to optimize the configuration and scheduling of construction resources, clearly present the resource requirements of each work surface, avoid resource waste, and improve utilization efficiency.

[0223] In summary, the automatic construction method of the construction simulation model based on video images of the embodiment of the present application adopts target detection technology, target tracking technology and clustering technology to analyze the video images of the construction site, so as to quickly and automatically determine the construction resource distribution information, and use the automatically determined construction resource distribution information for the instantiation of queue elements and delay elements. On the one hand, because the embodiment of the present application does not need to manually count the construction resource distribution information one by one, the collection time of the construction resource distribution information can be shortened. On the other hand, in view of the fact that the embodiment of the present application eliminates the link of manually inputting the construction resource distribution information, it is possible to save manual input time. In summary, the embodiment of the present application can improve the construction efficiency of the construction simulation model.

[0224] It should be noted that, for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0225] Based on the above embodiment, this embodiment also provides a device for automatically building a construction simulation model based on video images. Figure 8 The device may specifically include: a model building module 801, a video analysis module 802 and an instantiation module 803.

[0226] Among them, the model construction module 801 is used to establish a construction simulation model for the construction operation. The construction simulation model includes: construction unit intelligent agent objects, construction resource scheduling intelligent agent objects, construction resource intelligent agent objects, and construction process simulation models corresponding to multiple construction unit intelligent agent objects respectively. The construction process simulation model includes: one or more construction process simulation models. The construction process simulation model includes: queue elements and delay elements. The queue elements are used to simulate the queuing and waiting process of construction resource intelligent agent objects in the construction process. The delay elements are used to simulate the duration of the construction process.

[0227] The simulation process of the construction simulation model includes: the construction unit intelligent agent object sends a resource request message to the construction resource scheduling intelligent agent object according to the corresponding construction process simulation model; the construction resource scheduling intelligent agent object sends a task assignment message to at least one selected target construction resource intelligent agent object according to the resource request message; after receiving the task assignment message, at least one target construction resource intelligent agent object moves to the target construction operation surface where the construction unit intelligent agent object is located; the construction resource intelligent agent objects that meet the set conditions at the target construction operation surface are put into the queue represented by the queue elements for queuing and waiting; the construction simulation of the construction process is performed by using the construction resource intelligent agent objects that do not need to queue and wait or are taken out from the queue elements at the target construction operation surface; after the simulation of the construction process of the construction unit intelligent agent object is completed, the corresponding delay element ends.

[0228] The video analysis module 802 is used to determine the construction resource distribution information corresponding to the construction operation surface at the construction operation site according to the video image of the construction operation site by using target detection technology, target tracking technology and clustering technology.

[0229] The instantiation module 803 is used to instantiate the queue elements and delay elements in the corresponding construction process simulation model according to the construction resource distribution information.

[0230] Optionally, the video analysis module includes:

[0231] The target detection and target tracking module is used to perform target detection on the video image to obtain the detection frames, resource categories and identification serial numbers corresponding to multiple construction resources in the video image.

[0232] The clustering module is used to cluster multiple construction resources in the video image according to the center point coordinates corresponding to the detection frames to obtain several clusters; the detection frames within the same cluster belong to the same construction operation surface.

[0233] The distribution determination module is used to determine the construction resource distribution information corresponding to the construction operation surface at the construction operation site according to the resource categories corresponding to the detection frames within the same cluster.

[0234] Optionally, the video analysis module is specifically configured to send a call request to the video image analysis module when a preset call condition is met, and the video image analysis module determines the construction resource distribution information corresponding to the construction operation surface at the construction operation site according to the video image of the construction operation site;

[0235] The preset call conditions include:

[0236] The start of construction operations; and / or

[0237] The update of the construction process; and / or

[0238] The change of construction resources; and / or

[0239] The arrival of the next time period.

[0240] Optionally, the construction simulation model further includes: the connection relationship between different construction process simulation models; the connection relationship includes: parallel connection;

[0241] The device further includes:

[0242] Determine the number of construction operation surfaces at the construction operation site according to the video image of the construction operation site;

[0243] Instantiate the number of parallel construction process models according to the number of construction operation surfaces.

[0244] Optionally, the set conditions include:

[0245] For the first construction resource intelligent agent object at the target construction operation surface, the first construction resource intelligent agent object corresponds to the first resource category, and the number of the first type of construction resource intelligent agent objects at the target construction operation surface exceeds the maximum simultaneous working number; and / or

[0246] For the first construction resource intelligent agent object at the target construction operation surface, the first construction resource intelligent agent object corresponds to the first resource category, and the construction resource intelligent agent objects of the first resource category cooperate with the construction resource intelligent agent objects of the second resource category to jointly simulate the construction process.

[0247] Optionally, the construction resource distribution information includes: at least one resource category, and the resource quantity corresponding to each resource category;

[0248] The instantiation module includes:

[0249] The first sorting module is used to, if the resource quantity of a resource category in the construction resource distribution information exceeds the maximum simultaneous working number, perform a first sorting on the construction resource intelligent agent objects that exceed the maximum simultaneous working number according to the arrival time or priority, and obtain a first sorting result;

[0250] A first storage module, configured to store the first sorting result into a first queue corresponding to a queue element in a corresponding construction process simulation model.

[0251] Optionally, the device further includes:

[0252] A first dequeue module, configured to, when a construction resource agent object of a resource category completes the simulation of a construction process and the current number of working units of a resource category is less than the maximum number of simultaneous working units, take out the agent construction resource agent object of the corresponding resource category from the first queue for further simulation of the construction process.

[0253] Optionally, the construction resource distribution information includes: at least one resource category, and the resource quantity corresponding to each resource category;

[0254] The instantiation module includes:

[0255] A matching module, configured to match a first construction resource agent object and a second construction resource agent object corresponding to the construction resource distribution information according to the cooperation relationship between a first resource category and a second resource category, so as to obtain a plurality of matching pairs; a matching pair includes: a first construction resource agent object and a second construction resource agent object; the number of the matching pairs does not exceed the maximum number of simultaneous working units;

[0256] A second sorting module, configured to perform a second sorting on the first construction resource agent objects not in the matching pairs according to the arrival time or priority to obtain a second sorting result;

[0257] A third sorting module, configured to perform a third sorting on the second construction resource agent objects not in the matching pairs according to the arrival time or priority to obtain a third sorting result;

[0258] A second storage module, configured to store the second sorting result into a second queue corresponding to a queue element in a corresponding construction process simulation model, and store the third sorting result into a third queue corresponding to a queue element in a corresponding construction process simulation model.

[0259] Optionally, the device further includes:

[0260] A second dequeue module, configured to, when a matching pair completes the simulation of a construction process and the number of working matching pairs is less than the maximum number of simultaneous working units, take out the construction resource agent objects of the corresponding resource categories from the second and third queues respectively for further simulation of the construction process.

[0261] Optionally, after receiving the task assignment message, the at least one target construction resource agent object moves to the target construction operation surface where the construction unit agent object is located, including:

[0262] Generate a road network model based on the traffic data of the construction operation area;

[0263] Set road attribute parameters in the road network model; the road attribute parameters include: road grade, allowed maximum vehicle speed, allowed maximum traffic density, number of lanes, and road surface width;

[0264] Set marks for the set areas in the road network model; the set areas are the starting point or the ending point of transportation; the set areas include at least one of: construction operation surface, material storage area, and equipment parking area;

[0265] Generate a path file corresponding to the target construction resource agent object according to the road network model, and the path file includes: starting point, ending point, passing points, and path length of transportation;

[0266] After receiving the task assignment message, the at least one target construction resource agent object moves from the starting place to the target construction operation surface where the construction unit agent object is located according to the path file.

[0267] Optionally, the device further includes:

[0268] A data structure generation module, configured to generate a process data structure according to the sequential relationship between the construction processes of different construction units; the process data structure is used to describe the sequential relationship between the construction processes of different construction units;

[0269] A connection module, configured to connect the construction process simulation models corresponding to different construction units according to the process data structure; the connection relationships between different construction process simulation models include: parallel or serial.

[0270] Optionally, the states of the construction resource scheduling agent object include: initial state, construction task existence judgment state, waiting for construction task state, service construction simulation object selection state, idle construction resource judgment state, waiting for construction resource idle state, construction resource scheduling state, task distribution, etc. states;

[0271] After the construction simulation model starts to simulate, the state of the construction resource scheduling agent changes from the initial state to the construction task existence judgment state. If a resource request message sent by any construction unit agent object is received, it is considered that there is a construction task, and the state changes to the service construction simulation object selection state; if no resource request message is received, the state changes to the waiting for construction task state, and the loop judgment continues until a resource request message is received;

[0272] When the state of the construction resource scheduling agent changes to the service construction simulation object selection state, according to the first preset rule, select a target construction unit agent object from multiple construction unit agent objects that send requests, and the state changes to the idle construction resource judgment state; if there is a construction resource agent object in the idle state, the state changes to the construction resource scheduling state; if not, the state changes to the waiting for construction resource idle state, and loop to judge until the state of a construction resource agent changes to the idle state.

[0273] When the state of the construction resource scheduling agent changes to the construction resource scheduling state, according to the second preset rule, select one or more target construction resource agent objects from multiple construction resource agent objects in the idle state, the state changes to the task assignment state, and send a task assignment message to the target construction resource agent object; the task assignment message is used to prompt the target construction resource agent object to go to the target construction operation surface corresponding to the target construction unit agent object to perform the operation of the corresponding construction process.

[0274] The embodiment of the present application also provides a non-volatile readable storage medium, in which one or more modules (prograPs) are stored. When the one or more modules are applied to a device, the device can be made to execute the instructions (instructions) of the method steps in the embodiment of the present application.

[0275] The embodiment of the present application provides one or more machine-readable media, on which instructions are stored. When executed by one or more processors, the electronic device is made to execute the method as described in one or more of the above embodiments. In the embodiment of the present application, the electronic device includes various types of devices such as terminal devices and servers (clusters).

[0276] The embodiment of the present application provides a computer program product containing instructions. When it runs on a computer, the computer is made to execute the method for automatically constructing a construction simulation model based on video images as described in any one of the above embodiments.

[0277] The embodiments of the present disclosure can be implemented as a device configured as desired using any suitable hardware, firmware, software, or any combination thereof. The device may include: electronic devices such as terminal devices and servers (clusters). Figure 9 Schematically shows an exemplary device 1300 that can be used to implement the various embodiments described in the present application.

[0278] For one embodiment, Figure 9An exemplary device 1300 is shown, which has one or more processors 1302, a control module (chipset) 1304 coupled to at least one of the (one or more) processors 1302, a memory 1306 coupled to the control module 1304, an NVP (non-volatile memory) / storage device 1308 coupled to the control module 1304, one or more input / output devices 1310 coupled to the control module 1304, and a network interface 1312 coupled to the control module 1304.

[0279] The processor 1302 may include one or more single-core or multi-core processors, and the processor 1302 may include any combination of general-purpose processors or dedicated processors (such as graphics processors, application processors, baseband processors, etc.). In some embodiments, the device 1300 can function as devices such as the terminal device, server (cluster), etc. described in the embodiments of the present application.

[0280] In some embodiments, the device 1300 may include one or more computer-readable media (such as the memory 1306 or non-volatile memory / storage device 1308) having instructions 1314, and one or more processors 1302 combined with the one or more computer-readable media and configured to execute the instructions 1314 to implement modules to perform the actions described in the present disclosure.

[0281] For one embodiment, the control module 1304 may include any suitable interface controller to provide any suitable interface to at least one of the (one or more) processors 1302 and / or any suitable device or component communicating with the control module 1304.

[0282] The control module 1304 may include a memory controller module to provide an interface to the memory 1306. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0283] The memory 1306 may be used, for example, to load and store data and / or instructions 1314 for the device 1300. For one embodiment, the memory 1306 may include any suitable volatile memory, such as a suitable DRAP (Dynamic Random Access Memory). In some embodiments, the memory 1306 may include a double data rate type four synchronous dynamic random access memory.

[0284] For one embodiment, the control module 1304 may include one or more input / output controllers to provide an interface to the non-volatile memory / storage device 1308 and the (one or more) input / output devices 1310.

[0285] For example, the non-volatile memory / storage device 1308 can be used to store data and / or instructions 1314. The non-volatile memory / storage device 1308 can include any suitable non-volatile memory (e.g., flash memory) and / or can include any suitable (one or more) non-volatile storage devices (e.g., one or more hard disk drives, one or more optical disk drives, and / or one or more digital versatile disk drives).

[0286] The non-volatile memory / storage device 1308 can include storage resources that are physically part of a device mounted on the apparatus 1300, or it can be accessible to the apparatus without being part of the apparatus. For example, the non-volatile memory / storage device 1308 can be accessed via a network through the (one or more) input / output devices 1310.

[0287] (One or more) input / output devices 1310 can provide an interface for the apparatus 1300 to communicate with any other suitable devices. The input / output devices 1310 can include communication components, audio components, sensor components, etc. The network interface 1312 can provide an interface for the apparatus 1300 to communicate through one or more networks. The apparatus 1300 can wirelessly communicate with one or more components of a wireless network according to any one of one or more wireless network standards and / or protocols. For example, it can access a wireless network based on a communication standard, such as WiFi (Wireless Fidelity), 2G (2-Generation wireless telephone technology), 3G (3-Generation wireless telephone technology), 4G (4-Generation wireless telephone technology), 5G (5-Generation wireless telephone technology), etc., or a combination thereof for wireless communication.

[0288] For one embodiment, at least one of the (one or more) processors 1302 may be logically packaged together with one or more controllers of the control module 1304 (e.g., a memory controller module). For one embodiment, at least one of the (one or more) processors 1302 may be logically packaged together with one or more controllers of the control module 1304 to form a system-in-package. For one embodiment, at least one of the (one or more) processors 1302 may be logically integrated with one or more controllers of the control module 1304 on the same die. For one embodiment, at least one of the (one or more) processors 1302 may be logically integrated with one or more controllers of the control module 1304 on the same die to form a system-on-chip.

[0289] In various embodiments, the device 1300 can be, but is not limited to, a server, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a touchscreen device, a netbook, etc.), such as a terminal device. In various embodiments, the device 1300 may have more or fewer components and / or a different architecture. For example, in some embodiments, the device 1300 includes one or more cameras, a keyboard, a liquid crystal display screen (including a touchscreen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit, and speakers.

[0290] Among them, a main control chip can be used as a processor or a control module in the detection device, sensor data, location information, etc. are stored in a memory or a non-volatile memory / storage device, the sensor group can be used as an input / output device, and the communication interface can include a network interface.

[0291] For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For the relevant parts, refer to the partial description of the method embodiment.

[0292] Each embodiment in this specification is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other.

[0293] Embodiments of the present application are described with reference to the flowcharts and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing terminal devices generate a device for implementing the functions specified in one or more flows of the flowchart and / or one or more blocks of the block diagram.

[0294] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that implements the functions specified in one or more flows of the flowchart and / or one or more blocks of the block diagram.

[0295] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, such that a series of operation steps are executed on the computer or other programmable terminal device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one or more flows of the flowchart and / or one or more blocks of the block diagram.

[0296] Although the preferred embodiments of the embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concept. Therefore, the appended claims are intended to be construed as including the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present application.

[0297] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or elements inherent to such process, method, article or terminal device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or terminal device including the said element.

[0298] The above has introduced in detail a method and apparatus for automatically constructing a construction simulation model based on video images, an electronic device, and a machine-readable medium provided by the present application. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scenarios. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. An automatic construction simulation model building method based on video images, characterized in that The method includes: For the construction operation, a construction simulation model is established; the construction simulation model includes: construction unit agent objects, construction resource scheduling agent objects, construction resource agent objects, and construction process simulation models corresponding to multiple construction unit agent objects respectively; the construction process simulation model includes: one or more construction operation simulation models; the construction operation simulation model includes: queue elements and delay elements; the queue elements are used to simulate the queuing and waiting process of construction resource agent objects in the construction operation; the delay elements are used to simulate the duration of the construction operation; The simulation process of the construction simulation model includes: the construction unit agent object sends a resource request message to the construction resource scheduling agent object according to the corresponding construction process simulation model; the construction resource scheduling agent object sends a task assignment message to at least one selected target construction resource agent object according to the resource request message; after receiving the task assignment message, at least one target construction resource agent object moves to the target construction operation surface where the construction unit agent object is located; the construction resource agent objects that meet the set conditions at the target construction operation surface are put into the queue represented by the queue elements for queuing and waiting; the construction simulation of the construction operation is performed by using the construction resource agent objects that do not need to queue and wait or are taken out from the queue elements at the target construction operation surface; after the simulation of the construction operation of the construction unit agent object is completed, the corresponding delay element ends; According to the video image of the construction operation site, the construction resource distribution information corresponding to the construction operation surface of the construction operation site is determined by using object detection technology, object tracking technology and clustering technology; According to the construction resource distribution information, the queue elements and delay elements in the corresponding construction operation simulation model are instantiated; a delay element instance is created for each construction operation, and the delay element instance is used to store and accumulate relevant time information; the relevant time information includes: total duration, total waiting time, total operation time; among them, according to the entry queue time and departure queue time of the construction resource agent object, the waiting time of each construction resource agent object in the queue corresponding to the queue element is calculated, and the waiting time is accumulated into the total waiting time of the construction resource agent object; according to the start time and end time of each construction operation executed by the construction resource agent object, the duration of each construction operation is calculated, and the duration is accumulated into the total operation time of the delay element instance; the total waiting time and the total operation time are added together to obtain the total duration of the construction operation; Among them, determining the construction resource distribution information corresponding to the construction operation surface at the construction operation site according to the video image of the construction operation site includes: performing object detection and object tracking on the video image to obtain the detection frames, resource categories, and identification serial numbers corresponding to multiple construction resources in the video image; clustering the multiple construction resources in the video image according to the central point coordinates corresponding to the detection frames to obtain several clusters; the detection frames within the same cluster belong to the same construction operation surface; determining the construction resource distribution information corresponding to the construction operation surface at the construction operation site according to the resource categories corresponding to the detection frames within the same cluster.

2. The method according to claim 1, characterized in that, Determining the construction resource distribution information corresponding to the construction operation surface at the construction operation site according to the video image of the construction operation site includes: When meeting the preset calling conditions, sending a calling request to the video image analysis module, and the video image analysis module determines the construction resource distribution information corresponding to the construction operation surface at the construction operation site according to the video image of the construction operation site; The preset calling conditions include: The start of construction operations; and / or The update of construction processes; and / or The change of construction resources; and / or The arrival of the next time period.

3. The method according to claim 1, wherein The construction simulation model further includes: the connection relationship between different construction process simulation models; the connection relationship includes: parallel connection; The method further includes: Determining the number of construction operation surfaces at the construction operation site according to the video image of the construction operation site; Instantiating the number of parallel construction process models according to the number of construction operation surfaces.

4. The method according to claim 1, characterized in that The set conditions include: For the first construction resource intelligent agent object at the target construction operation surface, the first construction resource intelligent agent object corresponds to the first resource category, and the number of the first type of construction resource intelligent agent objects at the target construction operation surface exceeds the maximum simultaneous working number; and / or For the first construction resource intelligent agent object at the target construction operation surface, the first construction resource intelligent agent object corresponds to the first resource category, and the construction resource intelligent agent objects of the first resource category cooperate with the construction resource intelligent agent objects of the second resource category to jointly simulate the construction process.

5. The method according to claim 1, wherein The construction resource distribution information includes: at least one resource category, and the resource quantity corresponding to each resource category; Instantiating the queue elements and delay elements in the corresponding construction process simulation model according to the construction resource distribution information includes: If the resource quantity of a resource category in the construction resource distribution information exceeds the maximum simultaneous working number, then perform the first sorting on the construction resource intelligent agent objects exceeding the maximum simultaneous working number according to the arrival time or priority to obtain the first sorting result; Save the first sorting result to the first queue corresponding to the queue elements in the corresponding construction process simulation model.

6. The method according to claim 5, wherein The method further includes: In the case where the construction resource intelligent agent objects of a resource category complete the simulation of the construction process and the current working number of a resource category is less than the maximum simultaneous working number, take out the intelligent agent construction resource intelligent agent objects corresponding to the resource category from the first queue for the simulation of the construction process.

7. The method according to claim 1, characterized in that, The construction resource distribution information includes: at least one resource category and the corresponding resource quantity for each resource category; Instantiating queue elements and delay elements in the corresponding construction process simulation model according to the construction resource distribution information includes: Matching the first construction resource agent object and the second construction resource agent object corresponding to the construction resource distribution information according to the cooperation relationship between the first resource category and the second resource category to obtain a number of matching pairs; a matching pair includes: a first construction resource agent object and a second construction resource agent object; the number of matching pairs does not exceed the maximum simultaneous working quantity; Performing a second sorting on the first construction resource agent objects not in the matching pairs according to the arrival time or priority to obtain a second sorting result; Performing a third sorting on the second construction resource agent objects not in the matching pairs according to the arrival time or priority to obtain a third sorting result; Saving the second sorting result to the second queue corresponding to the queue elements in the corresponding construction process simulation model, and saving the third sorting result to the third queue corresponding to the queue elements in the corresponding construction process simulation model.

8. The method according to claim 7, characterized in that, The method further includes: When a matching pair completes the simulation of the construction process and the number of working matching pairs is less than the maximum simultaneous working quantity, taking out the construction resource agent objects of the corresponding resource categories from the second and third queues respectively for the simulation of the construction process.

9. The method according to any one of claims 1 to 8, characterized in that The method further includes: Generating a process data structure according to the sequential relationship between the construction processes of different construction units; the process data structure is used to describe the sequential relationship between the construction processes of different construction units; Connecting the construction process simulation models corresponding to different construction units according to the process data structure; the connection relationship between different construction process simulation models includes: parallel or series.

10. The method according to any one of claims 1 to 8, characterized in that, The states of the construction resource scheduling agent object include: initial state, construction task existence judgment state, waiting for construction task state, service construction simulation object selection state, idle construction resource judgment state, waiting for construction resource idle state, construction resource scheduling state, task issuance state; After the construction simulation model starts to simulate, the state of the construction resource scheduling agent changes from the initial state to the construction task existence judgment state. If a resource request message sent by any construction unit agent object is received, it is considered that there is a construction task and the state changes to the service construction simulation object selection state; if no resource request message is received, the state changes to the waiting for construction task state and loops to judge until a resource request message is received; When the state of the construction resource scheduling agent changes to the service construction simulation object selection state, selecting a target construction unit agent object from multiple construction unit agent objects that send requests according to the first preset rule, and the state changes to the idle construction resource judgment state; if there are construction resource agent objects in the idle state, the state changes to the construction resource scheduling state; if not, the state changes to the waiting for construction resource idle state and loops to judge until the state of a construction resource agent changes to the idle state; When the state of the construction resource scheduling agent changes to the construction resource scheduling state, according to the second preset rule, one or more target construction resource agent objects are selected from multiple construction resource agent objects in the idle state, the state changes to the task assignment state, and a task assignment message is sent to the target construction resource agent object; the task assignment message is used to prompt the target construction resource agent object to go to the target construction surface corresponding to the target construction unit agent object to perform the operations of corresponding construction processes.

11. An automatic construction simulation model building device based on video images, characterized in that, The device includes: A model construction module, which is used to establish a construction simulation model for the construction operation; the construction simulation model includes: construction unit agent objects, construction resource scheduling agent objects, construction resource agent objects, and construction process simulation models respectively corresponding to multiple construction unit agent objects; the construction process simulation model includes: one or more construction process simulation models; the construction process simulation model includes: queue elements and delay elements; the queue elements are used to simulate the queuing and waiting process of construction resource agent objects in the construction process; the delay elements are used to simulate the duration of the construction process; The simulation process of the construction simulation model includes: the construction unit agent object sends a resource request message to the construction resource scheduling agent object according to the corresponding construction process simulation model; the construction resource scheduling agent object sends a task assignment message to at least one selected target construction resource agent object according to the resource request message; after receiving the task assignment message, at least one target construction resource agent object moves to the target construction surface where the construction unit agent object is located; the construction resource agent objects that meet the set conditions at the target construction surface are put into the queue represented by the queue elements for queuing and waiting; the construction simulation of the construction process is performed by using the construction resource agent objects that do not need to queue and wait or are taken out from the queue elements at the target construction surface; after the simulation of the construction process of the construction unit agent object is completed, the corresponding delay element ends; A video analysis module, which is used to determine the construction resource distribution information corresponding to the construction surface of the construction operation site according to the video image of the construction operation site by using object detection technology, object tracking technology and clustering technology; An instantiation module is used to instantiate queue elements and delay elements in the simulation model of corresponding construction processes according to the construction resource distribution information; create a delay element instance for each construction process, and the delay element instance is used to store and accumulate relevant time information; the relevant time information includes: total duration, total waiting time, and total operation time; wherein, according to the entry queue time and departure queue time of the construction resource agent object, calculate the waiting time of each construction resource agent object in the corresponding queue of the queue element, and accumulate the waiting time into the total waiting time of the construction resource agent object; according to the start time and end time of each construction operation of the construction resource agent object, calculate the duration of each construction operation, and accumulate the duration into the total operation time of the delay element instance; add the total waiting time and the total operation time to obtain the total duration of the construction process. Among them, the construction resource distribution information corresponding to the construction operation surface at the construction operation site is determined by using object detection technology, object tracking technology and clustering technology based on the video image of the construction operation site, including: performing object detection and object tracking on the video image to obtain the detection frames, resource categories and identification serial numbers corresponding to multiple construction resources in the video image; clustering the multiple construction resources in the video image according to the center point coordinates corresponding to the detection frames to obtain several clusters; the detection frames within the same cluster belong to the same construction operation surface; determine the construction resource distribution information corresponding to the construction operation surface at the construction operation site according to the resource categories corresponding to the detection frames within the same cluster.

12. An electronic device, characterized in that, Including: A processor; and A memory, on which executable code is stored, and when the executable code is executed, the processor is caused to execute the method according to any one of claims 1-10.

13. A machine-readable medium, on which executable code is stored, and when the executable code is executed, the processor is caused to execute the method according to any one of claims 1-10.

14. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, the method according to any one of claims 1-10 is implemented.

Images