Intelligent identification method and device for construction machinery and equipment activities

Abstract

The present invention provides a method and device for intelligently identifying the activities of construction machinery and equipment, comprising: decomposing an acquired monitoring task into several subtasks; determining the required main and auxiliary sensors based on all subtasks; executing the subtasks using data uploaded by the main sensor, and determining whether to enable the auxiliary sensor based on the execution requirements of the subtask; and providing real-time warnings for abnormal situations occurring during the execution of the subtasks. This application installs sensors on construction machinery and equipment, splitting the received task into several subtasks, and using vector formulas to evaluate whether the data collected by the main sensor can meet the requirements of the subtasks. When the data is below a threshold, the auxiliary sensor is enabled to obtain atomic motion information of the construction machinery and equipment to complete the activity monitoring task, thereby realizing the function of intelligent and comprehensive monitoring of the construction site and solving the coordination problem between the main and auxiliary sensors.

Description

Technical Field

[0001] The present application belongs to the field of smart construction site technology, and specifically relates to a method and device for intelligently identifying activities of construction machinery and equipment based on multi-source data fusion. Background Art

[0002] "Smart construction site" refers to the use of information and digital means to accurately design and simulate construction projects through a 3D design platform. It mainly focuses on the management links in the construction process, establishes an information ecosystem for construction projects with interconnected collaboration, intelligent production, and scientific management, and conducts data mining and analysis on engineering information collected by the Internet of Things in a virtual environment, thereby providing construction process trend forecasts and expert plans, realizing visual and intelligent management of engineering construction data, and improving the level of informationization of engineering management.

[0003] However, in current "smart construction site" projects, there are several main problems with collecting information through sensors:

[0004] 1. Single sensors alone cannot meet the requirements for identifying and monitoring dynamic construction element activities. Construction site conditions are complex, and dynamic construction element activities interact across multiple levels and objects. Furthermore, the resources available for monitoring dynamic construction elements within a project are limited. No single automatic detection and identification technology for construction element activities can independently meet the economic and technical requirements required to effectively monitor dynamic construction elements under these numerous constraints.

[0005] 2. The application concept of "single sensor as the main and multiple sensors as the auxiliary" is still in its early stages and fails to organically combine various types of data to achieve efficient and low-cost "analysis" and effective "feedback".

[0006] Although many people in the industry have proposed using multiple sensors to solve the defects of single sensors, the following problems still exist:

[0007] 1) There is no unified system to automatically integrate and analyze the multi-dimensional data collected by various sensors and then provide feedback to the various demand-side entities.

[0008] 2) Data "perception" by various types of sensors mechanically collects, transmits, and stores data, failing to leverage the redundancy of information contained in multivariate data to effectively reduce the amount of data collected, transmitted, and stored. In particular, visual data (videos and photos), which cover all scenarios and all time periods but contain a large amount of redundant information, occupies a large amount of automatic monitoring resources.

[0009] 3) The data from various sensors are not used for real-time, dynamic monitoring of construction elements and advance control and early warning, but are limited to "after-the-fact" tracing. Summary of the Invention

[0010] The present application provides a method and device for intelligently identifying the activities of construction machinery and equipment based on multi-source data fusion, in order to at least solve the problem that the single sensor currently installed on the construction machinery and equipment cannot adapt to the needs of identifying and monitoring the activities of dynamic construction elements.

[0011] According to one aspect of the present application, a method for intelligently identifying construction machinery and equipment activities based on multi-source data fusion is provided, comprising:

[0012] Decompose the acquired recognition task into several subtasks;

[0013] Determine the required primary and auxiliary sensors based on all subtasks;

[0014] Execute subtasks based on data uploaded by the main sensor, and determine whether to enable auxiliary sensors based on the subtask execution requirements;

[0015] Provide real-time warning or analysis of situations that occur during the execution of subtasks.

[0016] In one embodiment, the acquired recognition task is decomposed into several subtasks, including:

[0017] Perform field parsing for recognition tasks;

[0018] Extract the identification object field in the identification task, where the identification object is a construction machine;

[0019] The recognition task is decomposed into several subtasks according to the work type of the recognition object and the corresponding completion time of each work type.

[0020] In one embodiment, determining the required primary sensors and auxiliary sensors based on all subtasks includes:

[0021] Determine the sensors required for each subtask;

[0022] Select the sensor with the highest demand from the sensors as the main sensor;

[0023] The remaining sensors are used as auxiliary sensors.

[0024] In one embodiment, determining whether to enable the auxiliary sensor according to the execution requirements of the subtask includes:

[0025] Based on the subtask completion and evaluation confidence, it is determined in real time whether the data uploaded by the main sensor meets the execution requirements of the subtask;

[0026] If the conditions are not met, the auxiliary sensor is enabled to collect data and the subtask is continued to be executed based on the data collected by the auxiliary sensor.

[0027] According to another aspect of the present application, there is also provided an intelligent identification device for construction machinery and equipment activities based on multi-source data fusion, comprising:

[0028] A task decomposition unit, used to decompose the acquired recognition task into several subtasks;

[0029] A sensor selection unit is used to determine the required main sensors and auxiliary sensors according to all subtasks, and the main sensors and auxiliary sensors are set on the construction machinery and equipment;

[0030] The auxiliary sensor enabling unit is used to execute subtasks based on the data uploaded by the main sensor and determine whether to enable the auxiliary sensor according to the execution requirements of the subtask;

[0031] The real-time warning unit is used to provide real-time warnings for abnormal situations that occur during the execution of subtasks.

[0032] In one embodiment, the task decomposition unit includes:

[0033] Field parsing module, used to perform field parsing for recognition tasks;

[0034] The recognition object extraction module is used to extract the recognition object field in the recognition task, and the recognition object is a construction machinery and equipment;

[0035] The subtask decomposition module is used to decompose the recognition task into several subtasks according to the work type of the recognition object and the completion time corresponding to each work type.

[0036] In one embodiment, the sensor selection unit includes:

[0037] A sensor determination module is used to determine the sensors required for each subtask;

[0038] A main sensor determination module is used to select the sensor with the highest demand from the sensors as the main sensor;

[0039] The auxiliary sensor determination module is used to use the remaining sensors as auxiliary sensors.

[0040] In one embodiment, the auxiliary sensor enabling unit comprises:

[0041] An evaluation module, configured to determine in real time whether the data uploaded by the main sensor meets the execution requirements of the subtask based on the subtask completion degree and evaluation confidence;

[0042] The auxiliary sensor module is enabled, and is used for enabling the auxiliary sensor to collect data and continue to execute the subtask according to the data collected by the auxiliary sensor when the condition is not satisfied.

[0043] This application installs sensors on construction machinery and equipment to split the received tasks into several sub-tasks, and then uses the main sensor to monitor each atomic action according to the requirements of the sub-task. At the same time, it uses the vector formula to evaluate whether the data collected by the main sensor can meet the needs of the sub-task. When it is lower than the threshold, the auxiliary sensor is enabled to obtain the atomic action information of the construction machinery and equipment to complete the activity supervision task, realizing the function of intelligent and comprehensive monitoring of the construction site, and solving the coordination problem between the main sensor and the auxiliary sensor. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 This application provides a flow chart of a method for intelligently identifying activities of construction machinery and equipment.

[0046] Figure 2 This is a flow chart of a method for decomposing an acquired recognition task into several subtasks in an embodiment of the present application.

[0047] Figure 3 This is a flow chart of a method for determining required primary sensors and auxiliary sensors in an embodiment of the present application.

[0048] Figure 4 This is a flowchart of determining whether to enable an auxiliary sensor based on the execution requirements of a subtask in an embodiment of the present application.

[0049] Figure 5 This is a structural block diagram of a method and device for intelligently identifying activities of construction machinery and equipment provided in this application.

[0050] Figure 6 This is a structural block diagram of the task decomposition unit in an embodiment of the present application.

[0051] Figure 7 This is a structural block diagram of the sensor selection unit in an embodiment of the present application.

[0052] Figure 8 This is a structural block diagram of the auxiliary sensor activation unit in an embodiment of the present application.

[0053] Figure 9 This is a specific implementation of an electronic device in the embodiment of this application. DETAILED DESCRIPTION

[0054] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. The specific embodiments of this application can be used for (including but not limited to) improving production efficiency, quality management, safety management, automation and intelligence levels at the construction site. This is even more important and urgent as the construction labor dividend gradually disappears.

[0055] Currently, in smart construction site projects, the "single sensor as the primary, multi-sensor as the supplementary" approach cannot meet the requirements for identifying and monitoring dynamic construction elements and activities. This is mainly because this approach fails to organically combine various types of data for efficient and low-cost analysis and feedback. It also fails to leverage the redundancy between the information contained in multi-dimensional data to effectively reduce the amount of data collected, transmitted, and stored. In particular, visual data (videos and photos) that covers all scenes and all time periods but contains a large amount of redundant information occupies a large amount of automatic monitoring resources. Moreover, this approach is limited to "post-event tracing" and does not use the data from each sensor in real time for dynamic monitoring and pre-event control and early warning.

[0056] Based on the above problems, this application provides an intelligent identification method for construction machinery and equipment activities based on multi-source data fusion, such as Figure 1 Shown, including:

[0057] S101: Decompose the acquired recognition task into several subtasks.

[0058] S102: Determine the required primary and auxiliary sensors based on all subtasks. The primary and auxiliary sensors may or may not be installed on the construction machinery. For example, for an excavator, the angles of its various moving joints can be obtained by angle sensors directly installed on the excavator joints, from images from an onboard surveillance camera, from a global camera installed on-site, or by reading command signals from the excavator's operating console.

[0059] S103: Execute the subtask using the data uploaded by the main sensor, and determine whether to enable the auxiliary sensor based on the execution requirements of the subtask.

[0060] S104: Provide real-time warning for abnormal situations that occur during the execution of subtasks.

[0061] Figure 1The executors of the method shown can be computers, PCs, terminals, etc. This application has achieved a breakthrough in the technical barrier that existing motion recognition technology cannot perform real-time activity recognition, and has realized real-time activity recognition of construction equipment, which is beneficial to improving managers' management, analysis and resource allocation of equipment, improving construction industry productivity, and reducing the hardware, software, and operation and maintenance costs of automatic identification of construction equipment activities.

[0062] In the smart construction site sector, dynamic elements primarily include personnel and construction machinery and equipment, while static elements primarily include materials and the construction site environment. In addition to the impact of static elements on the project, dynamic elements also involve higher-frequency interactions, poor safety, and reduced work efficiency. The industry generally categorizes the movement of dynamic elements on a construction site into the following levels, from low to high: movement of a joint (component), movement of an individual, interaction between individuals, and finally cluster activity.

[0063] In a specific embodiment, rotating the excavator's body is an action, digging dirt is an individual movement, and colliding dirt into a dump truck is an interaction. Sensors installed on various components of the excavator can detect the excavator's atomic actions.

[0064] Current sensor types are divided into: sensors based on kinematic methods, sensors based on computer vision methods, sensors based on audio methods, and other physical sensors.

[0065] Sensors based on kinematic methods utilize multiple sensors, such as accelerometers and gyroscopes, to identify the distinct kinematic patterns of movement performed by construction workers and equipment. These sensors can be microfabricated into electronic chips, such as inertial measurement units (IMUs), to collect data that, after processing, provides information about the equipment's rotational speed and orientation. Some IMUs are based on a technology called microelectromechanical systems (MEMS), which are a popular sensor type due to their small size and low price. MEMS sensors are very common in other industries and have diverse applications in fields such as healthcare, unsupervised home monitoring (home telecare), fall detection, weather monitoring, athlete movement pattern recognition, asset management, and industrial control. Furthermore, these devices have recently been introduced and used in construction for a variety of purposes, such as physiological monitoring, environmental sensing, proximity detection, location tracking, activity detection, and safety measurement and monitoring.

[0066] Audio-based sensors primarily rely on recording the sound patterns of equipment performing certain tasks. In recent years, these methods have been adopted as a suitable alternative to kinematic-based methods. The most commonly used tools in this approach are standard microphones, contact microphones, and microphone arrays. Microphones can be categorized based on their pickup pattern and transducer type. The collected audio data is then analyzed using signal processing algorithms to identify different types of live events.

[0067] Computer vision-based sensors capture visual data from construction sites using two-dimensional (2D) image / video cameras and 3D range cameras for further processing. These methods process images or videos captured from construction sites using different types of cameras, such as depth cameras (Red, Green, Blue, Depth (RGB-D)), which must be installed in appropriate locations and maintain a clear line of sight to record all ongoing activities during construction.

[0068] Other physical sensors, such as the Global Positioning System (GPS) and certain technologies like radio frequency identification (RFID) tags and ultra-wideband (UWB), are also important for activity detection. For example, knowing that an excavator is near a truck likely means that the excavator is loading the truck. However, these sensors have more applications in construction machinery and worker location tracking. On the other hand, accelerometers and gyroscopes are relatively well-suited for reactance detection. These types of sensors capture acceleration and rotation along the x, y, and z axes. The location of these sensors on the equipment or worker can affect the accuracy of activity recognition.

[0069] Table 1: Applicability of various technologies to different construction site conditions

[0070]

[0071] Table 2: Differences in recognition performance of different activity recognition methods for different levels of activities

[0072]

[0073] Table 3: Resource consumption and recognition effect of different activity recognition methods

[0074]

[0075] When an external recognition task is given, it is first broken down into several subtasks to be completed separately. Based on the conditions in the subtasks and referring to the applicable conditions and performance of each type of sensor in Tables 1-3 above, the main sensor suitable for completing the subtask is determined. Then the main sensor starts to collect the atomic motion data and time of the relevant components of the construction machinery and equipment. Based on the information uploaded by the main sensor and the completion progress of the subtask, it is determined whether the auxiliary sensor needs to be started to cooperate in completing the recognition task.

[0076] In one embodiment, the acquired recognition task is decomposed into several subtasks, such as Figure 2 Shown, including:

[0077] S201: Perform field parsing on the recognition task.

[0078] S202: Extracting the identification object field in the identification task, where the identification object is a construction machine.

[0079] S203: Decompose the recognition task into several subtasks according to the work type of the recognition object and the completion time corresponding to each work type.

[0080] Decompose the recognition task into activities, actions, and atomic actions. The decomposition should also take into account the type of mechanical device. Atomic actions refer to the positional changes of a single joint of a single device. Actions refer to short-duration behaviors or the behavior of a single device, such as an excavator digging downward or a loader lifting and moving forward. Activities refer to behaviors that last longer, often involving the interaction of multiple objects within a longer video.

[0081] In a specific embodiment, a recognition task is received, which is "automatically obtain the construction log of the excavator". The recognition task is field parsed, and the recognition object field "excavator" is extracted from it, and it is known that the monitored object is an excavator.

[0082] Excavator work types can be divided into idle, moving, earthwork (loading, leveling, and slope cutting), and crushing. Based on the excavator's work type, the identification task is broken down into several subtasks. For example, subtask 1 monitors the excavator's loading of soil and the time it takes to complete each loading operation.

[0083] In one embodiment, the required main sensors and auxiliary sensors are determined based on all subtasks, such as Figure 3 Shown, including:

[0084] S301: Determine the sensors required for each subtask.

[0085] S302: Select the sensor with the highest demand from the sensors as the main sensor.

[0086] S303: Use the remaining sensors as auxiliary sensors.

[0087] In a specific embodiment, still taking the above-mentioned "automatically obtaining the work log of the excavator" as an example, for the activity recognition of the excavator, kinematic sensors (such as the motion status of each joint of the excavator, or the operation instructions issued by the operator through the operating lever) are used as the main sensor to identify the action and movement of the excavator, and visual sensors are used as auxiliary sensors to achieve scene understanding (Scence understanding), and GPS is used as a position sensor.

[0088] In one embodiment, whether to enable the auxiliary sensor is determined based on the execution requirements of the subtask, such as Figure 4 Shown, including:

[0089] S401: Determine in real time whether the data uploaded by the main sensor meets the execution requirements of the subtask based on the subtask completion degree and evaluation confidence.

[0090] S402: When the conditions are not met, enable the auxiliary sensor to collect data and continue to execute the subtask according to the data collected by the auxiliary sensor.

[0091] To determine whether the data uploaded by the main sensor meets the execution requirements of the subtask, a threshold and vector method can be used for evaluation. The vector is composed of task completion, evaluation confidence, and other factors (A, B, C). A represents task completion, B represents the confidence of the task evaluation result, and C represents the factor that requires the activation of auxiliary sensors in certain specific situations. The threshold is set in advance based on work experience. When the vector (A, B, C) is within the threshold range in space, the auxiliary sensor does not need to be activated. When the vector (A, B, C) exceeds the threshold range in space, the auxiliary sensor needs to be activated.

[0092] In a specific embodiment, the action of "unloading the slag in the bucket" of the excavator can be known through the "relaxing the bucket" instruction of the operating lever, and for example, a complete "loading slag into the dump truck" can be known through a series of sequentially arranged instructions such as "tightening the bucket", "rotating", "relaxing the bucket" and "rotating". Therefore, the operating instructions of the excavator can be collected, and the operation video of the corresponding excavator can be collected at the same time. Then, according to the operation video, the excavator action is divided and encoded on the time axis to generate corresponding label data. Based on the instruction data and action label data, an excavator operation instruction-action classification data set in the time domain is generated, and then the pre-built model is used for machine learning model training and evaluation, and the excavator action automatic recognition model in the time domain is preferably obtained. As for higher-level interactive activities, such as what the excavator is digging, whether the excavator is transferring cargo to an open space or loading cargo into a car, kinematic sensors alone cannot complete such tasks. In this case, the above-mentioned vector evaluation method determines the need to activate auxiliary sensors. Therefore, visual sensors that can provide rich information are needed to complete this task. The image and / or video data obtained by the visual sensor can be analyzed using corresponding scene understanding (Scene Understanding), image semantic segmentation (Semantic Segmentation), object detection (Object Detection), cross-camera tracking (Re-Identification), and visual real-time 3D reconstruction (such as monocular 3D reconstruction and binocular 3D reconstruction) models to obtain richer information to determine the remaining elements of the "automatically obtain excavator work log" task. Specifically, if the scene understanding model finds that the excavator's "bucket" is filled with "soil", and that the "soil" disappears from the "bucket" after the excavator bucket approaches the "truck", it can be determined that during the interactive activity, the excavator is loading the truck with soil. Of course, it should be noted that this consumes a large amount of computer resources (computing, storage, and transmission) and cannot guarantee full coverage in space and time. Visual sensors do not need to be turned on all the time; they only need to conduct regular or irregular "spot checks" to confirm this information. The full monitoring of the activity is still mainly undertaken by cost-effective operating instructions and analysis models with a wide range of spatiotemporal monitoring. As for "automatically obtaining the excavator's work log" to obtain the excavator's position information during the task, it is obviously very difficult to obtain it through visual sensors (such as positioning through 3D reconstruction) and kinematic sensors. However, GPS sensors can obtain accurate information at a very low cost.

[0093] In addition to achieving the aforementioned technical effects, this specific embodiment also introduces an independent third-party oversight and evaluation mechanism for the workload and efficiency of construction site equipment. With the trend toward groupization, globalization, and the leasing of machinery and equipment among construction companies, a fair and effective third-party evaluation mechanism for the workload and efficiency of machinery and equipment is of great significance to the internationalization of Chinese construction companies and the standardized management of the construction market by government regulators. This embodiment provides a foundational algorithm for an efficient, fair, and cost-effective evaluation mechanism.

[0094] Based on the same inventive concept, the embodiments of the present application also provide a construction machinery and equipment activity identification device based on multi-source data fusion, which can be used to implement the method described in the above embodiments, as described in the following embodiments. Since the principle of solving the problem by the construction machinery and equipment activity identification device based on multi-source data fusion is similar to that of the construction machinery and equipment activity identification method based on multi-source data fusion, the implementation of the construction machinery and equipment activity identification device based on multi-source data fusion can refer to the implementation of the construction machinery and equipment activity identification method based on multi-source data fusion, and the repeated parts will not be repeated. As used below, the term "unit" or "module" can be a combination of software and / or hardware that implements a predetermined function. Although the system described in the following embodiments is preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and conceived.

[0095] like Figure 5 As shown, the present application provides a construction machinery and equipment activity recognition device based on multi-source data fusion, comprising:

[0096] A task decomposition unit 501 is used to decompose the acquired recognition task into several subtasks;

[0097] A sensor selection unit 502 is used to determine the required main sensors and auxiliary sensors according to all subtasks, and the main sensors and auxiliary sensors are set on the construction machinery and equipment;

[0098] The auxiliary sensor enabling unit 503 is configured to execute a subtask based on the data uploaded by the main sensor and determine whether to enable the auxiliary sensor according to the execution requirements of the subtask;

[0099] The real-time warning unit 504 is used to issue a real-time warning for abnormal situations that occur during the execution of a subtask.

[0100] In one embodiment, if Figure 6 As shown, the task decomposition unit 501 includes:

[0101] Field parsing module 601, used to perform field parsing on the recognition task;

[0102] The monitoring object extraction module 602 is used to extract the identification object field in the identification task, and the identification object is a construction machinery and equipment;

[0103] The subtask decomposition module 603 is used to decompose the recognition task into a plurality of subtasks according to the work type of the recognition object and the completion time corresponding to each work type.

[0104] In one embodiment, if Figure 7 As shown, the sensor selection unit 502 includes:

[0105] A sensor determination module 701 is used to determine the sensors required for each subtask;

[0106] A primary sensor determination module 702 is configured to select a sensor with the highest demand from among the sensors as a primary sensor;

[0107] The auxiliary sensor determination module 703 is configured to use the remaining sensors as auxiliary sensors.

[0108] In one embodiment, if Figure 8 As shown, the auxiliary sensor enabling unit 503 includes:

[0109] Evaluation module 801, configured to determine in real time whether the data uploaded by the main sensor meets the execution requirements of the subtask based on the subtask completion degree and evaluation confidence;

[0110] The auxiliary sensor enabling module 802 is configured to enable the auxiliary sensor to collect data and continue to execute the subtask according to the data collected by the auxiliary sensor when the condition is not satisfied.

[0111] In a specific embodiment, a construction machinery and equipment activity recognition device based on multi-source data fusion may include the following hardware modules:

[0112] Data perception module: Various types of data intelligent collection modules located in the terminal; receive collection instructions from the central processing module and selectively collect data; and pre-process the data through the front-end intelligent chip to reduce data transmission and storage pressure.

[0113] Data transmission module: completes data transmission between functional modules; through wired or wireless forms. Data storage module: completes data storage.

[0114] Central processing module: receiving tasks, task planning and allocation, resource allocation, analysis of various types of data, data collection planning, data abandonment (discard or store) decisions; receiving tasks from external or internal feedback; planning and allocation of tasks, resource allocation according to task requirements and internal system configuration; analyzing various types of data according to the requirements of each task and providing quantitative indicators; planning and correcting data collection in accordance with task requirements; processing and storing the original data in accordance with the analyzed data according to task requirements and data traceability requirements, to facilitate future traceability and evidence collection. Intermediate data that is only in the analysis process is discarded to reduce the storage pressure of the memory.

[0115] Data feedback module: Feedback the analyzed data to each demand side; the demand side includes: field applications, supervisory agencies, analysis and decision-making departments, various project levels and central processing modules.

[0116] Data display module: displays data. Terminal application module: terminal application equipment, receives data from the feedback module, and realizes functions such as construction assistance, safety early warning and alarm, real-time quality supervision and correction.

[0117] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0118] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0119] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0120] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0121] Specific embodiments are used in the present invention to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.

[0122] The embodiments of the present application also provide a specific implementation of an electronic device that can implement all the steps in the method in the above embodiments, see Figure 9 , the electronic device specifically includes the following contents:

[0123] Processor 901, memory 902, communication interface 903, bus 904 and non-volatile memory 905;

[0124] The processor 901, memory 902, communication interface 903 and non-volatile memory 905 communicate with each other via the bus 904;

[0125] The processor 901 is configured to call the computer program in the memory 902 and the non-volatile memory 905. When the processor executes the computer program, all steps of the method in the above embodiment are implemented. For example, when the processor executes the computer program, the following steps are implemented:

[0126] S101: Decompose the acquired recognition task into several subtasks.

[0127] S102: Determine the required main sensors and auxiliary sensors based on all subtasks, and set the main sensors and auxiliary sensors on the construction machinery and equipment.

[0128] S103: Execute the subtask using the data uploaded by the main sensor, and determine whether to enable the auxiliary sensor based on the execution requirements of the subtask.

[0129] S104: Provide real-time warning for abnormal situations that occur during the execution of subtasks.

[0130] The embodiments of the present application also provide a computer-readable storage medium capable of implementing all the steps of the method in the above embodiments. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, all the steps of the method in the above embodiments are implemented. For example, when the processor executes the computer program, the following steps are implemented:

[0131] S101: Decompose the acquired recognition task into several subtasks.

[0132] S102: Determine the required main sensors and auxiliary sensors based on all subtasks, and set the main sensors and auxiliary sensors on the construction machinery and equipment.

[0133] S103: Execute the subtask using the data uploaded by the main sensor, and determine whether to enable the auxiliary sensor based on the execution requirements of the subtask.

[0134] S104: Provide real-time warning for abnormal situations that occur during the execution of subtasks.

[0135] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences from other embodiments. In particular, for hardware + program embodiments, since they are basically similar to method embodiments, the description is relatively simple. For relevant parts, refer to the partial description of the method embodiment. Although the embodiments of this specification provide method operation steps as described in the embodiments or flow charts, more or fewer operation steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is only one way of executing the steps among many, and does not represent the only execution order. When an actual device or terminal product is executed, the method can be executed sequentially or in parallel according to the embodiments or the figures (for example, in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, product or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, product or device. Without further limitation, it is not excluded that there are other identical or equivalent elements in the process, method, product or device including the elements. For ease of description, the above devices are described as being functionally divided into various modules and described separately. Of course, when implementing the embodiments of this specification, the functions of each module can be implemented in the same or multiple software and / or hardware, or the modules that implement the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components that can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, and can be electrical, mechanical or other forms. The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or box in the flowcharts and / or block diagrams, as well as the combination of processes and / or boxes in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce the instructions for implementing the process Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0136] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiments of this specification.

[0137] In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples, unless they contradict each other. The above is only an embodiment of the embodiment of this specification and is not intended to limit the embodiment of this specification. For those skilled in the art, the embodiment of this specification may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiment of this specification shall be included within the scope of the claims of the embodiment of this specification.

Claims

1. A method for intelligently identifying activities of construction machinery and equipment, characterized in that: include: Decompose the acquired recognition task into several subtasks; Determine the required primary and auxiliary sensors based on all of the subtasks; Executing the subtask using the data uploaded by the main sensor, and determining whether to enable the auxiliary sensor according to the execution requirements of the subtask; Identifying activities during the execution of the subtask to obtain activity information; The determining whether to enable the auxiliary sensor according to the execution requirement of the subtask includes: Based on the subtask completion and assessment confidence, a threshold and vector method are used to evaluate whether the data uploaded by the main sensor meets the execution requirements of the subtask. The vector is composed of the task completion, assessment confidence, and other factors. The other factors include: different construction site conditions, different levels of activities, identified resource consumption, and recognition effect. The threshold is set in advance based on work experience. When the vector is within the threshold range in space, the auxiliary sensor does not need to be activated. When the vector exceeds a threshold range in the space, enabling the auxiliary sensor to collect data and continuing to execute the subtask according to the data collected by the auxiliary sensor; The obtained recognition task is decomposed into several subtasks, including: Performing field parsing on the recognition task; Extracting an identification object field in the identification task, wherein the identification object is a construction machine; The recognition task is decomposed into several subtasks according to the work type of the recognition object and the completion time corresponding to each work type.

2. The intelligent identification method for construction machinery and equipment activities according to claim 1 is characterized in that: The determining of the required main sensors and auxiliary sensors according to all the subtasks includes: determining the sensors required for each of said subtasks; Selecting the sensor with the highest demand from the sensors as the main sensor; The remaining sensors are used as the auxiliary sensors.

3. The intelligent identification method for construction machinery and equipment activities according to claim 1 is characterized in that: Also includes: The activities of the construction machinery and equipment are analyzed based on the activity information, including efficiency analysis, progress analysis, quality analysis, safety monitoring, energy efficiency analysis, and automatic generation of construction logs.

4. The intelligent identification method for construction machinery and equipment activities according to claim 1 is characterized in that: Also includes: Abnormal conditions of construction machinery and equipment are monitored in real time based on the activity information and early warnings are issued.

5. The intelligent identification method for construction machinery and equipment activities according to any one of claims 1 to 4, characterized in that: When the construction machinery and equipment is an excavator, it includes: Decompose the acquired excavator activity recognition task into several subtasks; Determining the required main sensors and auxiliary sensors based on all the subtasks, the sensors including: sensors based on kinematic methods, sensors based on computer vision methods, sensors based on audio methods, and physical sensors on the excavator; Executing the subtask using the data uploaded by the main sensor, and determining whether to enable the auxiliary sensor according to the execution requirements of the subtask; The activity information of the excavator occurring during the execution of the subtask is collected and analyzed in real time.

6. An intelligent identification device for construction machinery and equipment activities based on multi-source data fusion, characterized in that: include: A task decomposition unit, used to decompose the acquired recognition task into several subtasks; a sensor selection unit, configured to determine required main sensors and auxiliary sensors according to all of the subtasks; an auxiliary sensor enabling unit, configured to execute the subtask using the data uploaded by the main sensor and determine whether to enable the auxiliary sensor according to the execution requirements of the subtask; A real-time early warning unit for identifying activities during the execution of the subtask; The auxiliary sensor enabling unit includes: An evaluation module, configured to evaluate whether the data uploaded by the main sensor meets the execution requirements of the subtask based on the subtask completion and evaluation confidence, using a threshold and vector method. The vector is composed of the task completion, evaluation confidence, and other factors, including different construction site conditions, different levels of activities, identified resource consumption, and recognition effect. The threshold is set in advance based on work experience. When the vector is within the threshold range in space, the auxiliary sensor does not need to be activated. an auxiliary sensor module enabled, configured to enable the auxiliary sensor to collect data and continue to execute the subtask according to the data collected by the auxiliary sensor when the vector exceeds a threshold range in the space; The task decomposition unit includes: A field parsing module, used to perform field parsing on the recognition task; An identification object extraction module is used to extract the identification object field in the identification task, where the identification object is a construction machine; The subtask decomposition module is used to decompose the recognition task into a plurality of subtasks according to the work type of the recognition object and the completion time corresponding to each work type.

7. The intelligent identification device for construction machinery and equipment activities according to claim 6, characterized in that: The sensor selection unit includes: A sensor determination module, configured to determine the sensor required for each of the subtasks; a main sensor determination module, configured to select the sensor with the highest demand from the sensors as the main sensor; The auxiliary sensor determination module is configured to use the remaining sensors as the auxiliary sensors.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the method for intelligently identifying activities of construction machinery and equipment described in any one of claims 1 to 5 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for intelligently identifying the activities of construction machinery and equipment described in any one of claims 1 to 5 are implemented.

Images