Method and device for quantifying working efficiency of construction machine

Abstract

The invention discloses a method and a device for quantifying the working efficiency of engineering machinery. The method comprises the following steps: receiving sensing data of various sensors arranged on the engineering machinery to a working environment; according to the perception data and a pre-trained action recognition model, action features of the working state of the engineering machinery are extracted, and the pre-trained action recognition model is a neural network which is based on deep learning and used for generating the action features of the working state of the engineering machinery; according to the action characteristics, the operation indexes of the engineering machinery are quantified, and the operation indexes are used for representing the operation efficiency of the engineering machinery. Therefore, by adopting the embodiment of the invention, the continuous real-time monitoring of the working process of the engineering machinery can be realized, and meanwhile, the evaluation accuracy of the working efficiency of the engineering machinery is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and device for quantifying the operating efficiency of engineering machinery. Background Art

[0002] In the field of construction machinery, construction machinery (such as excavators) is a key operating equipment and is widely used in industries such as construction, mining and infrastructure construction. The operating efficiency of construction machinery is directly related to the progress and cost-effectiveness of engineering projects.

[0003] In the related technologies, the traditional method of evaluating the operating efficiency of construction machinery mainly relies on manual observation and empirical judgment. Since it relies on the personal experience and observation of the operator, the evaluation results are easily affected by subjective factors, resulting in limited consistency and accuracy of the evaluation. At the same time, manual observation cannot achieve continuous real-time monitoring of the operation process of construction machinery, which limits the immediate evaluation of operating efficiency and problem response. Summary of the invention

[0004] The embodiments of the present application provide a method and device for quantifying the operating efficiency of construction machinery. In order to have a basic understanding of some aspects of the disclosed embodiments, a simple summary is given below. This summary is not a general review, nor is it intended to identify key / important components or describe the scope of protection of these embodiments. Its only purpose is to present some concepts in a simple form as a preface to the detailed description that follows.

[0005] In a first aspect, an embodiment of the present application provides a method for quantifying the operating efficiency of a construction machinery, the method comprising:

[0006] Receive data about the working environment from various sensors installed on the construction machinery;

[0007] Extracting motion features of the working state of the construction machinery based on the perception data and a pre-trained motion recognition model, wherein the pre-trained motion recognition model is a neural network based on deep learning for generating motion features of the working state of the construction machinery;

[0008] According to the action characteristics, the operation indicators of the construction machinery are quantified, and the operation indicators are used to characterize the operation efficiency of the construction machinery.

[0009] Optionally, based on the perception data and the pre-trained motion recognition model, motion features of the working state of the construction machinery are extracted, including:

[0010] Input the perception data into a pre-trained motion recognition model to extract the motion features of the working state of the construction machinery;

[0011] Output action features corresponding to the perception data;

[0012] The motion features corresponding to the perception data are used as the motion features of the construction machinery operation status.

[0013] Optionally, generate a pre-trained action recognition model by following these steps, including:

[0014] Collect historical perception data of the working environment from various sensors installed on the construction machinery, including image frames, inertial measurement unit data and mechanical status data;

[0015] Label the motion features of the working state of the construction machinery on the image frames, IMU data and machinery state data to obtain model training samples;

[0016] Inject a preset loss function into the neural network to obtain an action recognition model;

[0017] The action recognition model is deeply learned based on the model training samples to generate a pre-trained action recognition model.

[0018] Optionally, the preset loss function is a cross entropy loss function, which is:

[0019]

[0020] Among them, L is the cross entropy loss value, N is the number of samples of model training samples, C is the number of categories of action features, and y i,c is the true category label of the action feature, p i,c is the predicted category label of the action feature.

[0021] Optionally, the action features include the action type and the action execution time;

[0022] According to the action characteristics, the operation indicators of construction machinery are quantified, including:

[0023] When the action type meets the preset action end condition and the loading completion instruction fed back by the engineering machinery is received, the action execution time is taken as the action end time; wherein the preset action end condition is that the engineering machinery loads the object to be loaded into the transport vehicle;

[0024] Get the action start time corresponding to the action type;

[0025] Calculate and cache the time difference between the action end time and the action start time;

[0026] The operation cycle and the average value of the operation cycle are calculated according to the cached time series to obtain the operation index of the construction machinery.

[0027] Optionally, calculate the job cycle and the average of the job cycle based on the cached time series, including:

[0028] The cached time series are summed to obtain the total time required for the construction machinery to complete a whole loading process;

[0029] The calculation formula for the average value of the operation cycle is:

[0030]

[0031] in, is the average value of the operation cycle, t start ,t end are the end time and the start time of the action respectively, N cycles is the number of cycles, τ j is the j-th time difference in the cached time series.

[0032] Optionally, the method further includes:

[0033] When the action type meets the preset action end condition and no loading completion instruction is received from the engineering machinery, the step of receiving the perception data of the working environment from various sensors installed on the engineering machinery is continued.

[0034] Optionally, the method further includes:

[0035] When the action type meets the preset action start condition, the action execution time is used as the action start time corresponding to the action type; wherein the preset action start condition is that the engineering machinery obtains the object to be loaded.

[0036] Optionally, the action characteristics include the actual load under the operation action;

[0037] According to the action characteristics, the operation indicators of construction machinery are quantified, including:

[0038] Calculate the ratio of the actual load under the operation action to the actual load of the preset maximum load;

[0039] Convert the ratio into percentage to obtain the operation index of the construction machinery.

[0040] In a second aspect, an embodiment of the present application provides a device for quantifying the operating efficiency of an engineering machinery, the device comprising:

[0041] A receiving module is used to receive the perception data of the working environment from various sensors installed on the engineering machinery;

[0042] An extraction module, used to extract motion features of the working state of the construction machinery according to the perception data and a pre-trained motion recognition model, where the pre-trained motion recognition model is a neural network based on deep learning for generating motion features of the working state of the construction machinery;

[0043] The quantification module is used to quantify the operation index of the construction machinery according to the action characteristics, and the operation index is used to characterize the operation efficiency of the construction machinery.

[0044] The technical solution provided by the embodiments of the present application may have the following beneficial effects:

[0045] In an embodiment of the present application, on the one hand, by receiving and processing perception data of the working environment from various sensors installed on the engineering machinery, the perception data is received in real time, and continuous real-time monitoring of the working process of the engineering machinery can be achieved; on the other hand, based on the perception data and a pre-trained motion recognition model, the motion characteristics of the working state of the engineering machinery are extracted. The model is a neural network based on deep learning for generating motion characteristics of the working state of the engineering machinery. The neural network can accurately identify the motion characteristics of the working state of the engineering machinery, thereby improving the accuracy of the evaluation of the working efficiency of the engineering machinery.

[0046] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0048] Figure 1 It is a flow chart of a method for quantifying the operating efficiency of engineering machinery provided in an embodiment of the present application;

[0049] Figure 2 It is a flowchart of a model training method provided by this application;

[0050] Figure 3 It is a structural schematic diagram of a device for quantifying the operating efficiency of an engineering machinery provided by the present application;

[0051] Figure 4 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The following description and the accompanying drawings sufficiently illustrate specific embodiments of the present application to enable those skilled in the art to practice them.

[0053] It should be clear that the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0054] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims.

[0055] In the description of the present application, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances. In addition, in the description of the present application, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0056] The present application provides a method and device for quantifying the operating efficiency of engineering machinery to solve the problems existing in the above-mentioned related technical problems. In the embodiment of the present application, on the one hand, by receiving and processing the perception data of the operating environment from various sensors installed on the engineering machinery, the perception data is received in real time, and continuous real-time monitoring of the operation process of the engineering machinery can be realized; on the other hand, according to the perception data and the pre-trained action recognition model, the action features of the working state of the engineering machinery are extracted. The model is a neural network based on deep learning for generating the action features of the working state of the engineering machinery. The neural network can accurately identify the action features of the working state of the engineering machinery, thereby improving the accuracy of the evaluation of the working efficiency of the engineering machinery. The following is a detailed description using an exemplary embodiment.

[0057] The following will be combined with the attached Figure 1 -Attached Figure 2 , a method for quantifying the operating efficiency of engineering machinery provided in an embodiment of the present application is introduced in detail. The method can be implemented by a computer program and can be run on an operating efficiency quantification device for engineering machinery based on the von Neumann system. The computer program can be integrated into an application or run as an independent tool application.

[0058] See also Figure 1 , which is a flow chart of a method for quantifying the operating efficiency of engineering machinery provided in the embodiment of the present application. Figure 1 As shown, the method of the embodiment of the present application may include the following steps:

[0059] S101, receiving the perception data of the working environment from various sensors installed on the engineering machinery;

[0060] Among them, engineering machinery is a mechanical vehicle currently performing an operation task, such as an excavator. Various types of sensors installed on engineering machinery are deployed on the target operating machinery by experts based on practical experience, and can be deployed in multiple different positions of the engineering machinery. Various types of sensors include high-definition cameras, inertial measurement units (IMUs), and mechanical state sensors. High-definition cameras are used to capture high-definition video images of the excavator loading operation process. These images can provide visual information of the excavator operation, including excavation actions, loading actions, etc. The inertial measurement unit provides measurement of the acceleration, angular velocity, and sometimes magnetic field direction of the excavator in space. The mechanical state sensor is used to monitor the mechanical state of the excavator, including but not limited to key parameters such as oil pressure, temperature, and engine speed.

[0061] Among them, the camera resolution can be set to 1920*1080, and the frame rate is 30fps, which can clearly capture the loading action of the construction machinery.

[0062] In some embodiments, according to the type of engineering machinery and the working requirements, appropriate sensor types are selected, such as high-definition cameras, IMUs, mechanical status sensors, etc.; sensors are arranged at key locations of the engineering machinery to ensure that the working environment and mechanical status information can be fully captured; data acquisition modules are installed on the engineering machinery, and these modules can interface with various sensors to receive sensor data in real time; a communication network is established, and the data acquisition modules transmit the sensor data to the server, including wired connections or wireless communication technologies, such as Wi-Fi, Bluetooth, 4G / 5G, etc.; the server receives the perception data of the working environment from various sensors installed on the engineering machinery.

[0063] S102, extracting motion features of the working state of the construction machinery according to the perception data and a pre-trained motion recognition model, where the pre-trained motion recognition model is a neural network based on deep learning for generating motion features of the working state of the construction machinery;

[0064] The neural network may be a convolutional neural network.

[0065] In some embodiments, the process of extracting motion features of the working state of engineering machinery based on perception data and a pre-trained motion recognition model specifically includes: inputting the perception data into a pre-trained motion recognition model to extract the motion features of the working state of engineering machinery; outputting the motion features corresponding to the perception data; and using the motion features corresponding to the perception data as the motion features of the working state of engineering machinery.

[0066] In an embodiment of the present application, a pre-trained action recognition model is generated according to the following steps: historical perception data of the working environment from various sensors installed on the engineering machinery are collected, and the historical perception data include image frames, inertial measurement unit data and mechanical state data; the image frames, IMU data and mechanical state data are annotated with action features of the working state of the engineering machinery to obtain model training samples; a preset loss function is injected into the neural network to obtain the action recognition model; the action recognition model is deeply learned according to the model training samples to generate a pre-trained action recognition model.

[0067] Specifically, the specific process of performing deep learning on the action recognition model according to the model training samples to generate a pre-trained action recognition model includes: inputting the model training samples into the action recognition model, outputting the cross entropy loss value of the model, and generating the pre-trained action recognition model when the cross entropy loss value reaches the minimum and the number of training times reaches a preset threshold, otherwise continuing to execute the step of inputting the model training samples into the action recognition model until the model loss value reaches the minimum.

[0068] Specifically, the model training process is carried out on a computer equipped with an NVIDIA GeForce RTX4090 graphics card. The number of training rounds is 300, and the batch size of the model training samples for each training is 64.

[0069] Furthermore, the action recognition accuracy can be determined during the model training process, and the accuracy calculation formula is:

[0070]

[0071] Among them, Accuracy is the accuracy rate, TP represents the number of samples that are actually positive and correctly predicted as positive, TN is the number of samples that are actually negative and correctly predicted as negative, FP is a false positive, and TN is a missed negative.

[0072] Specifically, the preset loss function is the cross entropy loss function, and the cross entropy loss function is:

[0073]

[0074] Among them, L is the cross entropy loss value, N is the number of samples of model training samples, C is the number of categories of action features, and y i,c is the true category label of the action feature, p i,c is the predicted category label of the action feature.

[0075] S103, quantifying the operation index of the engineering machinery according to the action characteristics, where the operation index is used to characterize the operation efficiency of the engineering machinery.

[0076] Among them, action features include action type and action execution time.

[0077] In some embodiments, based on action characteristics, the specific process of quantifying the operation index of engineering machinery includes: when the action type meets the preset action end condition and receives the loading completion instruction fed back by the engineering machinery, the action execution time is used as the action end time; wherein the preset action end condition is that the engineering machinery loads the object to be loaded into the transport vehicle; the action start time corresponding to the action type is obtained; the time difference between the action end time and the action start time is calculated and cached; the operation cycle and the average value of the operation cycle are calculated according to the cached time series to obtain the operation index of the engineering machinery.

[0078] Specifically, the specific process of calculating the operation cycle and the average value of the operation cycle according to the cached time series includes: summing the cached time series to obtain the total time required for the engineering machinery to complete a whole loading process;

[0079] The calculation formula for the average value of the operation cycle is:

[0080]

[0081] in, is the average value of the operation cycle, t start ,t end are the end time and the start time of the action respectively, N cycles is the number of cycles, τ j is the j-th time difference in the cached time series.

[0082] In some embodiments, when the action type meets the preset action end condition and no loading completion instruction is received from the engineering machinery, the step of receiving the perception data of the working environment from various sensors installed on the engineering machinery is continued.

[0083] In some embodiments, when the action type meets the preset action start condition, the action execution time is used as the action start time corresponding to the action type; wherein the preset action start condition is that the engineering machinery obtains the object to be loaded.

[0084] Among them, the action characteristics include the actual load under the operation action.

[0085] In some embodiments, based on the action characteristics, the specific process of quantifying the operation index of the engineering machinery includes: calculating the ratio of the actual load under the operation action to the actual load of the preset maximum load; converting the ratio into a percentage to obtain the operation index of the engineering machinery.

[0086] Furthermore, through the visual interface, the operating indicators of construction machinery can be displayed to operators and managers in real time, making it easy to realize real-time display and interaction of data.

[0087] In an embodiment of the present application, on the one hand, by receiving and processing perception data of the working environment from various sensors installed on the engineering machinery, the perception data is received in real time, and continuous real-time monitoring of the working process of the engineering machinery can be achieved; on the other hand, based on the perception data and a pre-trained motion recognition model, the motion characteristics of the working state of the engineering machinery are extracted. The model is a neural network based on deep learning for generating motion characteristics of the working state of the engineering machinery. The neural network can accurately identify the motion characteristics of the working state of the engineering machinery, thereby improving the accuracy of the evaluation of the working efficiency of the engineering machinery.

[0088] See also Figure 2 , which is a flow chart of a model training method provided in the embodiment of the present application. Figure 2 As shown, the method of the embodiment of the present application may include the following steps:

[0089] S201, collecting historical perception data of the working environment from various sensors installed on the engineering machinery, the historical perception data including image frames, inertial measurement unit data and machine status data;

[0090] S202, annotating the motion characteristics of the working state of the engineering machinery on the image frames, IMU data and machinery state data to obtain a model training sample;

[0091] S203, injecting a preset loss function into the neural network to obtain an action recognition model;

[0092] S204: Perform deep learning on the action recognition model according to the model training samples to generate a pre-trained action recognition model.

[0093] In an embodiment of the present application, on the one hand, by receiving and processing perception data of the working environment from various sensors installed on the engineering machinery, the perception data is received in real time, and continuous real-time monitoring of the working process of the engineering machinery can be achieved; on the other hand, based on the perception data and a pre-trained motion recognition model, the motion characteristics of the working state of the engineering machinery are extracted. The model is a neural network based on deep learning for generating motion characteristics of the working state of the engineering machinery. The neural network can accurately identify the motion characteristics of the working state of the engineering machinery, thereby improving the accuracy of the evaluation of the working efficiency of the engineering machinery.

[0094] The following are device embodiments of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0095] See also Figure 3, which shows a schematic diagram of the structure of a device for quantifying the operating efficiency of engineering machinery provided by an exemplary embodiment of the present application. The device for quantifying the operating efficiency of engineering machinery can be implemented as all or part of an electronic device through software, hardware, or a combination of both. The device 1 includes a receiving module 10, an extraction module 20, and a quantification module 30.

[0096] The receiving module 10 is used to receive the sensing data of the working environment from various sensors installed on the engineering machinery;

[0097] An extraction module 20, for extracting motion features of the working state of the engineering machinery according to the perception data and a pre-trained motion recognition model, wherein the pre-trained motion recognition model is a neural network based on deep learning for generating motion features of the working state of the engineering machinery;

[0098] The quantification module 30 is used to quantify the operation index of the engineering machinery according to the action characteristics, and the operation index is used to characterize the operation efficiency of the engineering machinery.

[0099] It should be noted that the device for quantifying the operating efficiency of engineering machinery provided in the above embodiment only uses the division of the above functional modules as an example when executing the method for quantifying the operating efficiency of engineering machinery. In actual applications, the above functional distribution can be completed by different functional modules as needed, that is, the internal structure of the equipment is divided into different functional modules to complete all or part of the functions described above. In addition, the device for quantifying the operating efficiency of engineering machinery provided in the above embodiment and the method for quantifying the operating efficiency of engineering machinery belong to the same concept, and the implementation process thereof is detailed in the method embodiment, which will not be repeated here.

[0100] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0101] In an embodiment of the present application, on the one hand, by receiving and processing perception data of the working environment from various sensors installed on the engineering machinery, the perception data is received in real time, and continuous real-time monitoring of the working process of the engineering machinery can be achieved; on the other hand, based on the perception data and a pre-trained motion recognition model, the motion characteristics of the working state of the engineering machinery are extracted. The model is a neural network based on deep learning for generating motion characteristics of the working state of the engineering machinery. The neural network can accurately identify the motion characteristics of the working state of the engineering machinery, thereby improving the accuracy of the evaluation of the working efficiency of the engineering machinery.

[0102] The present application also provides a computer-readable medium having program instructions stored thereon, which, when executed by a processor, implement the method for quantifying the operating efficiency of engineering machinery provided by the above-mentioned various method embodiments.

[0103] The present application also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method for quantifying the operating efficiency of engineering machinery in each of the above-mentioned method embodiments.

[0104] See also Figure 4 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 As shown, the electronic device 1000 may include: at least one processor 1001 , at least one network interface 1004 , a user interface 1003 , a memory 1005 , and at least one communication bus 1002 .

[0105] The communication bus 1002 is used to realize the connection and communication between these components.

[0106] The user interface 1003 may include a display screen (Display) and a camera (Camera), and the optional user interface 1003 may also include a standard wired interface and a wireless interface.

[0107] The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0108] Among them, the processor 1001 may include one or more processing cores. The processor 1001 uses various interfaces and lines to connect various parts within the entire electronic device 1000, and executes various functions and processes data of the electronic device 1000 by running or executing instructions, programs, code sets or instruction sets stored in the memory 1005, and calling data stored in the memory 1005. Optionally, the processor 1001 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 1001 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes operating devices, user interfaces and applications, etc.; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 1001, and it can be implemented separately through a chip.

[0109] Among them, the memory 1005 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 1005 includes a non-transitory computer-readable storage medium. The memory 1005 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 1005 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing the operating device, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 1005 may also be optionally at least one storage device located away from the aforementioned processor 1001. As Figure 4 As shown, the memory 1005 as a computer storage medium may include an operating device, a network communication module, a user interface module, and an application program for quantifying the operation efficiency of engineering machinery.

[0110] exist Figure 4 In the electronic device 1000 shown, the user interface 1003 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 1001 can be used to call the engineering machinery operation efficiency quantification application stored in the memory 1005, and specifically perform the following operations:

[0111] Receive data about the working environment from various sensors installed on the construction machinery;

[0112] Extracting motion features of the working state of the construction machinery based on the perception data and a pre-trained motion recognition model, wherein the pre-trained motion recognition model is a neural network based on deep learning for generating motion features of the working state of the construction machinery;

[0113] According to the action characteristics, the operation indicators of the construction machinery are quantified, and the operation indicators are used to characterize the operation efficiency of the construction machinery.

[0114] In one embodiment, when the processor 1001 extracts the motion features of the working state of the engineering machinery according to the perception data and the pre-trained motion recognition model, the processor 1001 specifically performs the following operations:

[0115] Input the perception data into a pre-trained motion recognition model to extract the motion features of the working state of the construction machinery;

[0116] Output action features corresponding to the perception data;

[0117] The motion features corresponding to the perception data are used as the motion features of the construction machinery operation status.

[0118] In one embodiment, when the processor 1001 generates a pre-trained action recognition model, the processor 1001 specifically performs the following operations:

[0119] Collect historical perception data of the working environment from various sensors installed on the construction machinery, including image frames, inertial measurement unit data and mechanical status data;

[0120] Label the motion features of the working state of the construction machinery on the image frames, IMU data and machinery state data to obtain model training samples;

[0121] Inject a preset loss function into the neural network to obtain an action recognition model;

[0122] The action recognition model is deeply learned based on the model training samples to generate a pre-trained action recognition model.

[0123] In one embodiment, when the processor 1001 quantifies the operation index of the engineering machinery according to the action characteristics, the processor 1001 specifically performs the following operations:

[0124] When the action type meets the preset action end condition and the loading completion instruction fed back by the engineering machinery is received, the action execution time is taken as the action end time; wherein the preset action end condition is that the engineering machinery loads the object to be loaded into the transport vehicle;

[0125] Get the action start time corresponding to the action type;

[0126] Calculate and cache the time difference between the action end time and the action start time;

[0127] The operation cycle and the average value of the operation cycle are calculated according to the cached time series to obtain the operation index of the construction machinery.

[0128] In one embodiment, the processor 1001 further performs the following operations:

[0129] When the action type meets the preset action end condition and no loading completion instruction is received from the engineering machinery, the step of receiving the perception data of the working environment from various sensors installed on the engineering machinery is continued.

[0130] In one embodiment, the processor 1001 further performs the following operations:

[0131] When the action type meets the preset action start condition, the action execution time is used as the action start time corresponding to the action type; wherein the preset action start condition is that the engineering machinery obtains the object to be loaded.

[0132] In one embodiment, when the processor 1001 generates the operation index of the engineering machinery according to the action characteristics, the processor 1001 specifically performs the following operations:

[0133] Calculate the ratio of the actual load under the operation action to the actual load of the preset maximum load;

[0134] Convert the ratio into percentage to obtain the operation index of the construction machinery.

[0135] In an embodiment of the present application, on the one hand, by receiving and processing perception data of the working environment from various sensors installed on the engineering machinery, the perception data is received in real time, and continuous real-time monitoring of the working process of the engineering machinery can be achieved; on the other hand, based on the perception data and a pre-trained motion recognition model, the motion characteristics of the working state of the engineering machinery are extracted. The model is a neural network based on deep learning for generating motion characteristics of the working state of the engineering machinery. The neural network can accurately identify the motion characteristics of the working state of the engineering machinery, thereby improving the accuracy of the evaluation of the working efficiency of the engineering machinery.

[0136] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program for quantifying the operation efficiency of engineering machinery can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium of the program for quantifying the operation efficiency of engineering machinery can be a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0137] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A method for quantifying the operating efficiency of construction machinery, characterized in that: The method comprises: Receive data about the working environment from various sensors installed on the construction machinery; Extracting motion features of the working state of the engineering machinery according to the perception data and a pre-trained motion recognition model, wherein the pre-trained motion recognition model is a neural network based on deep learning for generating motion features of the working state of the engineering machinery; According to the action characteristics, an operation index of the engineering machinery is quantified, and the operation index is used to characterize the operation efficiency of the engineering machinery.

2. The method according to claim 1, characterized in that The step of extracting the motion features of the working state of the engineering machinery according to the perception data and the pre-trained motion recognition model includes: Inputting the perception data into a pre-trained motion recognition model to extract motion features of the working state of the engineering machinery; Outputting action features corresponding to the perception data; The action features corresponding to the perception data are used as the action features of the working state of the engineering machinery.

3. The method according to claim 1 or 2, characterized in that: Follow these steps to generate a pre-trained action recognition model, including: Collect historical perception data of the working environment from various sensors installed on the engineering machinery, the historical perception data including image frames, inertial measurement unit data and mechanical state data; Annotating the image frames, IMU data, and machine state data with motion features of the working state of the engineering machinery to obtain model training samples; Inject a preset loss function into the neural network to obtain an action recognition model; The action recognition model is deeply learned according to the model training samples to generate a pre-trained action recognition model.

4. The method according to claim 4, characterized in that: The preset loss function is a cross entropy loss function, and the cross entropy loss function is: Among them, L is the cross entropy loss value, N is the number of samples of model training samples, C is the number of categories of action features, and y i,c is the true category label of the action feature, p i,c is the predicted category label of the action feature.

5. The method according to claim 1, characterized in that The action features include the action type and the action execution time; The step of quantifying the operation index of the engineering machinery according to the action characteristics includes: When the action type satisfies the preset action end condition and the loading completion instruction fed back by the engineering machinery is received, the action execution time is taken as the action end time; wherein the preset action end condition is that the engineering machinery loads the object to be loaded onto the transport vehicle; Obtaining the action start time corresponding to the action type; Calculating and caching the time difference between the end time of the action and the start time of the action; The operation cycle and the average value of the operation cycle are calculated according to the cached time series to obtain the operation index of the engineering machinery.

6. The method according to claim 5, characterized in that The calculating of the operation cycle and the average value of the operation cycle according to the cached time series includes: The cached time series are summed to obtain the total time required for the engineering machinery to complete a whole loading process; The calculation formula for the average value of the operation cycle is: in, is the average value of the operation cycle, t start ,t end are the end time of the action and the start time of the action respectively, N cycles is the number of cycles, τ j is the j-th time difference in the cached time series.

7. The method according to claim 5, characterized in that The method further comprises: When the action type meets the preset action end condition and no loading completion instruction is received from the engineering machinery, the step of receiving the perception data of the working environment from various sensors installed on the engineering machinery is continued.

8. The method according to claim 5, characterized in that The method further comprises: In the case where the action type meets a preset action start condition, the action execution time is used as the action start time corresponding to the action type; wherein the preset action start condition is that the engineering machinery obtains the object to be loaded.

9. The method according to claim 1, characterized in that: The action characteristics include the actual load under the operation action; The step of quantifying the operation index of the engineering machinery according to the action characteristics includes: Calculate the ratio of the actual load under the operation action to the actual load of the preset maximum load; The ratio is converted into a percentage expression to obtain the operation index of the engineering machinery.

10. A device for quantifying the working efficiency of an engineering machine, characterized in that: The device comprises: A receiving module is used to receive the perception data of the working environment from various sensors installed on the engineering machinery; An extraction module, used to extract the motion features of the working state of the engineering machinery according to the perception data and a pre-trained motion recognition model, wherein the pre-trained motion recognition model is a neural network based on deep learning for generating the motion features of the working state of the engineering machinery; The quantification module is used to quantify the operation index of the engineering machinery according to the action characteristics, and the operation index is used to characterize the operation efficiency of the engineering machinery.

Images