Hopper overflow detection method, device, engineering machinery and storage medium

Abstract

The present application relates to the technical field of engineering machinery, in particular to a method, device, engineering machinery and storage medium for detecting hopper overflow. The method includes: real-time acquisition of a video image of the hopper; obtaining the recognition result of the current frame image; assigning a value to the constructed initial parameter group; when the result variable of the current frame image is equal to the algorithm result parameter after the assignment, adding one to the value of the count variable to obtain an updated count variable; when the value of the updated count variable is greater than the count parameter after the assignment, obtaining the value of the frame number parameter; when the value of the updated count variable is greater than the value of the frame number parameter, returning the value of the updated count variable to zero, and assigning the algorithm result parameter after the assignment to the result variable of the previous frame image to obtain the final value of the result variable of the previous frame image; determining whether the hopper overflows according to the final value of the result variable of the previous frame image, realizing fully automated overflow detection, and improving detection accuracy.

Description

Technical Field

[0001] The present application relates to the technical field of engineering machinery, and in particular to a method and device for detecting overflow in a hopper, engineering machinery and a storage medium. Background Art

[0002] With the continuous development of society, more and more infrastructure such as roads and bridges are being built. In the construction of infrastructure, concrete is an indispensable raw material. The unloading process of concrete refers to the process of unloading concrete from the mixer truck to the hopper of the construction machinery. During the unloading process, the unloading flow rate is too fast, which often causes the concrete to overflow the hopper, that is, overflow often occurs, which will cause waste of concrete and increase costs. Therefore, it is very important to prevent overflow during the unloading process.

[0003] In the prior art, it is generally done by manually observing whether the hopper is overflowing, that is, the staff always keeps an eye on the material level of the hopper during the pumping operation. If the staff fails to adjust the discharge flow rate in time, it is easy to cause the concrete in the hopper to be emptied or overflow the hopper. Moreover, the method of completely relying on manual labor not only has high labor costs, but also has a low degree of automation, which is prone to omissions and leads to inaccurate detection results. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a hopper overflow detection method, device, engineering machinery and storage medium to solve the problems of inaccurate overflow detection and low degree of automation in the prior art.

[0005] In order to achieve the above-mentioned purpose, the first aspect of the present application provides a method for detecting hopper overflow, which is applied to engineering machinery. The detection method comprises:

[0006] During the operation of the construction machinery, real-time video images of the hopper are collected;

[0007] Inputting the current frame image in the video image into the image recognition algorithm model to obtain the recognition result of the current frame image;

[0008] Assigning values ​​to the constructed initial parameter group, wherein the initial parameter group includes an algorithm result parameter, a counting parameter, and a frame number parameter, assigning the recognition result to the algorithm result parameter, and assigning the value of the counting variable to the counting parameter, wherein the counting variable, the result variable of the previous frame image, and the result variable of the current frame image constitute the constructed variable array;

[0009] Determine whether the result variable of the current frame image is equal to the algorithm result parameter after assignment;

[0010] When the result variable of the current frame image is equal to the algorithm result parameter after the assignment, the value of the count variable is increased by one to obtain an updated count variable;

[0011] Determine whether the value of the updated count variable is greater than the assigned count parameter;

[0012] When the value of the updated count variable is greater than the assigned count parameter, the value of the frame number parameter is obtained, and it is determined whether the value of the updated count variable is greater than the value of the frame number parameter;

[0013] When the value of the updated counting variable is greater than the value of the frame number parameter, the value of the updated counting variable is reset to zero, and the assigned algorithm result parameter is assigned to the result variable of the previous frame image to obtain the final value of the result variable of the previous frame image;

[0014] When the final value of the result variable of the previous frame image is the first preset value, it is determined that the hopper overflows;

[0015] When the final value of the result variable of the previous frame image is the second preset value, it is determined that no overflow occurs in the hopper.

[0016] In an embodiment of the present application, the initial parameter group also includes the result parameter of the previous frame image, and the value of the result variable of the previous frame image is assigned to the result parameter of the previous frame image. The detection method also includes: when the value of the updated count variable is less than or equal to the value of the frame number parameter, the assigned algorithm result parameter is assigned to the result variable of the current frame image, and the assigned result parameter of the previous frame image is assigned to the result variable of the previous frame image; determining whether overflow occurs in the hopper based on the result variable of the previous frame image after the assignment, and returning to the step of assigning the constructed initial parameter group again until the value of the updated count variable is greater than the value of the frame number parameter.

[0017] In an embodiment of the present application, the detection method also includes: when the result variable of the current frame image is not equal to the algorithm result parameter after assignment, the value of the counting variable is reset to zero, and the assigned algorithm result parameter is assigned to the result variable of the current frame image, and the result parameter of the previous frame image after assignment is assigned to the result variable of the previous frame image; the value of the counting variable after being reset to zero is assigned to the assigned counting parameter; and returning to the step of determining whether the result variable of the current frame image is equal to the assigned algorithm result parameter until the result variable of the current frame image is equal to the assigned algorithm result parameter.

[0018] In an embodiment of the present application, the detection method also includes: when the value of the updated count variable is less than or equal to the assigned count parameter, assigning the assigned algorithm result parameter to the result variable of the previous frame image to obtain the final value of the result variable of the previous frame image; determining whether overflow occurs in the hopper based on the assigned result variable of the previous frame image, and returning to the step of assigning the constructed initial parameter group until the value of the updated count variable is greater than the assigned count parameter.

[0019] In an embodiment of the present application, the initial parameter group also includes the result parameter of the current frame image, and the value of the result variable of the current frame image is assigned to the result parameter of the current frame image. The detection method also includes: when the value of the updated count variable is less than or equal to the assigned count parameter, the assigned algorithm result parameter is assigned to the result variable of the current frame image to obtain the final value of the result variable of the current frame image.

[0020] In an embodiment of the present application, the detection method further includes: when it is determined that overflow occurs in the hopper, issuing a warning signal.

[0021] In an embodiment of the present application, inputting the current frame image in the video image into the image recognition algorithm model includes: cropping the current frame image to obtain a cropped image; and inputting the cropped image into the image recognition algorithm model to obtain a recognition result.

[0022] A second aspect of the present application provides a hopper overflow detection device, comprising:

[0023] a memory configured to store instructions;

[0024] The processor is configured to call instructions from the memory and implement the above-mentioned hopper overflow detection method when executing the instructions.

[0025] A third aspect of the present application provides an engineering machine, comprising:

[0026] hopper;

[0027] A video acquisition device is used to collect video data of the hopper in real time during the operation of the construction machinery;

[0028] The above-mentioned hopper overflow detection device.

[0029] A fourth aspect of the present application provides a machine-readable storage medium having instructions stored thereon, the instructions being used to enable a machine to execute the above-mentioned method for detecting hopper overflow.

[0030] Through the above technical scheme, the video image of the hopper can be collected in real time, and the recognition result of the current frame image can be determined through the image recognition algorithm model; then the variable array is used to assign the constructed initial parameter array, and the assigned initial parameter array and variable array are looped to obtain the final value of the result variable of the previous frame image of the current frame image, and the final value of the result variable of the previous frame image is determined as the recognition result of the smoothed image recognition algorithm model to determine whether the hopper is overflowing, realize fully automated overflow detection, improve detection accuracy, avoid waste of resources caused by overflow, and improve the operating efficiency of construction machinery.

[0031] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present application, but do not constitute a limitation on the embodiments of the present application. In the accompanying drawings:

[0033] Figure 1 A schematic diagram of a first process of a method for detecting overflow in a hopper according to an embodiment of the present application is shown;

[0034] Figure 2 A schematic diagram of a second flow chart of a method for detecting overflow in a hopper according to an embodiment of the present application is schematically shown;

[0035] Figure 3 A schematic diagram of a third flow chart of a hopper overflow detection method according to an embodiment of the present application is schematically shown;

[0036] Figure 4 The internal structure diagram of a computer device according to an embodiment of the present application is schematically shown.

[0037] Description of Reference Numerals

[0038] A01: processor; A02: network interface; A03: internal memory; A04: non-volatile storage medium; B01: operating system; B02: computer program. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the specific implementation methods described herein are only used to illustrate and explain the embodiments of the present application, and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0040] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back...), such directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0041] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0042] Figure 1 The first flow chart of a method for detecting hopper overflow according to an embodiment of the present application is schematically shown. Figure 1 As shown, an embodiment of the present application provides a method for detecting overflow in a hopper, which is applied to engineering machinery. The detection method may include the following steps.

[0043] Step 101: During the operation of the construction machinery, a video image of the hopper is collected in real time.

[0044] Step 102: Input the current frame image in the video image into the image recognition algorithm model to obtain the recognition result of the current frame image.

[0045] Engineering machinery is an important part of the equipment industry. In general, the mechanical equipment required for earthwork construction, road construction and maintenance, mobile lifting and loading and unloading operations, and comprehensive mechanized construction projects required for various construction projects are called engineering machinery. In a specific embodiment, the engineering machinery may be a mixer truck, a pump truck, etc. During the operation of the engineering machinery, the operation process may be a concrete pumping operation. The processor may capture a video image of the hopper in real time. In an optional embodiment, the processor may obtain a video image of the hopper captured in real time by a video acquisition device such as a camera or a camera. After obtaining the video image of the hopper, the processor may input the current frame image in the video image into the image recognition algorithm model to obtain the recognition result of the current frame image. The image recognition algorithm model may be obtained by training with a pre-constructed training data set.

[0046] In an embodiment of the present application, inputting the current frame image in the video image into the image recognition algorithm model includes: cropping the current frame image to obtain a cropped image; and inputting the cropped image into the image recognition algorithm model to obtain a recognition result.

[0047] The processor may input the current frame image in the video image into the image recognition algorithm model. Specifically, the processor may crop the current frame image to obtain a cropped image. For example, a hopper ROI (Region of Interest) cropping may be performed to obtain an ROI cropped image. After obtaining the cropped image, the processor may input the cropped image into the image recognition algorithm model to obtain a recognition result.

[0048] Step 103: Assign values ​​to the constructed initial parameter group, wherein the initial parameter group includes an algorithm result parameter, a counting parameter, and a frame number parameter. Assign the recognition result to the algorithm result parameter, and assign the value of the counting variable to the counting parameter. The counting variable, the result variable of the previous frame image, and the result variable of the current frame image constitute the constructed variable array.

[0049] Step 104: Determine whether the result variable of the current frame image is equal to the algorithm result parameter after assignment.

[0050] Step 105: When the result variable of the current frame image is equal to the algorithm result parameter after the assignment, the value of the count variable is increased by one to obtain an updated count variable.

[0051] The processor may assign values ​​to the constructed initial parameter group. The initial parameter group includes an algorithm result parameter, a counting parameter, and a frame number parameter. The processor may assign the recognition result to the algorithm result parameter, and assign the value of the counting variable to the counting parameter. The counting variable, the result variable of the previous frame image, and the result variable of the current frame image constitute a constructed variable array. The initial value of each variable in the constructed variable array may be determined based on actual conditions. The initial value of each parameter of the constructed initial parameter group may be determined based on actual conditions. After assigning values ​​to the constructed initial parameter group, the processor may determine whether the result variable of the current frame image is equal to the algorithm result parameter after assignment. When the result variable of the current frame image is equal to the algorithm result parameter after assignment, the processor may add one to the value of the counting variable to obtain an updated counting variable.

[0052] In an embodiment of the present application, the detection method also includes: when the result variable of the current frame image is not equal to the algorithm result parameter after assignment, the value of the counting variable is reset to zero, and the assigned algorithm result parameter is assigned to the result variable of the current frame image, and the result parameter of the previous frame image after assignment is assigned to the result variable of the previous frame image; the value of the counting variable after being reset to zero is assigned to the assigned counting parameter; and returning to the step of determining whether the result variable of the current frame image is equal to the assigned algorithm result parameter until the result variable of the current frame image is equal to the assigned algorithm result parameter.

[0053] In the case where the result variable of the current frame image is not equal to the algorithm result parameter after assignment, the processor may reset the value of the count variable to zero. The processor may assign the algorithm result parameter after assignment to the result variable of the current frame image, and assign the result parameter of the previous frame image after assignment to the result variable of the previous frame image, so that the result variable of the current frame image can accurately record the correct recognition result of the current frame image. The processor may also assign the value of the count variable after reset to zero to the count parameter after assignment. After assigning the value of the count variable after reset to zero to the count parameter after assignment, the processor may return to the step of determining whether the result variable of the current frame image is equal to the algorithm result parameter after assignment, until the result variable of the current frame image is equal to the algorithm result parameter after assignment.

[0054] Step 106: Determine whether the value of the updated counting variable is greater than the assigned counting parameter.

[0055] Step 107: When the updated value of the counter variable is greater than the assigned counter parameter, obtain the value of the frame number parameter and determine whether the updated value of the counter variable is greater than the value of the frame number parameter.

[0056] Step 108: When the value of the updated count variable is greater than the value of the frame number parameter, the value of the updated count variable is reset to zero, and the assigned algorithm result parameter is assigned to the result variable of the previous frame image to obtain the final value of the result variable of the previous frame image.

[0057] After adding one to the value of the count variable to obtain the updated count variable, the processor may determine whether the value of the updated count variable is greater than the assigned count parameter. In the case where the value of the updated count variable is greater than the assigned count parameter, the processor may obtain the value of the frame number parameter. After obtaining the value of the frame number parameter, the processor may determine whether the value of the updated count variable is greater than the value of the frame number parameter. In the case where the value of the updated count variable is greater than the value of the frame number parameter, the processor may reset the value of the updated count variable to zero and assign the assigned algorithm result parameter to the result variable of the previous frame image to obtain the final value of the result variable of the previous frame image.

[0058] In an embodiment of the present application, the initial parameter group also includes the result parameter of the previous frame image, and the value of the result variable of the previous frame image is assigned to the result parameter of the previous frame image. The detection method also includes: when the value of the updated count variable is less than or equal to the value of the frame number parameter, the assigned algorithm result parameter is assigned to the result variable of the current frame image, and the assigned result parameter of the previous frame image is assigned to the result variable of the previous frame image; determining whether overflow occurs in the hopper based on the result variable of the previous frame image after the assignment, and returning to the step of assigning the constructed initial parameter group again until the value of the updated count variable is greater than the value of the frame number parameter.

[0059] The initial parameter group also includes the result parameter of the previous frame image, and the value of the result variable of the previous frame image is assigned to the result parameter of the previous frame image. When the value of the updated count variable is less than or equal to the value of the frame number parameter, the processor can assign the assigned algorithm result parameter to the result variable of the current frame image, and assign the assigned result parameter of the previous frame image to the result variable of the previous frame image. After obtaining the result variable of the previous frame image after the assignment, the processor can determine whether the hopper has overflowed according to the result variable of the previous frame image after the assignment, and return to the step of assigning the constructed initial parameter group again until the value of the updated count variable is greater than the value of the frame number parameter. For example, when the result variable of the previous frame image after the assignment is 1, the processor can determine that the hopper has overflowed, and output the identification result hopper overflow, and return to the step of assigning the constructed initial parameter group again until the value of the updated count variable is greater than the value of the frame number parameter. When the result variable of the previous frame image after assignment is 0, the processor can determine that the hopper has not overflowed, and output the recognition result that the hopper has not overflowed, and return to the step of assigning the constructed initial parameter group until the value of the updated count variable is greater than the value of the frame number parameter, so as to improve the detection accuracy of whether the hopper has overflowed.

[0060] In an embodiment of the present application, the detection method also includes: when the value of the updated count variable is less than or equal to the assigned count parameter, assigning the assigned algorithm result parameter to the result variable of the previous frame image to obtain the final value of the result variable of the previous frame image; determining whether overflow occurs in the hopper based on the assigned result variable of the previous frame image, and returning to the step of assigning the constructed initial parameter group until the value of the updated count variable is greater than the assigned count parameter.

[0061] When the value of the updated count variable is less than or equal to the assigned count parameter, the processor can assign the assigned algorithm result parameter to the result variable of the previous frame image to obtain the final value of the result variable of the previous frame image. After obtaining the final value of the result variable of the previous frame image, the processor can determine whether the hopper overflows according to the assigned result variable of the previous frame image, and return to the step of assigning the constructed initial parameter group again until the value of the updated count variable is greater than the assigned count parameter.

[0062] In an embodiment of the present application, the initial parameter group also includes the result parameter of the current frame image, and the value of the result variable of the current frame image is assigned to the result parameter of the current frame image. The detection method also includes: when the value of the updated count variable is less than or equal to the assigned count parameter, the assigned algorithm result parameter is assigned to the result variable of the current frame image to obtain the final value of the result variable of the current frame image.

[0063] The initial parameter group also includes the result parameter of the current frame image, and the value of the result variable of the current frame image is assigned to the result parameter of the current frame image. When the value of the updated count variable is less than or equal to the count parameter after the assignment, the processor can assign the assigned algorithm result parameter to the result variable of the current frame image to obtain the final value of the result variable of the current frame image.

[0064] Step 109: When the final value of the result variable of the previous frame image is the first preset value, it is determined that overflow occurs in the hopper.

[0065] Step 110: When the final value of the result variable of the previous frame image is the second preset value, it is determined that no overflow occurs in the hopper.

[0066] After obtaining the final value of the result variable of the previous frame image, the processor can determine whether the final value of the result variable of the previous frame image is equal to the first preset value or the second preset value, wherein the first preset value and the second preset value can be two unequal values ​​determined based on actual conditions. For example, the first preset value can be 1, and the second preset value can be 0. When the final value of the result variable of the previous frame image is the first preset value, the processor can determine that the hopper has overflowed. In an embodiment of the present application, when it is determined that the hopper has overflowed, the processor can issue an early warning signal. When the final value of the result variable of the previous frame image is the second preset value, the processor can determine that the hopper has not overflowed.

[0067] In one embodiment, Figure 2As shown, the processor can collect hopper images as a data set. After obtaining the data set, the processor can preprocess the data set, specifically including image enhancement and making overflow feature training data sets. The processor can perform algorithm training, convert the format after generating the model, and deploy it on the edge device. The processor can crop the hopper RIO area of ​​the camera video frame and pass it into the model. After the model is detected, the data is smoothed and then it is determined whether there is overflow. If there is overflow, an overflow signal is output. If there is no overflow, it returns to cropping the hopper RIO area of ​​the camera video frame and passing it into the model.

[0068] Specifically, Figure 3 As shown, the processor can set 5 initial parameters, namely, algorithm result parameter, frame number parameter, count parameter, previous frame result parameter, and current frame result parameter, wherein the current frame result parameter refers to the current frame result parameter of the current frame image, and the previous frame result parameter refers to the previous frame result parameter of the previous frame image of the current frame image. The processor can set 3 variable values ​​corresponding to 3 parameter values, and the 3 variables are count variable, previous frame result variable, and current frame result variable, which correspond to count parameter, previous frame result parameter, and current frame result parameter, respectively. The processor can assign the algorithm real-time detection result to parameter 1 (algorithm result parameter), and the 3 variable values ​​are assigned to the corresponding parameters, that is, the count variable is assigned to the count parameter, the previous frame result variable is assigned to the previous frame result parameter, and the current frame result variable is assigned to the current frame result parameter. The processor can determine whether the value of variable 3 (current frame result variable) is equal to parameter 1 (algorithm result parameter). When the value of variable 3 is not equal to parameter 1, the processor can reset variable 1 (count variable) to 0, assign parameter 1 to variable 3, assign parameter 4 (previous frame result parameter) to variable 2 (previous frame result variable), output the value of variable 2, and return to the step of assigning the real-time detection result of the algorithm to parameter 1 (algorithm result parameter), and assigning the values ​​of the three variables to the corresponding parameters respectively, until the value of variable 3 is equal to parameter 1.

[0069] When the value of variable 3 is equal to parameter 1, the processor can determine that the count of variable 1 is plus 1 to obtain the updated variable 1, and determine whether the updated value of variable 1 is greater than the count parameter. When the value of variable 1 after the update is less than or equal to the count parameter, the processor can assign parameter 1 to variable 3, assign parameter 4 to variable 2, and output the value of variable 2. When the value of variable 1 after the update is greater than the count parameter, the processor can determine whether the value of variable 1 after the update is greater than the value of parameter 2 (number of frames parameter). When the value of variable 1 after the update is greater than the value of parameter 2, the processor can assign parameter 1 to variable 3, assign parameter 4 to variable 2, output the value of variable 2, and return to the step of assigning the algorithm real-time detection result to parameter 1 (algorithm result parameter), and assigning the three variable values ​​to the corresponding parameters respectively, until the updated value of variable 1 is greater than the value of parameter 2. When the value of variable 1 after the update is greater than the value of parameter 2, the processor can return the updated value of variable 1 to 0, assign parameter 1 to variables 2 and variables 3, and output the value of variable 2, so that the value of variable 2 is used as the smoothed result.

[0070] Through the above technical solution, fully automated overflow detection can be achieved, the detection accuracy can be improved, the waste of resources caused by overflow can be avoided, and the operating efficiency of construction machinery can be improved.

[0071] Figure 1 , Figure 2 as well as Figure 3 FIG. 1 is a flow chart of a method for detecting a hopper overflow in one embodiment. It should be understood that although Figure 1 , Figure 2 as well as Figure 3 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 , Figure 2 as well as Figure 3 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0072] The present application also provides a device for detecting overflow in a hopper, comprising:

[0073] a memory configured to store instructions;

[0074] The processor is configured to call instructions from the memory and implement the above-mentioned hopper overflow detection method when executing the instructions.

[0075] The present application also provides an engineering machine, including:

[0076] hopper;

[0077] A video acquisition device is used to collect video data of the hopper in real time during the operation of the construction machinery;

[0078] The above-mentioned hopper overflow detection device.

[0079] The engineering machinery of this embodiment includes traditional engineering machinery vehicles and also includes new energy vehicles used in the field of engineering machinery, such as new energy mixer trucks, new energy pump trucks, new energy excavators, etc.; in addition, the engineering machinery vehicles of this embodiment also belong to intelligent networked vehicles. The engineering machinery vehicles include sensor systems, communication systems, etc., and collect vehicle operation data and vehicle surrounding environment information through the sensor system in the vehicle. The network connection with other vehicles and the cloud is realized through the communication system, and the collected vehicle operation data, vehicle surrounding environment information, etc. are shared to the cloud and other authorized vehicles to realize data sharing, remote analysis, intelligent driving and other operations.

[0080] An embodiment of the present application also provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute the above-mentioned hopper overflow detection method.

[0081] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor A01, a network interface A02, a memory (not shown in the figure) and a database (not shown in the figure) connected through a system bus. Among them, the processor A01 of the computer device is used to provide computing and control capabilities. The memory of the computer device includes an internal memory A03 and a non-volatile storage medium A04. The non-volatile storage medium A04 stores an operating system B01, a computer program B02 and a database (not shown in the figure). The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 in the non-volatile storage medium A04. The database of the computer device is used to store data such as video images, recognition results, initial parameter groups, variable arrays, etc. The network interface A02 of the computer device is used to communicate with an external terminal through a network connection. When the computer program B02 is executed by the processor A01, a method for detecting hopper overflow is implemented.

[0082] Those skilled in the art will understand that Figure 4The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0083] The embodiment of the present application provides a device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, the following steps are implemented: during the operation of the engineering machinery, real-time video images of the hopper are collected; the current frame image in the video image is input into the image recognition algorithm model to obtain the recognition result of the current frame image; the constructed initial parameter group is assigned, wherein the initial parameter group includes an algorithm result parameter, a counting parameter, and a frame number parameter, the recognition result is assigned to the algorithm result parameter, and the value of the counting variable is assigned to the counting parameter, wherein the counting variable, the result variable of the previous frame image, and the result variable of the current frame image constitute the constructed variable array; it is determined whether the result variable of the current frame image is equal to the algorithm result parameter after the assignment; the result variable of the current frame image, etc. In the case of the algorithm result parameter after the assignment, the value of the counting variable is increased by one to obtain the updated counting variable; it is determined whether the value of the updated counting variable is greater than the counting parameter after the assignment; in the case that the value of the updated counting variable is greater than the counting parameter after the assignment, the value of the frame number parameter is obtained, and it is determined whether the value of the updated counting variable is greater than the value of the frame number parameter; in the case that the value of the updated counting variable is greater than the value of the frame number parameter, the value of the updated counting variable is reset to zero, and the algorithm result parameter after the assignment is assigned to the result variable of the previous frame image to obtain the final value of the result variable of the previous frame image; in the case that the final value of the result variable of the previous frame image is the first preset value, it is determined that the hopper has overflowed; in the case that the final value of the result variable of the previous frame image is the second preset value, it is determined that the hopper has not overflowed.

[0084] In one embodiment, the initial parameter group also includes the result parameter of the previous frame image, and the value of the result variable of the previous frame image is assigned to the result parameter of the previous frame image. The detection method also includes: when the value of the updated count variable is less than or equal to the value of the frame number parameter, the assigned algorithm result parameter is assigned to the result variable of the current frame image, and the assigned result parameter of the previous frame image is assigned to the result variable of the previous frame image; determining whether overflow occurs in the hopper based on the result variable of the previous frame image after the assignment, and returning to the step of assigning the constructed initial parameter group again until the value of the updated count variable is greater than the value of the frame number parameter.

[0085] In one embodiment, the detection method also includes: when the result variable of the current frame image is not equal to the algorithm result parameter after assignment, the value of the counting variable is reset to zero, and the assigned algorithm result parameter is assigned to the result variable of the current frame image, and the result parameter of the previous frame image after assignment is assigned to the result variable of the previous frame image; the value of the counting variable after reset to zero is assigned to the assigned counting parameter; and returning to the step of determining whether the result variable of the current frame image is equal to the assigned algorithm result parameter until the result variable of the current frame image is equal to the assigned algorithm result parameter.

[0086] In one embodiment, the detection method also includes: when the value of the updated count variable is less than or equal to the assigned count parameter, assigning the assigned algorithm result parameter to the result variable of the previous frame image to obtain the final value of the result variable of the previous frame image; determining whether overflow occurs in the hopper based on the assigned result variable of the previous frame image, and returning to the step of assigning the constructed initial parameter group until the value of the updated count variable is greater than the assigned count parameter.

[0087] In one embodiment, the initial parameter group also includes the result parameter of the current frame image, and the value of the result variable of the current frame image is assigned to the result parameter of the current frame image. The detection method also includes: when the value of the updated count variable is less than or equal to the assigned count parameter, the assigned algorithm result parameter is assigned to the result variable of the current frame image to obtain the final value of the result variable of the current frame image.

[0088] In one embodiment, the detection method further comprises: when it is determined that overflow occurs in the hopper, issuing an early warning signal.

[0089] In one embodiment, inputting the current frame image in the video image into the image recognition algorithm model includes: cropping the current frame image to obtain a cropped image; and inputting the cropped image into the image recognition algorithm model to obtain a recognition result.

[0090] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program for initializing steps of a method for detecting hopper overflow.

[0091] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0092] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0093] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0094] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0095] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0096] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0097] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0098] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0099] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for detecting hopper overflow, characterized in that: Applied to engineering machinery, the detection method includes: During the operation of the engineering machinery, real-time video images of the hopper are collected; Inputting the current frame image in the video image into an image recognition algorithm model to obtain a recognition result of the current frame image; Assigning values ​​to the constructed initial parameter group, wherein the initial parameter group includes an algorithm result parameter, a counting parameter, and a frame number parameter, assigning the recognition result to the algorithm result parameter, and assigning the value of the counting variable to the counting parameter, wherein the counting variable, the result variable of the previous frame image, and the result variable of the current frame image constitute the constructed variable array; Determine whether the result variable of the current frame image is equal to the algorithm result parameter after assignment; When the result variable of the current frame image is equal to the algorithm result parameter after the assignment, the value of the count variable is increased by one to obtain an updated count variable; Determine whether the value of the updated count variable is greater than the assigned count parameter; In the case where the value of the updated count variable is greater than the assigned count parameter, obtaining the value of the frame number parameter, and determining whether the value of the updated count variable is greater than the value of the frame number parameter; In the case where the value of the updated count variable is greater than the value of the frame number parameter, the value of the updated count variable is reset to zero, and the assigned algorithm result parameter is assigned to the result variable of the previous frame image to obtain the final value of the result variable of the previous frame image; When the final value of the result variable of the previous frame image is a first preset value, it is determined that overflow occurs in the hopper; When the final value of the result variable of the previous frame image is the second preset value, it is determined that no overflow occurs in the hopper.

2. The method for detecting hopper overflow according to claim 1, characterized in that: The initial parameter group also includes a result parameter of a previous frame image, and the value of the result variable of the previous frame image is assigned to the result parameter of the previous frame image. The detection method also includes: When the value of the updated count variable is less than or equal to the value of the frame number parameter, the algorithm result parameter after the assignment is assigned to the result variable of the current frame image, and the result parameter of the previous frame image after the assignment is assigned to the result variable of the previous frame image; Determine whether the hopper overflows according to the result variable of the last frame image after assignment, and return to the step of assigning the constructed initial parameter group again until the value of the updated count variable is greater than the value of the frame number parameter.

3. The method for detecting hopper overflow according to claim 2, characterized in that: The detection method further comprises: In the case where the result variable of the current frame image is not equal to the algorithm result parameter after the assignment, the value of the counting variable is reset to zero, and the algorithm result parameter after the assignment is assigned to the result variable of the current frame image, and the result parameter of the previous frame image after the assignment is assigned to the result variable of the previous frame image; Assign the value of the count variable after zeroing to the count parameter after assignment; Return to the step of determining whether the result variable of the current frame image is equal to the algorithm result parameter after the assignment, until the result variable of the current frame image is equal to the algorithm result parameter after the assignment.

4. The method for detecting hopper overflow according to claim 1, characterized in that: The detection method further comprises: When the value of the updated counting variable is less than or equal to the assigned counting parameter, assigning the assigned algorithm result parameter to the result variable of the previous frame image to obtain the final value of the result variable of the previous frame image; Determine whether the hopper overflows according to the result variable of the last frame image after assignment, and return to the step of assigning the constructed initial parameter group until the value of the updated count variable is greater than the assigned count parameter.

5. The method for detecting hopper overflow according to claim 4, characterized in that: The initial parameter group also includes a result parameter of the current frame image, and the value of the result variable of the current frame image is assigned to the result parameter of the current frame image. The detection method also includes: When the value of the updated count variable is less than or equal to the assigned count parameter, the assigned algorithm result parameter is assigned to the result variable of the current frame image to obtain the final value of the result variable of the current frame image.

6. The method for detecting hopper overflow according to claim 1, characterized in that: The detection method further comprises: When it is determined that the hopper is overflowing, an early warning signal is issued.

7. The method for detecting hopper overflow according to claim 1, characterized in that: The step of inputting the current frame image in the video image into the image recognition algorithm model comprises: Cropping the current frame image to obtain a cropped image; The cropped image is input into the image recognition algorithm model to obtain the recognition result.

8. A device for detecting overflow of a hopper, characterized in that: include: a memory configured to store instructions; A processor is configured to call the instructions from the memory and implement the hopper overflow detection method according to any one of claims 1 to 7 when executing the instructions.

9. An engineering machine, characterized in that: include: hopper; A video acquisition device, used to acquire video data of the hopper in real time during the operation of the engineering machinery; A hopper overflow detection device according to claim 8.

10. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores instructions for enabling a machine to execute the method for detecting overflow in a hopper according to any one of claims 1 to 7.

Images