Overhead Crane Operation Warning Method, Device, Electronic Device and Readable Storage Medium

Abstract

The present invention provides a warning method, device, electronic device and readable storage medium for overhead crane operations. Based on the image of the overhead crane operation area captured by a monocular camera vertically arranged downward and fixed on the overhead crane crossbeam, it avoids the problem in the prior art that the recognition of real space information is inaccurate due to the installation of the monocular camera on the wall. Further, through the recognition frame of the overhead crane suspended target recognized from the image of the overhead crane operation area, a danger area frame that can accurately represent the current front and rear danger areas of the overhead crane suspended target is generated. Then, it is determined whether a warning is needed based on the danger area frame, with low labor costs, reduced probability of false alarms or missed alarms, and improved safety.

Description

Technical Field

[0001] The present invention belongs to the technical field of overhead crane operation warning, and particularly relates to an overhead crane operation warning method, device, electronic device and readable storage medium. Background Art

[0002] In the overhead crane operation scenario, for safety considerations, it is often required that there should be no one within a certain distance in front of and behind the suspended goods of the overhead crane. For example, when hoisting pipes, the left and right sides of the suspended pipes are the areas for the staff to operate, and people can be present in the non-dangerous areas. However, the area within 2 meters in front of and behind the suspended pipes is a dangerous area where no one is allowed.

[0003] In order to avoid accidents, the following methods are usually adopted:

[0004] 1. Arrange safety supervisors to conduct full-process supervision of the overhead crane operation on-site. This method has a high labor cost and depends on the working attitude of the safety supervisors, and the safety cannot be guaranteed.

[0005] 2. Determine whether there is anyone in the dangerous area through visual recognition and give an alarm when there is someone. This method is usually based on the images captured by a monocular camera installed on the wall. Since the imaging target at different angles and distances does not have an equal ratio relationship and a linear relationship with its actual distance in the real space, when the overhead crane operation is far from the camera, the target is relatively small in the image, and when the overhead crane operation is close to the camera, the target is relatively large in the image. Even with calculation compensation, it is very difficult to accurately represent the real distance between the person and the suspended goods, resulting in false alarms or missed alarms, and the effect is not good. Summary of the Invention

[0006] Based on this, in view of the above technical problems, an overhead crane operation warning method, device, electronic device and readable storage medium are provided.

[0007] The technical solution adopted by the present invention is as follows:

[0008] As the first aspect of the present invention, an overhead crane operation warning method is provided, including:

[0009] S101. Obtain the image of the overhead crane operation area captured by a monocular camera, and the monocular camera is arranged vertically downward and fixed on the overhead crane beam;

[0010] S102. Identify the overhead crane suspended target and the human target in the image of the overhead crane operation area, and determine the coordinates of two corner points corresponding to the two ends of the overhead crane suspended target among the corner points of the recognition frame of the overhead crane suspended target;

[0011] S103. Generate a dangerous area frame for the overhead crane suspended target according to the coordinates of the two corner points:

[0012] S1031) Determine, for each of the two corner points, the corresponding first target point and second target point:

[0013] a) Generate the plurality of edge points in the overhead crane operation area image according to the preset coordinates of the plurality of edge points, and determine the left and right edge boundaries of the overhead crane operation area based on the plurality of edge points;

[0014] b) Determine the three edge points closest to the current corner point;

[0015] c) Determine the slopes and intercepts of the first line and the second line, where the first line and the second line are respectively the lines connecting the point in the middle in the up and down direction of the three edge points to the other two points;

[0016] d) Translate the first line and the second line respectively to pass through the current corner point;

[0017] e) Determine the coordinates of the first target point on the first line according to the slope and intercept of the first line, and determine the coordinates of the second target point on the second line according to the slope and intercept of the second line, where the distance between the first target point and the current corner point is equal to a first preset value, the distance between the second target point and the current corner point is equal to a second preset value, and the first preset value and the second preset value respectively correspond to the preset rear-side dangerous distance and the preset front-side dangerous distance of the overhead crane suspended target;

[0018] S1032) Connect the two corner points and the first target point and the second target point of each corner point in sequence to form the dangerous area frame;

[0019] S104. Determine whether the human target recognition frame is located within the left and right edge boundaries and within the dangerous area frame. If so, perform an alarm process.

[0020] As a second aspect of the present invention, there is provided an overhead crane operation alarm device, including:

[0021] An image acquisition module, configured to acquire an overhead crane operation area image captured by a monocular camera, where the monocular camera is arranged vertically downward and fixed on the overhead crane crossbeam;

[0022] A target recognition module, configured to recognize the overhead crane suspended target and the human target in the overhead crane operation area image, and determine the coordinates of two corner points corresponding to both ends of the overhead crane suspended target among the corner points of the overhead crane suspended target recognition frame;

[0023] A dangerous area frame generation module, configured to generate a dangerous area frame of the overhead crane suspended target according to the coordinates of the two corner points:

[0024] Determine, for each of the two corner points, the corresponding first target point and second target point:

[0025] a) Generate the multiple edge points in the overhead crane operation area screen according to the preset coordinates of the multiple edge points, and determine the left and right edge boundaries of the overhead crane operation area according to the multiple edge points;

[0026] b) Determine the three edge points closest to the current corner point;

[0027] c) Determine the slopes and intercepts of the first line and the second line, where the first line and the second line are respectively the lines connecting the point in the middle in the up and down direction of the screen among the three edge points and the other two points;

[0028] d) Translate the first line and the second line respectively to pass through the current corner point;

[0029] e) Determine the coordinates of the first target point on the first line according to the slope and intercept of the first line, and determine the coordinates of the second target point on the second line according to the slope and intercept of the second line, where the distance between the first target point and the current corner point is equal to a first preset value, the distance between the second target point and the current corner point is equal to a second preset value, and the first preset value and the second preset value respectively correspond to the preset rear-side dangerous distance and the preset front-side dangerous distance of the overhead crane suspended target;

[0030] Connect the two corner points and the first target point and the second target point of each corner point in sequence to form the dangerous area frame;

[0031] An alarm module, configured to determine whether the human target recognition frame satisfies being within the left and right edge boundaries and within the dangerous area frame, and if so, perform an alarm process.

[0032] As a third aspect of the present invention, there is provided an electronic device, including a storage module, where the storage module includes instructions loaded and executed by a processor, and when the instructions are executed, the processor executes an overhead crane operation alarm method according to the first aspect above.

[0033] As a fourth aspect of the present invention, there is provided a computer-readable storage medium, where the computer-readable storage medium stores one or more programs, and when the one or more programs are executed by a processor, an overhead crane operation alarm method according to the first aspect above is implemented.

[0034] The present invention is based on the image of the overhead crane operation area captured by a monocular camera vertically arranged downward and fixed on the overhead crane beam, thus avoiding the problem in the prior art that the recognition of real space information is inaccurate due to the installation of the monocular camera on the wall. Further, through the recognition frame of the overhead crane suspended target recognized from the image of the overhead crane operation area, a danger area frame that can accurately represent the current front and rear danger areas of the overhead crane suspended target is generated, and then it is judged whether an alarm is needed based on the danger area frame. The labor cost is low, the probability of false alarm or missed alarm is reduced, and the safety is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The present invention will be described in detail below with reference to the drawings and specific embodiments:

[0036] Figure 1 It is a flowchart of a crane operation warning method provided by an embodiment of the present invention;

[0037] Figure 2 It is a schematic diagram of the recognition frame of the overhead crane suspended target and the recognition frame of the human target of the present invention;

[0038] Figure 3 It is a schematic diagram of the nearest three edge points of the present invention;

[0039] Figure 4 It is a schematic diagram of the principle for determining the first target point of the present invention;

[0040] Figure 5 It is a schematic diagram of the danger area frame of the present invention;

[0041] Figure 6 It is a schematic diagram of an overhead crane operation warning device provided by an embodiment of the present invention;

[0042] Figure 7 It is a schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0043] The embodiments of the present invention will be described below with reference to the accompanying drawings of the specification. It should be noted that the embodiments involved in this specification are not exhaustive and do not represent the only embodiments of the present invention. The following corresponding embodiments are only for clearly explaining the inventive content of the present invention and do not limit its embodiments. For those of ordinary skill in the art, different forms of changes and modifications can be made based on the description of this embodiment. Any obvious changes or modifications that belong to the technical concept and inventive content of the present invention are also within the protection scope of the present invention.

[0044] As Figure 1 shown, an embodiment of the present invention provides an overhead crane operation warning method, including:

[0045] S101. Obtain the image of the overhead crane operation area captured by a monocular camera. The monocular camera is arranged vertically downward and fixed on the overhead crane beam. In this way, during the overhead crane operation, since the height of the lifted goods is basically unchanged, and the monocular camera always moves horizontally synchronously with the overhead crane beam, the problem of inaccurate recognition of real space information caused by installing the monocular camera on the wall in the prior art can be avoided.

[0046] S102. Through a deep learning object detection model (such as the YOLO model), identify the overhead crane suspended object and the human object in the image of the overhead crane operation area, obtain the recognition frames of the overhead crane suspended object and the human object, and determine the coordinates of two corner points corresponding to the two ends of the overhead crane suspended object among the corner points of the recognition frame of the overhead crane suspended object.

[0047] In the coordinate system of the image of the overhead crane operation area, the left - right direction is the x - axis, and the up - down direction is the y - axis.

[0048] During the overhead crane operation, the overhead crane suspended object usually deflects, and its two ends often lie at the diagonals of the recognition frame. The above - mentioned two corner points corresponding to the two ends of the overhead crane suspended object refer to the diagonals of the recognition frame where the two ends of the overhead crane suspended object are located. In this embodiment, as Figure 2 shown, the overhead crane suspended object refers to the overhead crane suspended pipe, that is, the pipe suspended after being lifted. The above - mentioned two corner points are the upper - left corner point A and the lower - right corner point B.

[0049] Among them, the training process of the deep learning object detection model is as follows:

[0050] Training sample acquisition:

[0051] a. Collect the first video of the overhead crane operation area with overhead crane operation captured by the monocular camera, the second video of the overhead crane operation area without overhead crane operation but with goods and people, and the third video of the overhead crane operation area without overhead crane operation, goods, and people.

[0052] b. Obtain pictures from the first video as sample A, pictures from the second video as sample B, and pictures from the third video as sample C. The ratio of sample A, sample B, and sample C is 6:3:1.

[0053] c. Use an annotation tool (such as labelImg) to annotate the following labels for the sample pictures:

[0054] Overhead_Travelling_Crane label: Annotate the overhead crane suspended object;

[0055] person label: Annotate the human object;

[0056] Input the annotated sample pictures into the deep learning object detection model for training.

[0057] S103. Generate a dangerous area box for the suspended target of the overhead crane according to the coordinates of two corner points:

[0058] S1031) Determine the corresponding first target point and second target point for each of the two corner points:

[0059] a) Generate multiple edge points in the overhead crane operation area image according to the preset coordinates of multiple edge points, and determine the left and right edge boundaries of the overhead crane operation area based on the multiple edge points. The left and right edge boundary lines are shown in Figure 2 .

[0060] Among them, the image of the overhead crane operation area can be pre-shot by a monocular camera, and multiple edge points can be manually scanned in the image to determine and save the coordinates of the above-mentioned multiple edge points. Thus, multiple edge points can be generated based on these pre-saved coordinates. Since the monocular camera is fixed on the overhead crane crossbeam and moves horizontally synchronously with the overhead crane crossbeam, the coordinates of these edge points are valid in any overhead crane operation area image captured by the monocular camera.

[0061] b) Determine the three edge points closest to the current corner point.

[0062] c) Determine the slopes and intercepts of the first line and the second line. The first line and the second line are respectively the lines connecting the point in the middle in the up and down direction of the three edge points with the other two points.

[0063] As Figure 3 shown, taking the upper left corner point A(X1, Y1) as an example, the three edge points closest to it are a1, a2, and a3. Among them, a2 is the point in the middle in the up and down direction (y-axis direction) of the image. The line connecting a1 and a2 is the first line, denoted as S1, and the line connecting a2 and a3 is the second line, denoted as S2.

[0064] The slope and intercept of S1 can be calculated from the coordinates of a1(x1, y1) and a2(x2, y2):

[0065] The slope k1 = (y2 - y1) / (x2 - x1), and then according to the slope formula y = k * x + b, calculate the intercept b1 = y1 - k1 * x1.

[0066] Similarly, the slope and intercept of S2 can be calculated, which will not be elaborated here.

[0067] d) Translate the first line and the second line respectively to pass through the current corner point.

[0068] As Figure 4 shown, translate S1 and S2 to pass through the upper left corner point A.

[0069] e) Determine the coordinates of the first target point on the first line based on the slope and intercept of the first line, and determine the coordinates of the second target point on the second line based on the slope and intercept of the second line, where the distance between the first target point and the current corner point is equal to the first preset value, and the distance between the second target point and the current corner point is equal to the second preset value. The first preset value and the second preset value respectively correspond to the preset rear dangerous distance and the preset front dangerous distance of the suspended target of the overhead crane.

[0070] As Figure 4 shown, taking the first line S1 as an example, the determination process of the first target point A1 is as follows:

[0071] Calculate the angle q between S1 and the x-axis: q = atan(k1);

[0072] Calculate the deviation delta_x in the x-axis direction between the first target point A1(1_x, 1_y) and the upper left corner point A(X1, Y1): delta_x = meter * cos(q), where meter is the above-mentioned first preset value. It should be noted that both the first preset value and the second preset value are empirical values. In this embodiment, both the first preset value and the second preset value are set to 20 pixels, corresponding to a dangerous distance of 2 meters at the rear and 2 meters at the front of the suspended target of the overhead crane. That is, in the image captured by the monocular camera of this application, 20 pixels is equivalent to 2 meters of actual distance. Of course, the first preset value and the second preset value can also be set to be different.

[0073] Calculate the coordinates of A1(1_x, 1_y): 1_x = X1 - delta_x, 1_y = k1 * 1_x + b1.

[0074] Similarly, the coordinates of the second target point A2 can be obtained.

[0075] Similarly, the coordinates of the corresponding first target point B1 and second target point B2 of the lower right corner point B can be obtained, which will not be elaborated here.

[0076] S1032) Connect the two corner points and the first target point and the second target point of each corner point in sequence to form a dangerous area box.

[0077] As Figure 5 shown, connect A1, A, A2, B2, B, and B1 in sequence to form a dangerous area box newbox, which can accurately represent the current front and rear dangerous areas of the suspended target of the overhead crane. Figure 5 Among them, oldbox represents the recognition box of the suspended target.

[0078] S104. Determine whether the human target recognition box satisfies being within the left and right edge boundaries and within the dangerous area box. If so, perform an alarm process.

[0079] In this embodiment, in order to improve the recognition accuracy, it is determined whether an alarm needs to be processed based on the center point of the lower edge of the human target recognition frame, that is, it is determined whether the center point of the lower edge of the human target recognition frame is located within the left and right edge boundaries and within the dangerous area frame. If so, an alarm is processed.

[0080] As can be seen from the above, the method of this embodiment is based on the image of the overhead crane operation area captured by a monocular camera vertically downward and fixed on the overhead crane crossbeam, thus avoiding the problem of inaccurate recognition of real space information in the prior art due to the installation of the monocular camera on the wall. Further, through the recognition frame of the overhead crane suspended target recognized from the overhead crane operation area image, a dangerous area frame that can accurately represent the current front and rear dangerous areas of the overhead crane suspended target is generated, and then it is determined whether an alarm is needed based on this dangerous area frame. The labor cost is low, the probability of false alarms or missed alarms is reduced, and the safety is improved.

[0081] The following will describe in detail the overhead crane operation alarm device according to one or more embodiments of the present invention. Those skilled in the art can understand that these alarm devices can all be configured by using commercially available hardware components through the steps taught by this solution. Figure 6 The following shows an overhead crane operation alarm device provided by an embodiment of the present invention, as Figure 6 shown, the alarm device includes an image acquisition module 11, a target recognition module 12, a dangerous area frame generation module 13, and an alarm module 14.

[0082] The image acquisition module 11 is used to acquire the image of the overhead crane operation area captured by the monocular camera. The monocular camera is vertically downward and fixed on the overhead crane crossbeam. In this way, during the overhead crane operation, since the height of the lifted goods is basically unchanged, and the monocular camera always moves horizontally synchronously with the overhead crane crossbeam, the problem of inaccurate recognition of real space information in the prior art due to the installation of the monocular camera on the wall can be avoided.

[0083] The target recognition module 12 is used to recognize the overhead crane suspended target and the human target in the overhead crane operation area image through a deep learning target detection model (such as the yolo model), obtain the overhead crane suspended target recognition frame and the human target recognition frame, and determine the coordinates of two corner points corresponding to the two ends of the overhead crane suspended target among the corner points of the overhead crane suspended target recognition frame.

[0084] In the coordinate system of the overhead crane operation area image, the left and right directions are the x-axis, and the up and down directions are the y-axis.

[0085] During the overhead crane operation, the overhead crane suspended target usually deflects, and its two ends often lie at the diagonals of the recognition frame. The above two corner points corresponding to the two ends of the overhead crane suspended target refer to the diagonals of the recognition frame where the two ends of the overhead crane suspended target are located. In this embodiment, as Figure 2As shown, the overhead travelling crane suspended target refers to the overhead travelling crane suspended pipe, that is, the pipe suspended after being lifted. The above two corner points are the upper left corner point A and the lower right corner point B.

[0086] Among them, the training process of the deep learning object detection model is as follows:

[0087] Training sample acquisition:

[0088] a. Collect the first video with overhead travelling crane operations in the overhead travelling crane operation area captured by a monocular camera, the second video with goods and people but no overhead travelling crane operations in the overhead travelling crane operation area, and the third video with no overhead travelling crane operations, goods, and people in the overhead travelling crane operation area.

[0089] b. Obtain pictures from the first video as sample A, obtain pictures from the second video as sample B, and obtain pictures from the third video as sample C. The ratio of sample A, sample B, and sample C is 6:3:1.

[0090] c. Use a labeling tool (such as labelImg) to label the following tags for the sample pictures:

[0091] Overhead_Travelling_Crane tag: Label the overhead travelling crane suspended target;

[0092] person tag: Label the person target;

[0093] Input the labeled sample pictures into the deep learning object detection model for training.

[0094] The dangerous area box generation module 13 is used to generate a dangerous area box for the overhead travelling crane suspended target according to the coordinates of the two corner points:

[0095] S1031) Determine the corresponding first target point and second target point for each corner point among the two corner points:

[0096] a) Generate multiple edge points in the overhead travelling crane operation area picture according to the preset coordinates of multiple edge points, and determine the left and right edge boundaries of the overhead travelling crane operation area according to the multiple edge points.

[0097] Among them, the overhead travelling crane operation area picture can be pre-captured by a monocular camera, and multiple edge points are manually scanned in the picture, and the coordinates of the above multiple edge points are determined and saved. Thus, multiple edge points can be generated based on these pre-saved coordinates. Since the monocular camera is fixed on the overhead travelling crane crossbeam and moves horizontally synchronously with the overhead travelling crane crossbeam, the coordinates of these edge points are valid in any overhead travelling crane operation area picture captured by this monocular camera.

[0098] b) Determine the three edge points closest to the current corner point.

[0099] c) Determine the slopes and intercepts of the first line and the second line, where the first line and the second line are respectively the lines connecting the point in the middle in the up-down direction of the screen among the three edge points and the other two points.

[0100] As Figure 3 shown, taking the upper left corner point A(X1, Y1) as an example, the three edge points closest to it are a1, a2, and a3. Among them, a2 is the point in the middle in the up-down direction (y-axis direction) of the screen. The line connecting a1 and a2 is the first line, denoted as S1, and the line connecting a2 and a3 is the second line, denoted as S2.

[0101] The slope and intercept of S1 can be calculated through the coordinates of a1(x1, y1) and a2(x2, y2):

[0102] The slope k1 = (y2 - y1) / (x2 - x1), and then according to the slope formula y = k * x + b, calculate the intercept b1 = y1 - k1 * x1.

[0103] Similarly, the slope and intercept of S2 can be calculated, which will not be elaborated here.

[0104] d) Translate the first line and the second line respectively to pass through the current corner point.

[0105] As Figure 4 shown, translate S1 and S2 to pass through the upper left corner point A.

[0106] e) According to the slope and intercept of the first line, determine the coordinates of the first target point on the first line. According to the slope and intercept of the second line, determine the coordinates of the second target point on the second line. Among them, the distance between the first target point and the current corner point is equal to the first preset value, and the distance between the second target point and the current corner point is equal to the second preset value. The first preset value and the second preset value respectively correspond to the preset rear dangerous distance and the preset front dangerous distance of the overhead crane suspended target.

[0107] As Figure 4 shown, taking the first line S1 as an example, the determination process of the first target point A1 is as follows:

[0108] Calculate the angle q between S1 and the x-axis: q = atan(k1);

[0109] Calculate the deviation delta_x in the x-axis direction between the first target point A1(1_x, 1_y) and the upper left corner point A(X1, Y1): delta_x = meter * cos(q), where meter is the above-mentioned first preset value. It should be noted that both the first preset value and the second preset value are empirical values. In this embodiment, both the first preset value and the second preset value are set to 20 pixels, corresponding to a dangerous distance of 2 meters behind and 2 meters in front of the suspended target of the overhead crane. That is, in the picture captured by the monocular camera of the present application, 20 pixels are equivalent to 2 meters of actual distance. Of course, the first preset value and the second preset value can also be set to be different.

[0110] Calculate the coordinates of A1(1_x, 1_y): 1_x = X1 - delta_x, 1_y = k1 * 1_x + b1.

[0111] Similarly, the coordinates of the second target point A2 can be obtained.

[0112] Similarly, the coordinates of the first target point B1 and the second target point B2 corresponding to the lower right corner point B can be obtained, which will not be elaborated here.

[0113] S1032) Connect the two corner points and the first target point and the second target point of each corner point in sequence to form a dangerous area box.

[0114] As Figure 5 shown, connect A1, A, A2, B2, B, and B1 in sequence to form a dangerous area box newbox. This dangerous area box can accurately represent the current front and rear dangerous areas of the suspended target of the overhead crane. Figure 5 In it, oldbox represents the recognition box of the suspended target.

[0115] The warning module 14 is used to determine whether the human target recognition box satisfies being within the left and right edge boundaries and within the dangerous area box. If so, warning processing is performed.

[0116] In this embodiment, in order to improve the recognition accuracy, it is determined whether warning processing is required based on the center point of the lower edge of the human target recognition box, that is, to determine whether the center point of the lower edge of the human target recognition box satisfies being within the left and right edge boundaries and within the dangerous area box. If so, warning processing is performed.

[0117] In summary, the overhead crane operation warning device provided in the above embodiment can execute the overhead crane operation warning method provided in the foregoing embodiments.

[0118] With the same concept as above, the structure of the crane operation warning device shown above Figure 6 can be implemented as an electronic device. Figure 7 The structural schematic block diagram of an electronic device provided by an embodiment of the present invention is shown.

[0119] Exemplarily, the electronic device includes a storage module 21 and a processor 22. The storage module 21 includes instructions loaded and executed by the processor 22. When executed, the instructions cause the processor 22 to execute the steps according to various exemplary embodiments of the present invention described in the above-mentioned part of a gantry crane operation warning method of this specification.

[0120] It should be understood that the processor 22 may be a central processing unit (CPU), and the processor 22 may also be other general-purpose processors, digital signal processors

[0121] (Digital Signal Processor, DSP), application-specific integrated circuit

[0122] (Application Specific Integrated Circuit, ASIC), field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0123] An embodiment of the present invention also provides a computer-readable storage medium. The computer-readable storage medium stores one or more programs. When the one or more programs are executed by a processor, the steps according to various exemplary embodiments of the present invention described in the above-mentioned part of a gantry crane operation warning method are implemented.

[0124] Those of ordinary skill in the art can understand that all or some of the steps in the above-disclosed methods, and the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, and their appropriate combinations. In the hardware implementation, the division of the functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be executed by several physical components in cooperation. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable storage medium, and the computer-readable storage medium may include a computer-readable storage medium (or non-transitory medium) and a communication medium (or transitory medium).

[0125] As is well known to those of ordinary skill in the art, the term computer-readable storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVDs) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. Additionally, as is well known to those of ordinary skill in the art, communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and can include any information delivery media.

[0126] Exemplarily, the computer-readable storage medium can be the internal storage unit of the electronic device in the foregoing embodiments, such as the hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the electronic device.

[0127] The electronic device and the computer-readable storage medium provided in the foregoing embodiments are based on the picture of the overhead crane operation area captured by a monocular camera vertically downwardly arranged and fixed on the overhead crane crossbeam, thereby avoiding the problem of inaccurate recognition of real space information caused by the installation of the monocular camera on the wall in the prior art. Further, through the recognition frame of the overhead crane suspended target recognized from the picture of the overhead crane operation area, a danger area frame that can accurately represent the current front and rear danger areas of the overhead crane suspended target is generated, and then it is determined whether an alarm is needed based on this danger area frame, with low labor costs, reducing the probability of false alarms or missed alarms, and improving safety.

[0128] Obviously, those skilled in the art can make various changes and modifications to this application without departing from the scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalent technologies, this application is also intended to include these changes and modifications.

Claims

1. A warning method for overhead crane operations, characterized in that, Including: S101. Obtain the picture of the overhead crane operation area captured by a monocular camera, where the monocular camera is arranged vertically downward and fixed on the overhead crane beam; S102. Identify the overhead crane suspended target and the human target in the picture of the overhead crane operation area, and determine the coordinates of two corner points corresponding to both ends of the overhead crane suspended target among the corner points of the recognition frame of the overhead crane suspended target; S103. Generate a dangerous area frame for the overhead crane suspended target according to the coordinates of the two corner points: S1031) Determine the corresponding first target point and second target point for each corner point among the two corner points: a) Generate the multiple edge points in the picture of the overhead crane operation area according to the preset coordinates of multiple edge points, and determine the left and right edge boundaries of the overhead crane operation area according to the multiple edge points; b) Determine the three edge points closest to the current corner point; c) Determine the slopes and intercepts of the first line and the second line, where the first line and the second line are respectively the lines connecting the point in the middle in the up and down direction of the three edge points with the other two points; d) Translate the first line and the second line respectively to pass through the current corner point; e) Determine the coordinates of the first target point on the first line according to the slope and intercept of the first line, and determine the coordinates of the second target point on the second line according to the slope and intercept of the second line, where the distance between the first target point and the current corner point is equal to the first preset value, the distance between the second target point and the current corner point is equal to the second preset value, and the first preset value and the second preset value respectively correspond to the preset rear dangerous distance and the preset front dangerous distance of the overhead crane suspended target; S1032) Connect the two corner points and the first target point and the second target point of each corner point in sequence to form the dangerous area frame; S104. Determine whether the recognition frame of the human target satisfies being within the left and right edge boundaries and within the dangerous area frame. If so, perform an alarm process.

2. The warning method for overhead crane operation according to claim 1, wherein, The identifying the overhead crane suspended target and the human target in the picture of the overhead crane operation area further includes: Identifying the overhead crane suspended target and the human target in the picture of the overhead crane operation area through a deep learning target detection model.

3. The warning method for overhead crane operation according to claim 2, characterized in that, The deep learning target detection model adopts the YOLO model.

4. The warning method for gantry crane operation according to claim 3, wherein The deep learning target detection model is trained through the following steps: Training sample acquisition: a. Collect the first video with overhead crane operation in the overhead crane operation area captured by the monocular camera, the second video with goods and people but without overhead crane operation in the overhead crane operation area, and the third video without overhead crane operation, goods and people in the overhead crane operation area; b. Obtain pictures from the first video as sample A, obtain pictures from the second video as sample B, and obtain pictures from the third video as sample C. The ratio of sample A, sample B, and sample C is 6:3:1; c. Label the following labels for the sample pictures: Overhead_Travelli ng_Crane label: Label the overhead crane suspended target; person label: Label the human target; Input the labeled sample pictures into the deep learning target detection model for training.

5. The warning method for gantry crane operation according to claim 4, wherein The S104 further includes: Determine whether the lower center point of the human target recognition frame is located within the left and right edge boundaries and within the dangerous area frame. If so, perform an alarm process.

6. The warning method for gantry crane operation according to claim 5, wherein, The overhead crane suspended target is the overhead crane suspended pipe.

7. A gantry crane operation warning method according to claim 6, characterized in that Both the first preset value and the second preset value are 20 pixels, and the preset rear dangerous distance and the preset front dangerous distance of the overhead crane suspended target are both 2 meters.

8. An overhead crane operation warning device, characterized in that It includes: A picture acquisition module, configured to acquire a picture of the overhead crane operation area captured by a monocular camera, where the monocular camera is arranged vertically downward and fixed on the overhead crane crossbeam; A target recognition module, configured to recognize the overhead crane suspended target and the human target in the picture of the overhead crane operation area, and determine the coordinates of two corner points corresponding to both ends of the overhead crane suspended target among the corner points of the overhead crane suspended target recognition frame; A dangerous area frame generation module, configured to generate a dangerous area frame of the overhead crane suspended target according to the coordinates of the two corner points: Determine the first target point and the second target point corresponding to each of the two corner points: a) Generate the multiple edge points in the picture of the overhead crane operation area according to the preset multiple edge point coordinates, and determine the left and right edge boundaries of the overhead crane operation area according to the multiple edge points; b) Determine the three edge points closest to the current corner point; c) Determine the slopes and intercepts of the first line and the second line, where the first line and the second line are respectively the lines connecting the point in the middle in the up and down direction of the three edge points to the other two points; d) Translate the first line and the second line respectively to pass through the current corner point; e) Determine the coordinates of the first target point on the first line according to the slope and intercept of the first line, and determine the coordinates of the second target point on the second line according to the slope and intercept of the second line, where the distance between the first target point and the current corner point is equal to the first preset value, the distance between the second target point and the current corner point is equal to the second preset value, and the first preset value and the second preset value respectively correspond to the preset rear dangerous distance and the preset front dangerous distance of the overhead crane suspended target; Connect the two corner points and the first target point and the second target point of each corner point in sequence to form the dangerous area frame; An alarm module, configured to determine whether the human target recognition frame is located within the left and right edge boundaries and within the dangerous area frame. If so, perform an alarm process.

9. An electronic device, characterized in that, It includes a storage module, and the storage module includes instructions loaded and executed by a processor, and when the instructions are executed, the processor executes an overhead crane operation alarm method according to any one of claims 1-7.

10. A computer-readable storage medium storing one or more programs, characterized in that, When the one or more programs are executed by a processor, an overhead crane operation alarm method according to any one of claims 1-7 is implemented.

Images