Forklift illegal driving behavior identification method and system

Abstract

The invention provides a forklift illegal driving behavior identification method and a forklift illegal driving behavior identification system, and relates to the technical field of monitoring image processing. The forklift illegal driving behavior recognition method comprises the steps that target image information collected by monitoring equipment is obtained; setting at least one algorithm recognition area according to the target image information; determining whether the cargo is higher than the human body according to the coordinates of the personnel identification area and the cargo identification area in the algorithm identification domain; judging whether the target forklift is in a motion state or not according to the multi-frame image information of the forklift recognition area; the movement direction of the target forklift is judged according to the coordinates of the first forklift feature recognition area and the second forklift feature recognition area in the algorithm recognition domain; and under the condition that the movement direction of the target forklift is the advancing direction, it is determined that the illegal driving behavior of the first forklift exists. According to the invention, the illegal driving behavior of the driver can be accurately identified, alarm prompting can be carried out in time, and the hidden danger problem of safe operation is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of monitoring image processing, and in particular to a forklift illegal driving behavior recognition method and system thereof. Background Art

[0002] A forklift is a wheeled transport vehicle used for loading, unloading, stacking and short-distance transportation of palletized goods. It is widely used in manufacturing, logistics, warehousing, transportation and other fields. Identification of forklift driving violations is an important measure to monitor forklift safety, which can effectively detect non-compliant forklift operating behaviors.

[0003] At present, the monitoring system can monitor the illegal driving behavior of forklifts in real time. However, the inventors found that the current methods for identifying illegal driving behavior of forklifts mostly judge whether the forklift has violated the regulations based on the width-to-height ratio of the forklift cargo area, and cannot judge the relative status of the cargo and the personnel, resulting in low efficiency in identifying illegal driving behaviors. Summary of the invention

[0004] The present invention provides a forklift illegal driving behavior recognition method and a forklift illegal driving behavior recognition system, which are used to solve the problems that the current forklift illegal driving behavior recognition method cannot judge the relative status of goods and personnel and the illegal driving behavior recognition efficiency is low.

[0005] One aspect of the present invention provides a method for identifying illegal driving behaviors of forklifts, comprising: acquiring target image information collected by a monitoring device; setting at least one algorithm recognition area according to the target image information; in the algorithm recognition area, determining a forklift recognition area, a personnel recognition area, a cargo recognition area, a first feature recognition area of ​​the forklift, and a second feature recognition area of ​​the forklift based on a forklift feature code; determining whether the cargo is higher than a human body according to the coordinates of the personnel recognition area and the cargo recognition area in the algorithm recognition domain; if the cargo is higher than the human body, judging whether the target forklift is in motion according to multi-frame image information of the forklift recognition area; if the target forklift is in motion, judging the direction of motion of the target forklift according to the coordinates of the first feature recognition area of ​​the forklift and the second feature recognition area of ​​the forklift in the algorithm recognition domain; and if the direction of motion of the target forklift is a forward direction, determining that there is a first illegal driving behavior of the forklift.

[0006] According to some embodiments of the present invention, determining whether cargo is higher than a human body based on the coordinates of a personnel identification area and a cargo identification area in an algorithm identification domain includes: determining the coordinates of key feature points of a person in the personnel identification area; determining the coordinates of key feature points of cargo in the cargo identification area; judging the heights of a person and cargo based on the coordinates of the key feature points of the person and the coordinates of the key feature points of the cargo; and determining that the cargo is higher than a human body when the height of the cargo is higher than the height of the person.

[0007] According to some embodiments of the present invention, judging whether a target forklift is in motion based on multiple frames of image information of a forklift identification area includes: determining the coordinates of a first forklift key feature point of a first frame of image information of the forklift identification area, the first width information of the forklift identification area, and the first height information of the forklift identification area; determining the coordinates of a first forklift area center point in the forklift identification area based on the first forklift key feature point coordinates, the first width information, and the first height information; determining the coordinates of a second forklift key feature point of a second frame of image information of the forklift identification area, the second width information of the forklift identification area, and the second height information of the forklift identification area; determining the coordinates of a second forklift area center point in the forklift identification area based on the second forklift key feature point coordinates, the second width information, and the second height information; determining a first displacement vector based on the coordinates of the first forklift area center point and the second forklift area center point coordinates; and determining that the target forklift is in motion when the first displacement vector is greater than a first preset threshold.

[0008] According to some embodiments of the present invention, judging the movement direction of the target forklift based on the coordinates of the first feature recognition area and the second feature recognition area of ​​the forklift in the algorithm recognition domain includes: determining the key feature point coordinates of the first feature recognition area of ​​the forklift, the fifth width information of the first feature recognition area of ​​the forklift, and the fifth height information of the first feature recognition area of ​​the forklift; determining the coordinates of the center point of the first feature recognition area of ​​the forklift based on the key feature point coordinates, the fifth width information, and the fifth height information of the first feature recognition area of ​​the forklift; determining the key feature point coordinates of the second feature recognition area of ​​the forklift, the sixth width information of the second feature recognition area of ​​the forklift, and the sixth height information of the second feature recognition area of ​​the forklift; determining the coordinates of the center point of the second feature recognition area of ​​the forklift based on the key feature point coordinates, the sixth width information, and the sixth height information of the second feature recognition area of ​​the forklift; determining the second displacement vector based on the coordinates of the center point of the first feature recognition area of ​​the forklift and the coordinates of the center point of the second feature recognition area of ​​the forklift; and determining the target forklift as being in the forward direction when the angle between the second displacement vector and the first displacement vector is less than a second preset threshold.

[0009] According to some embodiments of the present invention, the method for identifying illegal forklift driving behavior also includes: performing human body key point detection and human body segmentation on the personnel identification area to obtain human body key point information and human body contour area information; determining the hand contour area based on the human body key point information and the human body contour area information; performing image classification and recognition on the hand contour area, and when the image classification and recognition result is the operating electronic device type, determining that a second illegal forklift driving behavior exists.

[0010] Another aspect of the present invention provides a forklift illegal driving behavior recognition system, including an image acquisition module, a region recognition module and a behavior determination module. The image acquisition module acquires the target image information collected by the monitoring device; the region recognition module sets at least one algorithm recognition area according to the target image information, and in the algorithm recognition area, the forklift recognition area, the personnel recognition area, the cargo recognition area, the forklift first feature recognition area and the forklift second feature recognition area of ​​the target forklift are determined based on the forklift feature code; the behavior determination module determines whether the cargo is higher than the human body according to the coordinates of the personnel recognition area and the cargo recognition area in the algorithm recognition domain, and if the cargo is higher than the human body, it determines whether the target forklift is in motion according to the multi-frame image information of the forklift recognition area, and if the target forklift is in motion, it determines the moving direction of the target forklift according to the coordinates of the forklift first feature recognition area and the forklift second feature recognition area in the algorithm recognition domain, and if the moving direction of the target forklift is the forward direction, it determines that there is a first forklift illegal driving behavior.

[0011] According to some embodiments of the present invention, the behavior determination module determines the coordinates of the key feature points of the person in the personnel identification area; the behavior determination module determines the coordinates of the key feature points of the cargo in the cargo identification area; the behavior determination module judges the heights of the person and the cargo based on the coordinates of the key feature points of the person and the coordinates of the key feature points of the cargo; the behavior determination module determines that the cargo is taller than the human body when the height of the cargo is higher than the height of the person.

[0012] According to some embodiments of the present invention, a behavior determination module determines the coordinates of a first forklift key feature point of a first frame of image information of a forklift identification area, the first width information of the forklift identification area, and the first height information of the forklift identification area; the behavior determination module determines the coordinates of a first forklift area center point in the forklift identification area based on the first forklift key feature point coordinates, the first width information, and the first height information; the behavior determination module determines the coordinates of a second forklift key feature point of a second frame of image information of the forklift identification area, the second width information of the forklift identification area, and the second height information of the forklift identification area; the behavior determination module determines the coordinates of a second forklift area center point in the forklift identification area based on the second forklift key feature point coordinates, the second width information, and the second height information; the behavior determination module determines a first displacement vector based on the coordinates of the first forklift area center point and the second forklift area center point coordinates; the behavior determination module determines that the target forklift is in a moving state when the first displacement vector is greater than a first preset threshold.

[0013] According to some embodiments of the present invention, the behavior determination module determines the key feature point coordinates of the first feature recognition area of ​​the forklift, the fifth width information of the first feature recognition area of ​​the forklift, and the fifth height information of the first feature recognition area of ​​the forklift; the behavior determination module determines the center point coordinates of the first feature recognition area of ​​the forklift based on the key feature point coordinates, the fifth width information, and the fifth height information of the first feature recognition area of ​​the forklift; the behavior determination module determines the key feature point coordinates of the second feature recognition area of ​​the forklift, the sixth width information of the second feature recognition area of ​​the forklift, and the sixth height information of the second feature recognition area of ​​the forklift; the behavior determination module determines the center point coordinates of the second feature recognition area of ​​the forklift based on the key feature point coordinates, the sixth width information, and the sixth height information of the second feature recognition area of ​​the forklift; the behavior determination module determines the second displacement vector based on the center point coordinates of the first feature recognition area of ​​the forklift and the center point coordinates of the second feature recognition area of ​​the forklift; the behavior determination module determines that the target forklift is in the forward direction when the angle between the second displacement vector and the first displacement vector is less than the second preset threshold.

[0014] According to some embodiments of the present invention, the behavior determination module performs human key point detection and human body segmentation on the personnel identification area to obtain human key point information and human body contour area information; the behavior determination module determines the hand contour area based on the human key point information and the human body contour area information; the behavior determination module performs image classification and recognition on the hand contour area, and when the image classification and recognition result is the operation of an electronic device type, it is determined that there is a second forklift illegal driving behavior.

[0015] Another aspect of the present invention provides an electronic device, which includes: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors can implement the forklift illegal driving behavior identification method as described above.

[0016] Another aspect of the present invention further provides a non-volatile computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the forklift illegal driving behavior identification method as described above.

[0017] The present invention determines the algorithm recognition area through the image information collected by the monitoring equipment, determines the forklift recognition area, the personnel recognition area, the cargo recognition area, the first feature recognition area of ​​the forklift and the second feature recognition area of ​​the forklift of the target forklift based on the forklift feature code, and determines whether the cargo is higher than the human body according to the coordinates of the personnel recognition area and the cargo recognition area in the algorithm recognition domain. In the case where the cargo is higher than the human body, the present invention also determines whether the target forklift is in motion according to the multi-frame image information of the forklift recognition area, and in the case where the target forklift is in motion, determines the moving direction of the target forklift according to the coordinates of the first feature recognition area of ​​the forklift and the second feature recognition area of ​​the forklift in the algorithm recognition domain, and determines that there is illegal driving behavior of the first forklift when the moving direction of the target forklift is the forward direction.

[0018] The present invention can determine whether the cargo is higher than the human body through the coordinates corresponding to the personnel identification area and the cargo identification area, can determine the motion state of the target forklift based on the multi-frame image information of the forklift identification area, and can determine the forward direction of the target forklift based on the coordinates corresponding to the forklift first feature identification area and the forklift second feature identification area. The present invention can accurately identify the safety behavior of a person standing next to a forklift that is in operation and in a forward state, and can promptly issue an alarm prompt to reduce the hidden dangers of safe operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 A flow chart showing a method for identifying illegal driving behavior of a forklift according to an embodiment of the present invention;

[0021] Figure 2 Another flow chart showing the method for identifying illegal driving behavior of a forklift according to an embodiment of the present invention;

[0022] Figure 3 Another flow chart showing the method for identifying illegal driving behavior of a forklift according to an embodiment of the present invention;

[0023] Figure 4 Another flow chart showing the method for identifying illegal driving behavior of a forklift according to an embodiment of the present invention;

[0024] Figure 5 Another flow chart showing the method for identifying illegal driving behavior of a forklift according to an embodiment of the present invention;

[0025] Figure 6A schematic diagram showing key points of a human body according to an embodiment of the present invention;

[0026] Figure 7 A schematic diagram showing a forklift illegal driving behavior recognition system according to an embodiment of the present invention.

[0027] Description of reference numerals:

[0028] A forklift illegal driving behavior recognition system 1; an image acquisition module 10; a region recognition module 20; and a behavior determination module 30. DETAILED DESCRIPTION

[0029] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that the present invention will be comprehensive and complete and fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0030] The described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of these specific details, or other modes, components, materials, devices, etc. may be adopted. In these cases, known structures, methods, devices, implementations, materials or operations will not be shown or described in detail.

[0031] In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0032] The terms "first", "second" and the like in the specification, claims and the above drawings of the present invention are used to distinguish different objects rather than to describe a specific order.

[0033] The technical solution of the present invention is described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0034] The full names of the English abbreviations involved in the present invention and their Chinese meanings are:

[0035] RTSP: Real Time Streaming Protocol, real-time streaming protocol;

[0036] SSD: Single Shot MultiBox Detector, a target detection algorithm based on deep learning;

[0037] YOLO: You Only Look Once, a real-time target detection algorithm based on deep learning;

[0038] RetinaNet: a single-stage object detection model.

[0039] RT-DETR: Real-Time Detection Transformer, real-time end-to-end object detection model;

[0040] Deeplab: image semantic segmentation model;

[0041] Mask2Former:image segmentation model;

[0042] SegFormer: semantic segmentation model.

[0043] OpenPose: Human pose estimation technology based on deep learning;

[0044] AlphaPose - a human pose estimation library based on deep learning technology.

[0045] HRNet: High-ResolutionNet, high-resolution deep neural network.

[0046] According to one aspect of the present invention, a method for identifying illegal driving behaviors of forklifts is provided, which is used to identify whether there are illegal driving behaviors that do not comply with preset forklift operation regulations in the current monitoring screen based on image information collected by monitoring equipment. Exemplarily, illegal driving behaviors of forklifts include but are not limited to people standing next to a forklift that is in operation and moving forward, people operating electronic equipment next to a forklift that is moving forward, etc.

[0047] Figure 1 A flow chart showing a method for identifying illegal driving behavior of a forklift according to an embodiment of the present invention is shown. Figure 1 As shown, the forklift driving violation behavior recognition method may include steps S100-S700. Exemplarily, the forklift driving violation behavior recognition method may be executed by a forklift driving violation behavior recognition system having computing capability.

[0048] In the present invention, a forklift may refer to any operating vehicle that can perform transportation and handling operations such as loading and unloading and transporting of palletized cargo, but the present invention is not limited thereto.

[0049] According to an example embodiment, in step S100, the forklift illegal driving behavior recognition system obtains image information collected by a monitoring device.

[0050] For example, the monitoring device may be a monocular camera installed at a preset position according to monitoring requirements. For example, the preset position may be the rear area of ​​a forklift. The resolution of the monocular camera may be above 1080p, and may have a fixed top-down angle (e.g., between 15 and 45°), focal length, and installation position.

[0051] The forklift illegal driving behavior recognition system is connected to the monocular camera for communication, and can obtain the RTSP video stream from the monocular camera, and perform stream decoding processing on the RTSP video to obtain target image information including information such as forklifts or personnel.

[0052] Exemplarily, the target image information may be color image data or grayscale image data.

[0053] In step S200, the forklift illegal driving behavior recognition system sets at least one algorithm recognition area according to the target image information.

[0054] For example, taking color image data as an example, based on the color image data, the forklift illegal driving behavior recognition system responds to user instructions and sets one or more algorithm recognition areas according to user needs.

[0055] The algorithm recognition area can be a user-defined area with monitoring and recognition significance, that is, an alarm will be issued only when there is any illegal driving behavior of a forklift in the algorithm recognition area.

[0056] In step S300, the forklift illegal driving behavior recognition system determines the forklift recognition area, the personnel recognition area, the cargo recognition area, the forklift first feature recognition area and the forklift second feature recognition area of ​​the target forklift in the algorithm recognition area based on the forklift feature code.

[0057] For example, the forklift feature code may be unique number information of a forklift, and one forklift feature code represents only one designated target forklift.

[0058] The forklift illegal driving behavior recognition system can identify the coordinates of a series of points in the algorithm recognition area, and a polygonal area can be generated based on the coordinates of these points to form a recognition area.

[0059] The forklift identification area may be the area where the target forklift is located; the personnel identification area may be the area where the personnel are located; the cargo identification area may be the area where the cargo on the pallet of the target forklift is located; the forklift first feature identification area may be the area where the front of the target forklift is located; and the forklift second feature identification area may be the area where the rear of the target forklift is located.

[0060] In step S400, the forklift driving violation behavior recognition system determines whether the cargo is higher than the human body according to the coordinates of the personnel recognition area and the cargo recognition area in the algorithm recognition domain.

[0061] The forklift driving violation behavior recognition system can compare the height of the person and the cargo based on the coordinates of the key feature points in the person recognition area and the coordinates of the key feature points in the cargo recognition area. When it is determined that the height of the cargo is higher than the height of the person, it can be further determined that the height of the pallet is higher than the person, that is, it can be characterized that the target forklift is in operation (such as the pallet is raised or the pallet is being raised, etc.), and the person is standing near the bottom of the cargo.

[0062] Figure 2 Another flow chart of the method for identifying illegal driving behavior of a forklift according to an embodiment of the present invention is shown.

[0063] Alternatively, if Figure 2 As shown, step S400 may also include steps S410-S440.

[0064] In step S410, the forklift illegal driving behavior recognition system determines the coordinates of key feature points of the person in the person recognition area.

[0065] In step S420, the forklift illegal driving behavior recognition system determines the coordinates of the cargo key feature points in the cargo identification area.

[0066] In step S430, the forklift driving violation behavior recognition system determines the heights of the person and the cargo based on the coordinates of the key feature points of the person and the coordinates of the key feature points of the cargo.

[0067] In step S440, the forklift illegal driving behavior recognition system determines that the cargo is higher than the human body when the height of the cargo is higher than the height of the person.

[0068] For example, the coordinates of the key feature point of a person may be the coordinates of the upper left point or the upper right point of the person identification area; the coordinates of the key feature point of a cargo may be the coordinates of the upper left point or the upper right point of the cargo identification area.

[0069] Exemplarily, the forklift driving violation behavior recognition system determines the coordinates of the key feature points of the personnel and the key feature points of the cargo. For example, the coordinates of the key feature points of the personnel may be (x3, y3), where x3 is the horizontal coordinate of the key feature point of the personnel, and y3 is the vertical coordinate of the key feature point of the personnel. The coordinates of the key feature points of the cargo may be (x4, y4), where x4 is the horizontal coordinate of the key feature point of the cargo, and y4 is the vertical coordinate of the key feature point of the cargo.

[0070] The forklift illegal driving behavior recognition system compares the size of y3 and y4. When y4>y3, the forklift illegal driving behavior recognition system determines that the actual height of the cargo is higher than the actual height of the person. That is, in the actual scene, the target forklift is operating and the person is standing under the cargo.

[0071] With such a configuration, the present invention can accurately determine the positional relationship between personnel and cargo through the coordinates of key feature points in the personnel identification area and the cargo identification area, thereby further determining the working status of the target forklift, so as to facilitate the subsequent identification of illegal forklift driving behavior.

[0072] In step S500, when the target forklift is in operation, the forklift illegal driving behavior recognition system determines whether the target forklift is in motion based on the multi-frame image information of the forklift recognition area.

[0073] For example, the state of the target forklift may include a moving state (or operating state) and a stationary state (or non-operating state). The forklift illegal driving behavior recognition system can comprehensively determine whether the target forklift is in an operating state based on the position information of multiple frames of image information of the forklift recognition area.

[0074] Figure 3 Another flow chart of the method for identifying illegal driving behavior of a forklift according to an embodiment of the present invention is shown.

[0075] Alternatively, if Figure 3 As shown, step S500 may also include steps S510-S560.

[0076] In step S510, the forklift illegal driving behavior recognition system determines the coordinates of the first forklift key feature point of the first frame image information of the forklift recognition area, the first width information of the forklift recognition area, and the first height information of the forklift recognition area.

[0077] For example, the forklift driving violation behavior recognition system performs forklift image recognition on the first frame of image information, and can obtain the coordinates (x1, y1) of the first forklift key feature point of the first frame of image information, the first width information w1 of the forklift recognition area, and the first height information h1 of the forklift recognition area. Exemplarily, the first forklift key feature point can be the coordinates of the upper left point of the forklift recognition area.

[0078] In step S520, the forklift illegal driving behavior recognition system determines the coordinates of the first forklift area center point of the forklift recognition area according to the coordinates of the first forklift key feature point, the first width information and the first height information.

[0079] For example, the coordinates O1 of the center point of the first forklift area may be:

[0080]

[0081] In step S530, the forklift illegal driving behavior recognition system determines the coordinates of the second forklift key feature point of the second frame image information of the forklift recognition area, the second width information of the forklift recognition area, and the second height information of the forklift recognition area.

[0082] For example, the forklift illegal driving behavior recognition system tracks the forklift based on the image recognition technology, and performs forklift image recognition on the second frame image information to obtain the coordinates (x2, y2) of the second forklift key feature point of the second frame image information, the second width information w2 of the forklift recognition area, and the second height information h2 of the forklift recognition area. Exemplarily, the second forklift key feature point can be the coordinates of the upper left point of the forklift recognition area.

[0083] In step S540, the forklift illegal driving behavior recognition system determines the coordinates of the second forklift area center point in the forklift recognition area according to the coordinates of the second forklift key feature point, the second width information and the second height information.

[0084] For example, the coordinates O2 of the center point of the second forklift area may be:

[0085]

[0086] In step S550, the forklift illegal driving behavior recognition system determines a first displacement vector according to the coordinates of the center point of the first forklift area and the coordinates of the center point of the second forklift area.

[0087] For example, the first displacement vector Can be:

[0088]

[0089] In step S560, when the first displacement vector is greater than the first preset threshold, the forklift illegal driving behavior recognition system determines that the target forklift is in a moving state.

[0090] For example, if The forklift illegal driving behavior recognition system can determine that the position information of the target forklift in the first frame image information is different from that in the second frame image information, and can determine that the target forklift is moving, that is, in a moving state. Exemplarily, the first preset threshold can be customized according to user needs. For example, the first preset threshold can be A1, such as:

[0091]

[0092] With such a configuration, the present invention can determine a first displacement vector through the position information of the target forklift in multiple frames of image information, and based on the comparison between the first displacement vector and the first preset threshold, it can be determined whether the target forklift has moved, thereby accurately determining whether the target forklift is in motion.

[0093] In step S600, when the target forklift is in motion, the forklift illegal driving behavior recognition system determines the moving direction of the target forklift according to the coordinates of the first feature recognition area and the second feature recognition area of ​​the forklift in the algorithm recognition domain.

[0094] For example, the first feature recognition area of ​​a forklift may be the front area of ​​the target forklift, and the second feature recognition area of ​​the forklift may be the rear area of ​​the target forklift.

[0095] According to an example embodiment, the forklift driving violation behavior recognition system can determine whether the moving direction of the target forklift is a forward direction or a backward direction based on the coordinates of the front area and the coordinates of the rear area.

[0096] Figure 4 Another flow chart of the method for identifying illegal driving behavior of a forklift according to an embodiment of the present invention is shown.

[0097] Alternatively, if Figure 4 As shown, step S600 may also include steps S610-S660.

[0098] In step S610, the forklift illegal driving behavior recognition system determines the key feature point coordinates of the first feature recognition area of ​​the forklift, the fifth width information of the first feature recognition area of ​​the forklift, and the fifth height information of the first feature recognition area of ​​the forklift.

[0099] For example, the forklift driving violation behavior recognition system performs forklift image recognition on the forklift first feature recognition area, and can obtain the key feature point coordinates (x5, y5) in the forklift first feature recognition area, the fifth width information w5 of the forklift first feature recognition area, and the fifth height information h5 of the forklift recognition area. Exemplarily, the key feature point in the forklift first feature recognition area can be the coordinates of the upper left point of the forklift first feature recognition area.

[0100] In step S620, the forklift illegal driving behavior recognition system determines the coordinates of the center point of the first feature recognition area of ​​the forklift according to the key feature point coordinates of the first feature recognition area of ​​the forklift, the fifth width information and the fifth height information.

[0101] For example, the center point coordinate F1 of the first feature recognition area of ​​the forklift can be:

[0102]

[0103] In step S630, the forklift illegal driving behavior recognition system determines the key feature point coordinates of the second feature recognition area of ​​the forklift, the sixth width information of the second feature recognition area of ​​the forklift, and the sixth height information of the second feature recognition area of ​​the forklift.

[0104] For example, the forklift driving violation behavior recognition system performs forklift image recognition on the forklift second feature recognition area, and can obtain the key feature point coordinates (x6, y6) in the forklift second feature recognition area, the sixth width information w6 of the forklift second feature recognition area, and the sixth height information h6 of the forklift recognition area. Exemplarily, the key feature point in the forklift second feature recognition area can be the coordinates of the upper left point of the forklift second feature recognition area.

[0105] In step S640, the forklift illegal driving behavior recognition system determines the coordinates of the center point of the second feature recognition area of ​​the forklift according to the coordinates of the key feature points of the second feature recognition area of ​​the forklift, the sixth width information and the sixth height information.

[0106] For example, the center point coordinate B1 of the second feature recognition area of ​​the forklift can be:

[0107]

[0108] In step S650, the forklift illegal driving behavior recognition system determines a second displacement vector according to the center point coordinates of the first feature recognition area of ​​the forklift and the center point coordinates of the second feature recognition area of ​​the forklift.

[0109] For example, the second displacement vector Can be:

[0110]

[0111] In step S660, the forklift illegal driving behavior recognition system determines that the target forklift is in the forward direction when the angle between the second displacement vector and the first displacement vector is less than a second preset threshold.

[0112] For example, the first displacement vector With the second displacement vector The angle between them is θ, then the calculation formula of θ can be:

[0113]

[0114]

[0115] After the forklift illegal driving behavior recognition system calculates the angle θ, when θ is less than a second preset threshold (such as 90°), the movement direction of the target forklift is determined to be the forward direction; otherwise, when θ is greater than or equal to the second preset threshold (such as 90°), the movement direction of the target forklift is determined to be the backward direction.

[0116] With such a configuration, the present invention can determine the second displacement vector through the key feature point coordinates of the first feature recognition area of ​​the forklift and the key feature point coordinates of the second feature recognition area of ​​the forklift, and based on the angle between the second displacement vector and the first displacement vector, the movement direction of the target forklift can be accurately determined.

[0117] In step S700, the forklift driving violation behavior recognition system determines that there is a first forklift driving violation behavior when the moving direction of the target forklift is a forward direction.

[0118] For example, the first forklift driving violation behavior may include a person standing next to a forklift that is in operation and moving forward. When the forklift driving violation behavior recognition system determines that the target forklift is in operation (such as the height of the pallet is higher than the height of the person) and the target forklift is moving forward, it is determined that the first forklift driving violation behavior exists and an alarm is issued.

[0119] Through the above example embodiments, the present invention determines the algorithm recognition area through the image information collected by the monitoring device, determines the forklift recognition area, the personnel recognition area, the cargo recognition area, the forklift first feature recognition area and the forklift second feature recognition area of ​​the target forklift based on the forklift feature code, and determines whether the cargo is higher than the human body according to the coordinates of the personnel recognition area and the cargo recognition area in the algorithm recognition domain. In the case where the cargo is higher than the human body, the present invention also determines whether the target forklift is in motion according to the multi-frame image information of the forklift recognition area, and in the case where the target forklift is in motion, determines the direction of motion of the target forklift according to the coordinates of the forklift first feature recognition area and the forklift second feature recognition area in the algorithm recognition domain, and in the case where the target forklift is in motion, determines that there is illegal driving behavior of the first forklift.

[0120] The present invention can determine whether the cargo is higher than the human body through the coordinates corresponding to the personnel identification area and the cargo identification area, can determine the motion state of the target forklift based on the multi-frame image information of the forklift identification area, and can determine the forward direction of the target forklift based on the coordinates corresponding to the forklift first feature identification area and the forklift second feature identification area. The present invention can accurately identify the safety behavior of a person standing next to a forklift that is in operation and in a forward state, and can promptly issue an alarm prompt to reduce the hidden dangers of safe operation.

[0121] Figure 5 Another flow chart showing the method for identifying illegal driving behavior of a forklift according to an embodiment of the present invention; Figure 6 A schematic diagram showing key points of a human body according to an embodiment of the present invention.

[0122] Alternatively, if Figure 5 As shown, the forklift illegal driving behavior identification method may further include steps S800-S1000.

[0123] In step S800, the forklift illegal driving behavior recognition system performs human key point detection and human body segmentation on the human identification area to obtain human key point information and human body contour area information.

[0124] In step S900, the forklift illegal driving behavior recognition system determines the hand contour area according to the human body key point information and the human body contour area information.

[0125] For example, the forklift driving violation behavior recognition system determines the human body area image in the personnel recognition area based on image segmentation recognition technology, and detects the key points of the human body (such as Figure 6 The 17 human key points shown in the figure) are then used to obtain the human key point coordinates corresponding to the human key points.

[0126] Exemplarily, image segmentation and recognition technologies include but are not limited to SSD, YOLO, RetinaNet or RT-DETR, etc.; human body segmentation technologies include but are not limited to Deeplab, Mask2Former or SegFormer, etc., and the present invention is not limited to this.

[0127] like Figure 6 As shown, the wrist key points can be detected by the human key point information detection technology. The human key point detection technology includes but is not limited to OpenPose, AlphaPose or HRNet, etc., and the present invention is not limited to this.

[0128] In step S1000, the forklift driving violation behavior recognition system performs image classification and recognition on the hand contour area, and when the image classification and recognition result is the operation of an electronic device type, it is determined that the second forklift driving violation behavior exists.

[0129] For example, the second forklift driving violation behavior may be a person operating an electronic device next to a forklift moving forward. The forklift driving violation behavior recognition system determines a circumscribed rectangle of the hand contour area, and uses an image classification algorithm to perform image classification recognition on the circumscribed rectangle.

[0130] For example, Figure 6 As shown, when the wrist key point is higher than the elbow key point, and the angle between the elbow key point and the elbow is less than the third preset threshold (such as Figure 6 As shown, that is, when number 4 is higher than number 3 and the angle between the line connecting numbers 2 and 3 and the line connecting numbers 3 and 4 is greater than the third preset threshold and less than the fourth preset threshold, or when number 7 is higher than number 6 and the angle between the line connecting numbers 5 and 6 and the line connecting numbers 6 and 7 is greater than the third preset threshold and less than the fourth preset threshold), the hand contour area is obtained, and the hand contour area is expanded to obtain an expanded frame image, and then the expanded frame image is detected for electronic devices (such as mobile phones). If an electronic device is detected, the expanded frame image is classified to obtain an image classification recognition result. That is, the forklift illegal driving behavior recognition system can determine whether the person has a second forklift illegal driving behavior (that is, the person operates an electronic device next to a forklift moving forward).

[0131] The third preset threshold and the fourth preset threshold can be customized according to user needs. For example, the third preset threshold can be 45° ， The fourth preset threshold may be 135°.

[0132] With such a configuration, the present invention can determine the hand contour area through the human body key point information and the human body contour area information, and based on the image classification and recognition of the hand contour area, it can accurately determine whether there is any illegal forklift driving behavior in which a person operates an electronic device next to a forward-moving forklift.

[0133] According to another aspect of the present invention, a forklift driving violation behavior recognition system is provided, which is used to identify whether there is a driving violation behavior that does not comply with preset forklift operation regulations in the current monitoring screen based on the image information collected by the monitoring device. Exemplarily, the forklift driving violation behavior includes but is not limited to a person standing next to a forklift that is in operation and moving forward, a person operating an electronic device next to a forklift that is moving forward, etc.

[0134] Figure 7 A schematic diagram of a forklift driving violation behavior recognition system according to an embodiment of the present invention is shown. Figure 7 As shown, the forklift illegal driving behavior recognition system 1 may include an image acquisition module 10 , a region recognition module 20 and a behavior determination module 30 .

[0135] In the present invention, a forklift may refer to any operating vehicle that can perform transportation and handling operations such as loading and unloading and transporting of palletized cargo, but the present invention is not limited thereto.

[0136] According to an example embodiment, the image acquisition module 10 acquires image information collected by a monitoring device.

[0137] For example, the monitoring device may be a monocular camera installed at a preset position according to monitoring requirements. For example, the preset position may be the rear area of ​​a forklift. The resolution of the monocular camera may be above 1080p, and may have a fixed top-down angle (e.g., between 15 and 45°), focal length, and installation position.

[0138] The image acquisition module 10 is connected to the monocular camera for communication, and can acquire the RTSP video stream from the monocular camera, and perform stream decoding processing on the RTSP video to obtain target image information including information such as a forklift or a person.

[0139] Exemplarily, the target image information may be color image data or grayscale image data.

[0140] The region identification module 20 sets at least one algorithm to identify the region according to the target image information.

[0141] For example, taking color image data as an example, based on the color image data, the region identification module 20 responds to user instructions and sets one or more algorithms to identify regions according to user requirements.

[0142] The algorithm recognition area can be a user-defined area with monitoring and recognition significance, that is, an alarm will be issued only when there is any illegal driving behavior of a forklift in the algorithm recognition area.

[0143] The area recognition module 20 determines the forklift recognition area, the personnel recognition area, the cargo recognition area, the forklift first feature recognition area and the forklift second feature recognition area of ​​the target forklift in the algorithm recognition area based on the forklift feature code.

[0144] For example, the forklift feature code may be unique number information of a forklift, and one forklift feature code represents only one designated target forklift.

[0145] The region identification module 20 can identify the coordinates of a series of points in the algorithm identification region, and can generate a polygonal region according to the coordinates of these points to form an identification region.

[0146] The forklift identification area may be the area where the target forklift is located; the personnel identification area may be the area where the personnel are located; the cargo identification area may be the area where the cargo on the pallet of the target forklift is located; the forklift first feature identification area may be the area where the front of the target forklift is located; and the forklift second feature identification area may be the area where the rear of the target forklift is located.

[0147] The behavior determination module 30 determines whether the cargo is higher than the human body according to the coordinates of the person identification area and the cargo identification area in the algorithm identification domain.

[0148] The behavior determination module 30 can compare the heights of the person and the cargo based on the coordinates of the key feature points of the person identification area and the coordinates of the key feature points of the cargo identification area. When it is determined that the height of the cargo is higher than the height of the person, it can be further determined that the height of the pallet is higher than the person, that is, it can be characterized that the target forklift is in operation (such as the pallet is raised or the pallet is being raised, etc.), and the person is standing near the bottom of the cargo.

[0149] Optionally, the behavior determination module 30 determines the coordinates of key feature points of the person in the person recognition area.

[0150] The behavior determination module 30 determines the coordinates of the key feature points of the cargo in the cargo identification area.

[0151] The behavior determination module 30 determines the heights of the person and the cargo according to the coordinates of the key feature points of the person and the coordinates of the key feature points of the cargo.

[0152] The behavior determination module 30 determines that the cargo is higher than the human body when the cargo is higher than the human body.

[0153] For example, the coordinates of the key feature point of a person may be the coordinates of the upper left point or the upper right point of the person identification area; the coordinates of the key feature point of a cargo may be the coordinates of the upper left point or the upper right point of the cargo identification area.

[0154] Exemplarily, the behavior determination module 30 determines the coordinates of the key feature points of the personnel and the coordinates of the key feature points of the goods. For example, the coordinates of the key feature points of the personnel may be (x3, y3), where x3 is the horizontal coordinate of the key feature point of the personnel, and y3 is the vertical coordinate of the key feature point of the personnel. The coordinates of the key feature points of the goods may be (x4, y4), where x4 is the horizontal coordinate of the key feature point of the goods, and y4 is the vertical coordinate of the key feature point of the goods.

[0155] The behavior determination module 30 compares y3 and y4, and when y4>y3, the behavior determination module 30 determines that the actual height of the cargo is higher than the actual height of the person, that is, in the actual scene, the target forklift is operating and the person is standing under the cargo.

[0156] With such a configuration, the present invention can accurately determine the positional relationship between personnel and cargo through the coordinates of key feature points in the personnel identification area and the cargo identification area, thereby further determining the working status of the target forklift, so as to facilitate the subsequent identification of illegal forklift driving behavior.

[0157] When the working state of the target forklift is in operation, the behavior determination module 30 determines whether the target forklift is in a moving state according to the multi-frame image information of the forklift identification area.

[0158] For example, the state of the target forklift may include a moving state (or operating state) and a stationary state (or non-operating state). The behavior determination module 30 can comprehensively determine whether the target forklift is in an operating state based on the position information of the multi-frame image information of the forklift identification area.

[0159] Optionally, the behavior determination module 30 determines the coordinates of a first forklift key feature point of a first frame of image information of a forklift recognition area, first width information of the forklift recognition area, and first height information of the forklift recognition area.

[0160] For example, the behavior determination module 30 performs forklift image recognition on the first frame of image information, and can obtain the coordinates (x1, y1) of the first forklift key feature point of the first frame of image information, the first width information w1 of the forklift recognition area, and the first height information h1 of the forklift recognition area. Exemplarily, the first forklift key feature point can be the coordinates of the upper left point of the forklift recognition area.

[0161] The behavior determination module 30 determines the coordinates of the first forklift area center point of the forklift identification area according to the coordinates of the first forklift key feature point, the first width information and the first height information.

[0162] For example, the coordinates O1 of the center point of the first forklift area may be:

[0163]

[0164] The behavior determination module 30 determines the coordinates of the second forklift key feature point of the second frame image information of the forklift recognition area, the second width information of the forklift recognition area, and the second height information of the forklift recognition area.

[0165] For example, the behavior determination module 30 performs forklift tracking based on image recognition technology and performs forklift image recognition on the second frame image information to obtain the coordinates (x2, y2) of the second forklift key feature point of the second frame image information, the second width information w2 of the forklift recognition area, and the second height information h2 of the forklift recognition area. Exemplarily, the second forklift key feature point may be the coordinates of the upper left point of the forklift recognition area.

[0166] The behavior determination module 30 determines the coordinates of the second forklift area center point in the forklift identification area according to the coordinates of the second forklift key feature point, the second width information and the second height information.

[0167] For example, the coordinates O2 of the center point of the second forklift area may be:

[0168]

[0169] The behavior determination module 30 determines a first displacement vector according to the coordinates of the center point of the first forklift area and the coordinates of the center point of the second forklift area.

[0170] For example, the first displacement vector Can be:

[0171]

[0172] When the first displacement vector is greater than the first preset threshold, the behavior determination module 30 determines that the target forklift is in a moving state.

[0173] For example, if The behavior determination module 30 can determine that the position information of the target forklift in the first frame image information is different from that in the second frame image information, and can determine that the target forklift is moving, that is, in a moving state. Exemplarily, the first preset threshold can be customized according to user needs. For example, the first preset threshold can be A1, such as:

[0174]

[0175] With such a configuration, the present invention can determine a first displacement vector through the position information of the target forklift in multiple frames of image information, and based on the comparison between the first displacement vector and the first preset threshold, it can be determined whether the target forklift has moved, thereby accurately determining whether the target forklift is in motion.

[0176] According to an exemplary embodiment, when the target forklift is in motion, the behavior determination module 30 determines the moving direction of the target forklift according to the coordinates of the first feature recognition area and the second feature recognition area of ​​the forklift in the algorithm recognition domain.

[0177] For example, the first feature recognition area of ​​a forklift may be the front area of ​​the target forklift, and the second feature recognition area of ​​the forklift may be the rear area of ​​the target forklift.

[0178] According to an exemplary embodiment, the behavior determination module 30 may determine whether the moving direction of the target forklift is a forward direction or a backward direction according to the coordinates of the front area and the coordinates of the rear area.

[0179] Optionally, the behavior determination module 30 determines the key feature point coordinates of the first feature recognition area of ​​the forklift, the fifth width information of the first feature recognition area of ​​the forklift, and the fifth height information of the first feature recognition area of ​​the forklift.

[0180] For example, the behavior determination module 30 performs forklift image recognition on the forklift first feature recognition area, and can obtain the key feature point coordinates (x5, y5) in the forklift first feature recognition area, the fifth width information w5 of the forklift first feature recognition area, and the fifth height information h5 of the forklift recognition area. Exemplarily, the key feature point in the forklift first feature recognition area can be the coordinates of the upper left point of the forklift first feature recognition area.

[0181] The behavior determination module 30 determines the coordinates of the center point of the first feature recognition area of ​​the forklift according to the coordinates of the key feature points of the first feature recognition area of ​​the forklift, the fifth width information and the fifth height information.

[0182] For example, the center point coordinate F1 of the first feature recognition area of ​​the forklift can be:

[0183]

[0184] The behavior determination module 30 determines the key feature point coordinates of the second feature recognition area of ​​the forklift, the sixth width information of the second feature recognition area of ​​the forklift, and the sixth height information of the second feature recognition area of ​​the forklift.

[0185] For example, the behavior determination module 30 performs forklift image recognition on the forklift second feature recognition area, and can obtain the key feature point coordinates (x6, y6) in the forklift second feature recognition area, the sixth width information w6 of the forklift second feature recognition area, and the sixth height information h6 of the forklift recognition area. Exemplarily, the key feature point in the forklift second feature recognition area can be the coordinates of the upper left point of the forklift second feature recognition area.

[0186] The behavior determination module 30 determines the coordinates of the center point of the second feature recognition area of ​​the forklift according to the coordinates of the key feature points of the second feature recognition area of ​​the forklift, the sixth width information and the sixth height information.

[0187] For example, the center point coordinate B1 of the second feature recognition area of ​​the forklift can be:

[0188]

[0189] The behavior determination module 30 determines the second displacement vector according to the center point coordinates of the first feature recognition area of ​​the forklift and the center point coordinates of the second feature recognition area of ​​the forklift.

[0190] For example, the second displacement vector Can be:

[0191]

[0192] The behavior determination module 30 determines that the target forklift is in the forward direction when the angle between the second displacement vector and the first displacement vector is smaller than a second preset threshold.

[0193] For example, the first displacement vector With the second displacement vector The angle between them is θ, then the calculation formula of θ can be:

[0194]

[0195]

[0196] After the behavior determination module 30 calculates the angle θ, when θ is less than a second preset threshold (such as 90°), the moving direction of the target forklift is determined to be the forward direction; otherwise, when θ is greater than or equal to the second preset threshold (such as 90°), the moving direction of the target forklift is determined to be the backward direction.

[0197] With such a configuration, the present invention can determine the second displacement vector through the key feature point coordinates of the first feature recognition area of ​​the forklift and the key feature point coordinates of the second feature recognition area of ​​the forklift, and based on the angle between the second displacement vector and the first displacement vector, the movement direction of the target forklift can be accurately determined.

[0198] According to an example embodiment, the behavior determination module 30 determines that there is a first forklift illegal driving behavior when the moving direction of the target forklift is a forward direction.

[0199] For example, the first forklift's illegal driving behavior may include a person standing next to a forklift that is in operation and moving forward. When the behavior determination module 30 determines that the target forklift is in operation (e.g., the pallet height is higher than the person's height) and the target forklift is moving forward, it is determined that the first forklift's illegal driving behavior exists and an alarm is issued.

[0200] Through the above example embodiments, the present invention determines the algorithm recognition area through the image information collected by the monitoring device, determines the forklift recognition area, the personnel recognition area, the cargo recognition area, the forklift first feature recognition area and the forklift second feature recognition area of ​​the target forklift based on the forklift feature code, and determines whether the cargo is higher than the human body according to the coordinates of the personnel recognition area and the cargo recognition area in the algorithm recognition domain. In the case where the cargo is higher than the human body, the present invention also determines whether the target forklift is in motion according to the multi-frame image information of the forklift recognition area, and in the case where the target forklift is in motion, determines the direction of motion of the target forklift according to the coordinates of the forklift first feature recognition area and the forklift second feature recognition area in the algorithm recognition domain, and in the case where the target forklift is in motion, determines that there is illegal driving behavior of the first forklift.

[0201] The present invention can determine whether the cargo is higher than the human body through the coordinates corresponding to the personnel identification area and the cargo identification area, can determine the motion state of the target forklift based on the multi-frame image information of the forklift identification area, and can determine the forward direction of the target forklift based on the coordinates corresponding to the forklift first feature identification area and the forklift second feature identification area. The present invention can accurately identify the safety behavior of a person standing next to a forklift that is in operation and in a forward state, and can promptly issue an alarm prompt to reduce the hidden dangers of safe operation.

[0202] Optionally, the behavior determination module 30 performs human body key point detection and human body segmentation on the person recognition area to obtain human body key point information and human body contour area information.

[0203] The behavior determination module 30 determines the hand contour area according to the human body key point information and the human body contour area information.

[0204] For example, the behavior determination module 30 determines the human body region image in the personnel identification region based on the image segmentation recognition technology, and detects the human body key points (such as Figure 6 The 17 human key points shown in the figure) are then used to obtain the human key point coordinates corresponding to the human key points.

[0205] Exemplarily, image segmentation and recognition technologies include but are not limited to SSD, YOLO, RetinaNet or RT-DETR, etc.; human body segmentation technologies include but are not limited to Deeplab, Mask2Former or SegFormer, etc., and the present invention is not limited to this.

[0206] like Figure 6As shown, the wrist key points can be detected by the human key point information detection technology. The human key point detection technology includes but is not limited to OpenPose, AlphaPose or HRNet, etc., and the present invention is not limited to this.

[0207] The behavior determination module 30 performs image classification and recognition on the hand contour area, and when the image classification and recognition result is the operation of an electronic device type, determines that the second forklift illegal driving behavior exists.

[0208] For example, the second forklift driving violation behavior may be that a person operates an electronic device beside a forklift moving forward. The behavior determination module 30 determines a circumscribed rectangle of the hand contour area, and uses an image classification algorithm to perform image classification and recognition on the circumscribed rectangle.

[0209] For example, Figure 6 As shown, when the wrist key point is higher than the elbow key point, and the angle between the elbow key point and the elbow is less than the third preset threshold (such as Figure 6 As shown, that is, when number 4 is higher than number 3 and the angle between the line connecting numbers 2 and 3 and the line connecting numbers 3 and 4 is greater than the third preset threshold and less than the fourth preset threshold, or when number 7 is higher than number 6 and the angle between the line connecting numbers 5 and 6 and the line connecting numbers 6 and 7 is greater than the third preset threshold and less than the fourth preset threshold), the hand contour area is obtained, and the hand contour area is expanded to obtain an expanded frame image, and then the expanded frame image is detected for an electronic device (such as a mobile phone). If an electronic device is detected, the expanded frame image is image classified to obtain an image classification recognition result. That is, the behavior determination module 30 can determine whether the person has a second forklift illegal driving behavior (that is, the person operates an electronic device next to a forklift moving forward).

[0210] The third preset threshold and the fourth preset threshold can be customized according to user needs. For example, the third preset threshold can be 45° ， The fourth preset threshold can be 135°

[0211] With such a configuration, the present invention can determine the hand contour area through the human body key point information and the human body contour area information, and based on the image classification and recognition of the hand contour area, it can accurately determine whether there is any illegal forklift driving behavior in which a person operates an electronic device next to a forward-moving forklift.

[0212] According to another aspect of the present invention, an electronic device is provided, which includes: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors can implement the forklift illegal driving behavior identification method as described above.

[0213] According to another aspect of the present invention, a non-volatile computer-readable storage medium is provided, wherein a computer program is stored on the storage medium, and when the computer program is executed by a processor, the method for identifying illegal driving behavior of a forklift as described above can be implemented.

[0214] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention is described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions of the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for identifying illegal driving behavior of a forklift, characterized in that: include: Obtain target image information collected by monitoring equipment; Setting at least one algorithm recognition area according to the target image information; In the algorithm recognition area, a forklift recognition area, a personnel recognition area, a cargo recognition area, a forklift first feature recognition area and a forklift second feature recognition area of ​​the target forklift are determined based on the forklift feature code; Determine whether the cargo is higher than the human body according to the coordinates of the personnel identification area and the cargo identification area in the algorithm identification domain; In the case where the cargo is higher than the human body, judging whether the target forklift is in motion according to the multi-frame image information of the forklift identification area; When the target forklift is in motion, determining the moving direction of the target forklift according to the coordinates of the first feature recognition area of ​​the forklift and the second feature recognition area of ​​the forklift in the algorithm recognition domain; When the moving direction of the target forklift is the forward direction, it is determined that there is an illegal driving behavior of the first forklift.

2. The method for identifying illegal driving behavior of a forklift according to claim 1, characterized in that: The step of determining whether the cargo is higher than the human body according to the coordinates of the personnel identification area and the cargo identification area in the algorithm identification domain comprises: Determine the coordinates of key feature points of personnel in the personnel identification area; Determine the coordinates of the key characteristic points of the cargo in the cargo identification area; Determine the height of the person and the cargo according to the coordinates of the key feature points of the person and the coordinates of the key feature points of the cargo; In a case where the height of the cargo is higher than the height of the person, it is determined that the cargo is higher than the human body.

3. The method for identifying illegal forklift driving behavior according to claim 1, characterized in that: The step of judging whether the target forklift is in motion according to the multi-frame image information of the forklift identification area comprises: Determine the coordinates of a first forklift key feature point of a first frame of image information of the forklift identification area, first width information of the forklift identification area, and first height information of the forklift identification area; Determine the coordinates of the center point of the first forklift area in the forklift identification area according to the coordinates of the first forklift key feature point, the first width information and the first height information; Determine the coordinates of the second forklift key feature point of the second frame image information of the forklift identification area, the second width information of the forklift identification area, and the second height information of the forklift identification area; Determine the coordinates of the second forklift area center point in the forklift identification area according to the coordinates of the second forklift key feature point, the second width information and the second height information; Determine a first displacement vector according to the coordinates of the center point of the first forklift area and the coordinates of the center point of the second forklift area; When the first displacement vector is greater than a first preset threshold, it is determined that the target forklift is in the moving state.

4. The method for identifying illegal forklift driving behavior according to claim 3 is characterized in that: The step of determining the moving direction of the target forklift according to the coordinates of the first feature recognition area of ​​the forklift and the second feature recognition area of ​​the forklift in the algorithm recognition domain comprises: Determine the key feature point coordinates of the first feature recognition area of ​​the forklift, the fifth width information of the first feature recognition area of ​​the forklift, and the fifth height information of the first feature recognition area of ​​the forklift; Determine the coordinates of the center point of the first feature recognition area of ​​the forklift according to the coordinates of the key feature points of the first feature recognition area of ​​the forklift, the fifth width information and the fifth height information; Determine the key feature point coordinates of the second feature recognition area of ​​the forklift, the sixth width information of the second feature recognition area of ​​the forklift, and the sixth height information of the second feature recognition area of ​​the forklift; Determine the coordinates of the center point of the second feature recognition area of ​​the forklift according to the coordinates of the key feature points of the second feature recognition area of ​​the forklift, the sixth width information and the sixth height information; Determine a second displacement vector according to the coordinates of the center point of the first feature recognition area of ​​the forklift and the coordinates of the center point of the second feature recognition area of ​​the forklift; When the angle between the second displacement vector and the first displacement vector is smaller than a second preset threshold, it is determined that the target forklift is moving in a forward direction.

5. The method for identifying illegal driving behavior of a forklift according to claim 1, characterized in that: The forklift illegal driving behavior identification method also includes: Performing human body key point detection and human body segmentation on the personnel identification area to obtain human body key point information and human body contour area information; Determine a hand contour area according to the human body key point information and the human body contour area information; Image classification and recognition are performed on the hand contour area, and when the image classification and recognition result is the operation of an electronic device, it is determined that there is a second forklift illegal driving behavior.

6. A forklift illegal driving behavior recognition system, characterized in that: include: An image acquisition module acquires target image information collected by the monitoring device; A region recognition module is configured to set at least one algorithm recognition region according to the target image information, wherein a forklift recognition region, a personnel recognition region, a cargo recognition region, a forklift first feature recognition region and a forklift second feature recognition region of the target forklift are determined in the algorithm recognition region based on the forklift feature code; The behavior determination module determines whether the cargo is higher than the human body according to the coordinates of the personnel identification area and the cargo identification area in the algorithm identification domain. If the cargo is higher than the human body, it determines whether the target forklift is in motion according to the multi-frame image information of the forklift identification area. If the target forklift is in motion, it determines the moving direction of the target forklift according to the coordinates of the first feature identification area of ​​the forklift and the second feature identification area of ​​the forklift in the algorithm identification domain. If the moving direction of the target forklift is a forward direction, it determines that there is a first forklift illegal driving behavior.

7. The forklift driving violation behavior recognition system according to claim 6 is characterized in that: The behavior determination module determines the coordinates of key feature points of the personnel in the personnel identification area; The behavior determination module determines the coordinates of the key feature points of the cargo in the cargo identification area; The behavior determination module determines the height of the person and the cargo according to the coordinates of the key feature points of the person and the coordinates of the key feature points of the cargo; The behavior determination module determines that the cargo is higher than the human body when the cargo is higher than the human body.

8. The forklift illegal driving behavior recognition system according to claim 6 is characterized in that: The behavior determination module determines the coordinates of the first forklift key feature point of the first frame image information of the forklift identification area, the first width information of the forklift identification area, and the first height information of the forklift identification area; The behavior determination module determines the coordinates of the center point of the first forklift area in the forklift identification area according to the coordinates of the first forklift key feature point, the first width information and the first height information; The behavior determination module determines the coordinates of the second forklift key feature point of the second frame image information of the forklift identification area, the second width information of the forklift identification area, and the second height information of the forklift identification area; The behavior determination module determines the coordinates of the center point of the second forklift area in the forklift identification area according to the coordinates of the second forklift key feature point, the second width information and the second height information; The behavior determination module determines a first displacement vector according to the coordinates of the center point of the first forklift area and the coordinates of the center point of the second forklift area; The behavior determination module determines that the target forklift is in the moving state when the first displacement vector is greater than a first preset threshold.

9. The forklift illegal driving behavior recognition system according to claim 8 is characterized in that: The behavior determination module determines the key feature point coordinates of the first feature recognition area of ​​the forklift, the fifth width information of the first feature recognition area of ​​the forklift, and the fifth height information of the first feature recognition area of ​​the forklift; The behavior determination module determines the coordinates of the center point of the first feature recognition area of ​​the forklift according to the coordinates of the key feature points of the first feature recognition area of ​​the forklift, the fifth width information and the fifth height information; The behavior determination module determines the key feature point coordinates of the second feature recognition area of ​​the forklift, the sixth width information of the second feature recognition area of ​​the forklift, and the sixth height information of the second feature recognition area of ​​the forklift; The behavior determination module determines the coordinates of the center point of the second feature recognition area of ​​the forklift according to the coordinates of the key feature points of the second feature recognition area of ​​the forklift, the sixth width information and the sixth height information; The behavior determination module determines a second displacement vector according to the coordinates of the center point of the first feature recognition area of ​​the forklift and the coordinates of the center point of the second feature recognition area of ​​the forklift; The behavior determination module determines that the target forklift is in a forward direction when the angle between the second displacement vector and the first displacement vector is smaller than a second preset threshold.

10. The forklift driving violation behavior recognition system according to claim 6, characterized in that: The behavior determination module performs human body key point detection and human body segmentation on the personnel identification area to obtain human body key point information and human body contour area information; The behavior determination module determines the hand contour area according to the human body key point information and the human body contour area information; The behavior determination module performs image classification and recognition on the hand contour area, and determines that there is a second forklift illegal driving behavior when the image classification and recognition result is an operation of an electronic device.

Images