Forklift examination driving path monitoring method

Abstract

The invention relates to the technical field of computer vision and intelligent traffic, in particular to a forklift examination driving path monitoring method, which comprises the following steps: acquiring an examination site real-time video through a video acquisition module, and completing forklift target detection and central point coordinate extraction through a deep learning recognition module; the path matching module executes dynamic matching and point jumping prevention processing on the basis of a pre-stored standard path point file, then the behavior analysis module fuses inter-frame displacement, vector operation and a path number trend to judge the movement state of the forklift, and finally, the result output module generates an examination record video superposed with full-amount monitoring information. According to the invention, automation, precision and traceability of forklift examination path monitoring are realized, the problems of large subjective error of manual invigilation and poor adaptability of complex scenes of a traditional intelligent system are solved, the deployment cost is low, the real-time performance is high, and the method is suitable for various forklift operation skill examination scenes.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent transportation technology, and in particular to a method for monitoring the driving path of a forklift during testing. Background Technology

[0002] In forklift operation skills examinations, monitoring the compliance of forklift path selection is a crucial step in evaluating test scores. Currently, forklift path detection primarily relies on manual proctoring, where proctors must observe the forklift's trajectory in real time to determine if any violations occur. However, manual proctoring has significant drawbacks: firstly, proctors are susceptible to visual fatigue and subjective judgment differences, making it difficult to guarantee the accuracy and fairness of the results; secondly, manual proctoring cannot accurately record and retrospectively analyze the forklift's trajectory, making it difficult to effectively trace the source of any examination disputes.

[0003] While existing intelligent monitoring systems based on video detection can identify vehicle targets, they cannot accurately determine the degree of deviation in driving direction or path, especially in scenarios involving reversing, turning, or multiple intersections, where detection accuracy drops significantly. Therefore, a method for monitoring the driving path during forklift examinations needs to be designed to achieve automated and objective monitoring of the forklift examination process. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a method for monitoring the driving path of a forklift during testing.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a forklift driving path monitoring method, comprising the following steps: S1: Video data of forklifts traveling along the standard path is collected in advance by the video acquisition module. The deep learning recognition module filters the forklift detection results with the highest confidence frame by frame, calculates the coordinates of its center point, and assigns an incremental number according to the time sequence to generate a standard path point file, which is pre-stored in the path matching module. S2: The video acquisition module collects real-time video data from the forklift testing site and transmits it to the deep learning recognition module; S3: The deep learning recognition module performs normalization preprocessing on each frame of the video data, and performs confidence filtering and non-maximum suppression. After processing, the forklift target detection and tracking is completed through the ONNX format YOLOv8 lightweight model, the coordinates of the forklift center point are extracted, and the data are transmitted to the path matching module and behavior analysis module. S4: During the exam, the path matching module loads the pre-stored standard path point file in real time, receives the coordinates of the forklift center point output by the deep learning recognition module, calculates the minimum Euclidean distance d between the center point and each standard path point, and adopts a dynamic matching interval restriction algorithm to match only the path points whose number is greater than or equal to the current maximum pass number, avoiding path point jumps, and transmits the matching results to the behavior analysis module. S5: The behavior analysis module receives the minimum Euclidean distance output by the path matching module. When this distance is greater than the preset distance threshold, it is determined that the forklift has deviated from the path. S6: The behavior analysis module calculates the inter-frame displacement and motion vector based on the continuous frame forklift center point coordinates output by the deep learning recognition module. Combined with the path number sequence trend output by the path matching module, it constructs the motion state recognition logic. S7: The behavior analysis module calculates the driving direction angle based on the continuous position change data of the forklift center point output by the deep learning recognition module, assists in judging the compliance of turning or U-turn operations, and transmits the driving direction, motion status, and deviation warning information to the result output module. S8: The result output module receives the forklift ID output by the deep learning recognition module, the path number, driving direction, motion status and deviation warning information output by the behavior analysis module, and overlays them onto the video frame in real time. It then outputs an exam record video containing the above information and a timestamp for exam review and traceability.

[0006] As a further description of the above technical solution: The video acquisition module uses a monitoring camera with a fixed frame rate of 25-30fps and an image resolution of 1080P. The monitoring camera is fixed at the highest point of the examination site using a tripod, and the lens view covers the entire examination route. The acquired video data is transmitted to the deep learning recognition module in real time via TCP / IP protocol.

[0007] As a further description of the above technical solution: In the process of generating the standard path point file in step S1, the invalid coordinate screening conditions are: forklift detection confidence < 0.85, abrupt change in the coordinates of the center point of adjacent frames > 50 pixels, and an inter-frame time interval > 100ms. Coordinates that meet any of these conditions are eliminated, and the remaining valid coordinates are assigned consecutively increasing numbers in chronological order.

[0008] As a further description of the above technical solution: In step S3, the confidence filtering threshold is set to 0.8~0.9, and the IOU threshold for non-maximum suppression is set to 0.4~0.45. By using dual filtering to remove invalid detection results caused by occlusion and blurred frames, the accuracy of forklift center point coordinate extraction is ensured.

[0009] As a further description of the above technical solution: The calculation of the minimum Euclidean distance d in step S4 is based on two-dimensional plane coordinates, and the calculation formula is as follows: , in, These are the coordinates of the current forklift center point. These are the coordinates of the standard path point.

[0010] As a further description of the above technical solution: The preset distance threshold is set according to the width of the examination venue passage, with a value range of 0.3~0.8. When the passage width is ≤3m, the value is 0.3~0.5, and when the passage width is >3m, the value is 0.5~0.8. The stationary threshold is calculated based on the minimum forklift speed, with a value range of 1~3 pixels per frame. The maximum jump point interval is set according to the standard path point density, with a value range of 2~5 path points. A smaller value is used when the path point density is high, and a larger value is used when the density is low.

[0011] As a further description of the above technical solution: The motion state recognition logic in step S6 includes: S61: If the displacement of the center point of several consecutive frames is less than the preset static threshold, it is determined to be a static state; S62: If the path number decreases continuously and the dot product of the inter-frame motion vector and the overall driving vector is less than zero, then it is determined to be in reverse state; S63: If the path number increases continuously and the dot product of the inter-frame motion vector and the overall driving vector is greater than zero, then it is determined to be in a forward state.

[0012] As a further description of the above technical solution: The calculation method for the driving direction angle in step S7 is as follows: based on the coordinates of the forklift center point of 3 to 5 consecutive frames, the driving trajectory line segment is fitted, and the angle between the line segment and the standard path reference direction is calculated. The angle range is 0° to 360°, which is used to help determine whether the turning angle meets the test requirements.

[0013] As a further description of the above technical solution: The exam record video output in step S8 is in MP4 format, with a frame rate consistent with the capture frame rate of the video acquisition module. The video naming rules are exam date, exam room number, candidate number, and start time. The video file storage path can be customized.

[0014] The present invention has the following beneficial effects: 1. Compared with the prior art, the path matching module of the present invention adopts a dynamic interval restriction and number increment constraint anti-jump strategy, which strictly limits the matching search range to the last matching number and the maximum jump point interval, solves the problem of abnormal jump of path point number caused by instantaneous detection fluctuations in complex scenarios, and greatly improves the accuracy of path driving judgment.

[0015] 2. Compared with existing technologies, the behavior analysis module of this invention constructs multiple verification conditions, integrates the trend of path number sequence, the change of vehicle distance in adjacent frames and vector dot product analysis, and achieves accurate identification of four states: stationary, reversing, moving forward and deviating from the path. This reduces the false judgment rate and significantly improves the reliability of examination monitoring compared with traditional single-condition judgment, ensuring that no violations are missed or misjudged.

[0016] 3. Compared with existing technologies, this invention achieves computer-based proctoring through the cooperation of various modules, reducing the number of proctors and lowering examination costs; it also uses standardized quantitative criteria to determine violations, eliminating subjective differences from human intervention. Furthermore, it can generate videos with monitoring information for real-time exam playback, resolving potential disputes among examinees regarding their scores. Attached Figure Description

[0017] Figure 1 This is a route map of the forklift testing site for this invention; Figure 2 This is a flowchart of the YOLOv8 forklift detection module of the present invention; Figure 3 This is a flowchart of the path point matching and direction determination algorithm of the present invention; Figure 4 This is a flowchart of the reversing and deviation from the path detection logic of the present invention. Detailed Implementation

[0018] Reference Figure 1-4 The present invention provides a method for monitoring the driving path of a forklift during a driving test, comprising the following steps: S1: Video data of the forklift traveling along the standard path is pre-collected by the video acquisition module. The deep learning recognition module filters the forklift detection results with the highest confidence frame by frame, calculates the center point coordinates of the forklift, and assigns them incremental numbers in chronological order to generate a standard path point file, which is pre-stored in the path matching module. During the generation of the standard path point file, the invalid coordinates are filtered as follows: forklift detection confidence < 0.85, a sudden change in the center point coordinates of adjacent frames > 50 pixels, and an inter-frame time interval > 100ms. Coordinates that meet any of these conditions are removed, and the remaining valid coordinates are assigned consecutive incremental numbers in chronological order.

[0019] S2: The video acquisition module collects real-time video data from the forklift testing site and transmits it to the deep learning recognition module.

[0020] S3: The deep learning recognition module performs normalization preprocessing on each frame of the video data, and performs confidence filtering and non-maximum suppression. After processing, the forklift target detection and tracking are completed using the ONNX format YOLOv8 lightweight model. The coordinates of the forklift center point are extracted and transmitted to the path matching module and behavior analysis module. The threshold for confidence filtering is set to 0.8~0.9, and the IOU threshold for non-maximum suppression is set to 0.4~0.45. The double filtering removes invalid detection results caused by occlusion and blurred frames, ensuring the accuracy of the forklift center point coordinate extraction.

[0021] S4: During the examination, the path matching module loads the pre-stored standard path point file in real time, receives the forklift center point coordinates output by the deep learning recognition module, calculates the minimum Euclidean distance d between the center point and each standard path point, and uses a dynamic matching interval restriction algorithm to match only path points with numbers greater than or equal to the current maximum passable number to avoid path point jumps. The matching results are then transmitted to the behavior analysis module. The minimum Euclidean distance d is calculated based on two-dimensional plane coordinates, and the formula is: , in, These are the coordinates of the current forklift center point. These are the coordinates of the standard path point.

[0022] S5: The behavior analysis module receives the minimum Euclidean distance output by the path matching module. When this distance is greater than the preset distance threshold, it is determined that the forklift has deviated from the path. S6: The behavior analysis module calculates the inter-frame displacement and motion vector based on the continuous frame forklift center point coordinates output by the deep learning recognition module. Combined with the path number sequence trend output by the path matching module, it constructs the motion state recognition logic. S61: If the displacement of the center point of several consecutive frames is less than the preset static threshold, it is determined to be a static state; S62: If the path number decreases continuously and the dot product of the inter-frame motion vector and the overall driving vector is less than zero, then it is determined to be in reverse state; S63: If the path number increases continuously and the dot product of the inter-frame motion vector and the overall driving vector is greater than zero, then it is determined to be in a forward state.

[0023] S7: The behavior analysis module calculates the driving direction angle based on the continuous position change data of the forklift center point output by the deep learning recognition module, assists in judging the compliance of turning or U-turn operations, and transmits the driving direction, movement status, and deviation warning information to the result output module. The driving direction angle is calculated as follows: based on the coordinates of the forklift center point of 3 to 5 consecutive frames, the driving trajectory line segment is fitted, and the angle between the line segment and the standard path reference direction is calculated. The angle range is 0° to 360°, which is used to assist in judging whether the turning angle meets the examination requirements.

[0024] S8: The result output module receives the forklift ID output by the deep learning recognition module, the path number, driving direction, motion status, and deviation warning information output by the behavior analysis module, and overlays them onto the video frame in real time. It then outputs an exam record video containing the above information and a timestamp for exam review and traceability. The output exam record video is in MP4 format, with a frame rate consistent with the acquisition frame rate of the video acquisition module. The video naming rules are exam date, exam room number, candidate number, and start time. The video file storage path can be customized.

[0025] The video acquisition module uses a surveillance camera with a fixed frame rate of 25-30fps and an image resolution of 1080P. The surveillance camera is fixed at the highest point of the examination site using a tripod, and the lens view covers the entire examination route. The acquired video data is transmitted to the deep learning recognition module in real time via TCP / IP protocol.

[0026] The preset distance threshold is set according to the width of the examination site passage, with a value range of 0.3~0.8. When the passage width is ≤3m, the value is 0.3~0.5, and when the passage width is >3m, the value is 0.5~0.8. The stationary threshold is calculated based on the minimum travel speed of the forklift, with a value range of 1~3 pixels per frame. The maximum jump point interval is set according to the standard path point density, with a value range of 2~5 path points. A smaller value is used when the path point density is high, and a larger value is used when the density is low. Specific implementation examples:

[0027] (a) Preprocessing stage: generation of standard path point files Video capture module deployment: Fix the monitoring camera to the highest point of the examination site using a tripod, adjust the lens angle to cover the entire examination route, connect it to the computing device via gigabit network cable, start the camera and configure the acquisition parameters; Standard route driving video collection: Arrange skilled operators with forklift operation certificates to drive the forklift along the standard route specified in the examination (including operations such as entering and leaving the warehouse, turning, and stacking) 2-3 times. The video collection module collects driving videos simultaneously, with a total duration of ≥5 minutes. Coordinate extraction and file generation: The deep learning recognition module loads the ONNX format YOLOv8 model, imports the collected standard path driving video, and performs preprocessing on each frame. Perform forklift target detection, remove invalid detection results through confidence filtering (threshold 0.85) and non-maximum suppression (IOU threshold 0.45), and calculate the (x, y) coordinates of the forklift center point in each frame; Abnormal data is removed based on invalid coordinate filtering criteria (confidence level < 0.85, coordinate mutation > 50 pixels, frame interval > 100ms). The remaining valid coordinates are assigned numbers (1, 2, 3, ...) in chronological order, generating a standard path point file in (x, y, number) format, which is then stored in the pre-storage directory of the path matching module.

[0028] (II) Examination Phase: Real-time Monitoring System initialization: Start the monitoring system, select the standard path point file corresponding to the examination site through the graphical interface, set the core parameters (distance threshold 0.5, stationary threshold 2 pixels / frame, maximum jump interval 3, confidence threshold 0.8, non-maximum suppression IOU threshold 0.4), and start the video acquisition module and deep learning recognition module; Video Acquisition and Transmission: The video acquisition module acquires real-time video of the examination venue at 1080P@25fps and transmits it to the computing device via TCP / IP protocol, with a transmission latency of ≤30ms; Forklift target detection and tracking: The deep learning recognition module receives video frames, performs normalization and format conversion preprocessing, inputs them into the YOLOv8 model to complete detection, and extracts the forklift ID and center point coordinates after confidence filtering and non-maximum suppression. It also tracks the forklift target to ensure that the candidate's ID remains unchanged throughout the entire examination process, avoiding ID switching due to brief occlusion. The forklift ID and center point coordinates are synchronously transmitted to the path matching module, behavior analysis module, and result output module. Path matching and anti-jump point handling: The path matching module loads the pre-stored standard path point file, receives the coordinates of the current forklift center point, and calculates the distance between the coordinates and all standard path points according to the Euclidean distance formula. The matching range is limited to "current maximum passing number ~ current maximum passing number + 3" (maximum jump point interval 3). The standard path point with the minimum distance that meets the number requirement is selected as the optimal matching path point, and the "optimal path point number and minimum Euclidean distance" are transmitted to the behavior analysis module. Motion state and deviation from the path determination: The behavior analysis module receives the minimum Euclidean distance. If the distance is greater than 0.5, it is determined to be a deviation from the path, triggering an external speaker's audible and visual alarm, and simultaneously recording the time of the violation. Receive the coordinates of the forklift center point for 5 consecutive frames, calculate the inter-frame displacement, and if the displacement for 5 consecutive frames is less than 2 pixels / frame, it is determined to be in a stationary state. Calculate the motion vectors of the current frame and the previous frame. If the path number decreases continuously and the dot product of the path number and the overall driving vector is less than 0, it is determined to be in reverse mode; if the path number increases continuously and the dot product is greater than 0, it is determined to be in forward mode. The driving trajectory line segment is fitted based on the center point coordinates of three consecutive frames, the angle with the standard reference direction is calculated, and the result is transmitted to the result output module. Monitoring information overlay and video output: The output module overlays the forklift ID, path number, driving direction angle, and motion status onto the forklift detection frame. When the forklift deviates from the path, a red flashing border surrounds the detection frame and overlays the text "deviation from path". At the same time, a timestamp text is overlaid in the upper right corner of the video frame. The video is saved in real time in MP4 format, and an exam record file is generated according to the naming rules and stored in the specified directory. Exam End: After the candidate completes the exam, the invigilator clicks "End Exam" through the system interface. The system automatically saves the complete exam recording video and generates a brief exam report (including candidate number, exam time, number of violations, and type of violation).

[0029] (III) Review Stage: Result Tracing Video playback: Invigilators can use the system's video playback function to select the exam record video of the target candidate, and support operations such as speed playback, progress dragging, and pausing to take screenshots; Dispute Verification: If a candidate has a dispute about their score, the video clips corresponding to the time of the violation will be played back to verify the accuracy of the path matching, the logic of the status judgment, and the result of the deviation distance calculation. The violation will be verified intuitively by combining the superimposed monitoring information. Result Confirmation: After verification, the invigilators confirm the exam results in the system, generate the final score sheet, and archive and save the exam recording video and score sheet.

[0030] This invention constructs a closed-loop forklift examination path monitoring process through the close collaboration of five major modules and the progressive correlation between steps. It realizes full-process automation from data collection to result traceability, effectively solves the defects of existing technologies, and provides a reliable intelligent monitoring solution for forklift operation skills examinations, with broad application prospects.

[0031] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for monitoring the driving path during forklift testing, characterized in that, Includes the following steps: S1: Video data of forklifts traveling along the standard path is collected in advance by the video acquisition module. The deep learning recognition module filters the forklift detection results with the highest confidence frame by frame, calculates the coordinates of its center point, and assigns an incremental number according to the time sequence to generate a standard path point file, which is pre-stored in the path matching module. S2: The video acquisition module collects real-time video data from the forklift testing site and transmits it to the deep learning recognition module; S3: The deep learning recognition module performs normalization preprocessing on each frame of the video data, and performs confidence filtering and non-maximum suppression. After processing, the forklift target detection and tracking is completed through the ONNX format YOLOv8 lightweight model, the coordinates of the forklift center point are extracted, and the data are transmitted to the path matching module and behavior analysis module. S4: During the exam, the path matching module loads the pre-stored standard path point file in real time, receives the coordinates of the forklift center point output by the deep learning recognition module, calculates the minimum Euclidean distance d between the center point and each standard path point, and adopts a dynamic matching interval restriction algorithm to match only the path points whose number is greater than or equal to the current maximum pass number, avoiding path point jumps, and transmits the matching results to the behavior analysis module. S5: The behavior analysis module receives the minimum Euclidean distance output by the path matching module. When this distance is greater than the preset distance threshold, it is determined that the forklift has deviated from the path. S6: The behavior analysis module calculates the inter-frame displacement and motion vector based on the continuous frame forklift center point coordinates output by the deep learning recognition module. Combined with the path number sequence trend output by the path matching module, it constructs the motion state recognition logic. S7: The behavior analysis module calculates the driving direction angle based on the continuous position change data of the forklift center point output by the deep learning recognition module, assists in judging the compliance of turning or U-turn operations, and transmits the driving direction, motion status, and deviation warning information to the result output module. S8: The result output module receives the forklift ID output by the deep learning recognition module, the path number, driving direction, motion status and deviation warning information output by the behavior analysis module, and overlays them onto the video frame in real time. It then outputs an exam record video containing the above information and a timestamp for exam review and traceability.

2. The forklift driving path monitoring method according to claim 1, characterized in that: The video acquisition module uses a monitoring camera with a fixed frame rate of 25-30fps and an image resolution of 1080P. The monitoring camera is fixed at the highest point of the examination site using a tripod, and the lens view covers the entire examination route. The acquired video data is transmitted to the deep learning recognition module in real time via TCP / IP protocol.

3. The forklift driving path monitoring method according to claim 1, characterized in that: In the process of generating the standard path point file in step S1, the invalid coordinate screening conditions are: forklift detection confidence < 0.85, abrupt change in the coordinates of the center point of adjacent frames > 50 pixels, and an inter-frame time interval > 100ms. Coordinates that meet any of these conditions are eliminated, and the remaining valid coordinates are assigned consecutively increasing numbers in chronological order.

4. The forklift driving path monitoring method according to claim 1, characterized in that: In step S3, the confidence filtering threshold is set to 0.8~0.9, and the IOU threshold for non-maximum suppression is set to 0.4~0.

45. By using dual filtering to remove invalid detection results caused by occlusion and blurred frames, the accuracy of forklift center point coordinate extraction is ensured.

5. The forklift driving path monitoring method according to claim 1, characterized in that: The calculation of the minimum Euclidean distance d in step S4 is based on two-dimensional plane coordinates, and the calculation formula is as follows: ， in, These are the coordinates of the current forklift center point. These are the coordinates of the standard path point.

6. The forklift driving path monitoring method according to claim 1, characterized in that: The preset distance threshold is set according to the width of the examination venue passage, with a value range of 0.3~0.

8. When the passage width is ≤3m, the value is 0.3~0.5, and when the passage width is >3m, the value is 0.5~0.

8. The stationary threshold is calculated based on the minimum forklift speed, with a value range of 1~3 pixels per frame. The maximum jump point interval is set according to the standard path point density, with a value range of 2~5 path points. A smaller value is used when the path point density is high, and a larger value is used when the density is low.

7. The forklift driving path monitoring method according to claim 1, characterized in that: The motion state recognition logic in step S6 includes: S61: If the displacement of the center point of several consecutive frames is less than the preset static threshold, it is determined to be a static state; S62: If the path number decreases continuously and the dot product of the inter-frame motion vector and the overall driving vector is less than zero, then it is determined to be in reverse state; S63: If the path number increases continuously and the dot product of the inter-frame motion vector and the overall driving vector is greater than zero, then it is determined to be in a forward state.

8. The forklift driving path monitoring method according to claim 1, characterized in that: The calculation method for the driving direction angle in step S7 is as follows: based on the coordinates of the forklift center point of 3 to 5 consecutive frames, the driving trajectory line segment is fitted, and the angle between the line segment and the standard path reference direction is calculated. The angle range is 0° to 360°, which is used to help determine whether the turning angle meets the test requirements.

9. The forklift driving path monitoring method according to claim 1, characterized in that: The exam record video output in step S8 is in MP4 format, with a frame rate consistent with the capture frame rate of the video acquisition module. The video naming rules are exam date, exam room number, candidate number, and start time. The video file storage path can be customized.

Images