Forklift driver practical operation intelligent examination system based on machine vision

Abstract

The invention provides a forklift driver practical operation intelligent examination system based on machine vision, relates to the technical field of intelligent image processing, and provides real-time operation guidance and a virtual obstacle simulation environment for examinees through an integrated enhancement technology, and different working scenes are identified based on operation behaviors of the examinees. According to the invention, the method effectively improves the training effect and efficiency, improves the objectivity and fairness of evaluation compared with a conventional manual evaluation mode, greatly improves the safety and accuracy of operation, and improves the training efficiency. And powerful support is provided for cultivating high-quality professional skill operators in industries such as logistics, manufacturing and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and more specifically, to a practical intelligent examination system for forklift drivers based on machine vision. Background Art

[0002] In today's society, with the rapid development of industrial automation and intelligence, the demand for professional skilled operators is increasing, especially in logistics, manufacturing and other industries. Forklift drivers are important operating positions, and their skill level directly affects work efficiency and safety.

[0003] At present, the traditional practical examination method for forklift drivers mostly relies on the manual evaluation of examiners. This method is not only inefficient, but also easily affected by subjective factors, resulting in the evaluation results being not objective and fair.

[0004] Although there are some technical solutions for skill training and assessment on the market, these solutions often fail to fully utilize the advantages of machine vision technology in the field of forklift driver practical examinations. For example, traditional methods may be limited to manual playback review after video recording, which is not only time-consuming and labor-intensive, but also unable to provide real-time feedback. In addition, although the existing sensor-based evaluation system can record some operating parameters, it is difficult to fully capture and analyze the complex and dynamic behavior patterns during forklift operation, such as performance indicators such as the stability of cargo handling and the rationality of path planning. Summary of the invention

[0005] In order to solve the above technical problems, the present invention is proposed. The present invention provides a forklift driver practical intelligent examination system based on machine vision.

[0006] According to one aspect of the present invention, a forklift driver practical intelligent examination system based on machine vision is provided, which includes:

[0007] Machine vision building blocks to provide candidates with real-time operational guidance and a simulated environment with virtual obstacles through integrated augmentation technology;

[0008] The test image processing module is used to analyze the examinee's operating behavior and identify different work scenarios based on the examinee's operating behavior;

[0009] The forklift driver practical intelligent feedback module is used to provide scoring reports based on the different operating habits of candidates in different work scenarios. It reproduces the candidates' mistakes through the integrated enhancement technology of the machine vision building module to help candidates improve their skills.

[0010] Wherein, the machine vision building module includes:

[0011] A forklift driver practical operation image capturing unit, used to capture the forklift driver practical operation image information by using quantum dot technology in combination with a hyperspectral imaging device, wherein the forklift driver practical operation image information includes Class A information and Class B information, wherein the Class A information refers to the image information acquired by the camera, and the Class B information refers to the image information not acquired by the camera;

[0012] A hybrid generation interactive unit is used to present virtual obstacles in the examinee's field of vision through holographic projection technology based on the captured forklift driver's practical operation image information;

[0013] The situational awareness monitoring and optimization unit is used to monitor the emotional state and operating performance of candidates during the practical training phase of forklift drivers through the situational awareness monitoring and optimization unit;

[0014] When the system detects that the candidate is nervous or anxious, it will reduce the difficulty or provide encouraging guidance; conversely, it will increase challenging tasks to promote skill improvement; predictive maintenance risk assessment unit, which is used to monitor and conduct health assessments of forklifts and their working environment.

[0015] Furthermore, the test image processing module includes:

[0016] Dynamic behavior analysis unit, used to assess whether the examinee's behavior is in compliance with the specification and identify the work scenario;

[0017] The behavior pattern recognition unit is used to identify and classify the behavior patterns of candidates, which include repetitive actions and sudden operational reactions. It recognizes different types of scenarios by learning historical data and predicts the direction of workflow.

[0018] Furthermore, the forklift driver practical operation intelligent feedback module includes:

[0019] A personalized scoring engine unit, which is used to score candidates based on their performance in different work scenarios;

[0020] The guidance interactive feedback unit is used to provide operation guidance and feedback to candidates during training or examination.

[0021] When the system detects that a candidate has made a mistake, it will give a warning through voice prompts or text instructions; at the same time, it will adjust the guidance content according to the candidate's immediate response to complete the interactive teaching between the system and the candidate.

[0022] Furthermore, the dynamic behavior analysis unit includes:

[0023] The action recording and analysis subunit is used to record the examinee's operating actions, including the posture changes of the hands, arms and the whole body;

[0024] The action angle measurement subunit is used to consider the angle range of the candidate's operation through the preset forklift angle range;

[0025] The time synchronization efficiency analysis subunit is used to evaluate the time required by candidates to complete the task.

[0026] Furthermore, the situational awareness monitoring optimization unit includes a sub-unit under a training situation and a sub-unit under a test situation. The sub-unit under the training situation refers to a situation in which, during the test process, the test rules include N opportunities before the opportunity for a formal test.

[0027] Furthermore, the subunits in the training scenario include:

[0028] Interactive simulation feedback sub-unit, which is used to point out the problem when the examinee makes an error, and reproduce the moment of error through 3D reconstruction technology;

[0029] The learning progress tracking guidance sub-unit is used to predict the direction of progress through the questions where the candidates make mistakes.

[0030] Furthermore, the sub-units under the test scenario include:

[0031] The stress test emergency response assessment subunit is used to evaluate the psychological endurance and decision-making efficiency of candidates in combination with physiological signal monitoring;

[0032] The standard execution scoring subunit is used to automatically adjust the scoring weight according to different types of test questions to ensure the fairness of scoring;

[0033] The post-examination comprehensive analysis sub-unit is used to provide candidates with a score report after the examination.

[0034] Further, the action angle measurement subunit includes a forklift angle range definition calibration module and a candidate operation angle monitoring and evaluation module;

[0035] The forklift angle range definition and calibration module is used to define and calibrate the angle range of the forklift in different operating scenarios, and set the standard angle range of the forklift by combining 3D reconstruction technology.

[0036] Furthermore, the candidate's operation angle monitoring and evaluation module includes a module for monitoring the actual angle of the candidate when performing the operation, and evaluating the actual angle of the candidate when performing the operation with the angle range of the forklift in different operation scenarios.

[0037] According to another aspect of the present invention, a method for intelligent practical examination of forklift drivers based on machine vision is provided, which comprises:

[0038] Provide candidates with real-time operation guidance and a simulated environment of virtual obstacles through integrated augmented technology;

[0039] Analyze the candidates' operating behaviors and identify different work scenarios based on their operating behaviors;

[0040] Based on the different operating habits of candidates in different work scenarios, a scoring report is provided. The integrated enhancement technology of the machine vision building module is used to reproduce the candidates' mistaken moments to help them improve their skills.

[0041] Compared with the existing technology, the present invention realizes real-time and accurate evaluation of examinees' operating behaviors by integrating high-precision sensors, computer vision algorithms and deep learning technologies. The system can not only automatically identify key action angles, but also provide instant feedback, and use big data analysis and personalized scoring engines to generate detailed scoring reports to help examinees quickly correct mistakes. In addition, the interactive simulation feedback subunit reproduces the moment of error through 3D reconstruction technology and augmented reality, provides targeted improvement suggestions, and effectively improves the training effect and efficiency. Compared with traditional manual judgment methods, the present invention not only improves the objectivity and fairness of the evaluation, but also greatly enhances the safety and accuracy of operations, providing strong support for cultivating high-quality professional skilled operators in logistics, manufacturing and other industries. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0043] Figure 1 The present invention is a system block diagram of a machine vision-based intelligent practical examination system for forklift drivers according to an embodiment of the present invention.

[0044] Figure 2 The present invention is a schematic diagram of a forklift driver practical intelligent examination method based on a machine vision-based forklift driver practical intelligent examination system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] Below, the exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described here.

[0046] As mentioned in the above background technology, in today's society, with the rapid development of industrial automation and intelligence, the demand for professional skilled operators is increasing, especially in the logistics, manufacturing and other industries. Forklift drivers are important operating positions, and their skill level directly affects work efficiency and safety.

[0047] At present, the traditional practical examination method for forklift drivers mostly relies on the manual evaluation of examiners. This method is not only inefficient, but also easily affected by subjective factors, resulting in the evaluation results being not objective and fair.

[0048] Although there are some technical solutions for skill training and assessment on the market, these solutions often fail to fully utilize the advantages of machine vision technology in the field of forklift driver practical examinations. For example, traditional methods may be limited to manual playback review after video recording, which is not only time-consuming and labor-intensive, but also unable to provide real-time feedback. In addition, although the existing sensor-based evaluation system can record some operating parameters, it is difficult to fully capture and analyze the complex and dynamic behavior patterns during forklift operation, such as performance indicators such as the stability of cargo handling and the rationality of path planning.

[0049] Figure 1 FIG. 1 is a system block diagram of a forklift driver practical intelligent examination system based on machine vision according to Embodiment 1 of the present invention. Figure 1 As shown, the machine vision building module is used to provide candidates with real-time operation guidance and a simulated environment of virtual obstacles through integrated enhancement technology; the test image processing module is used to analyze the candidate's operating behavior and identify different work scenarios based on the candidate's operating behavior; the forklift driver practical intelligent feedback module is used to provide a scoring report based on the candidate's different operating habits in different work scenarios, and reproduce the candidate's error moments through the integrated enhancement technology of the machine vision building module to help the candidate improve his skills.

[0050] Among them, the machine vision building blocks include:

[0051] The forklift driver's actual operation image capture unit is used to capture the forklift driver's actual operation image information using quantum dot technology in combination with a hyperspectral imaging device. The forklift driver's actual operation image information includes Class A information and Class B information. Class A information refers to image information acquired by a camera, and Class B information refers to image information not acquired by the camera.

[0052] Image information that the camera does not obtain, such as material composition, surface damage and other detailed features, can help to more accurately identify the forklift driver's actual operating status and changes in the surrounding environment, thereby providing more precise data support for subsequent operational guidance.

[0053] The hybrid generation interactive unit is used to present virtual obstacles in the candidate's field of vision through the holographic projection technology based on the captured image information of the forklift driver's practical operation.

[0054] Furthermore, the hybrid generation interaction unit is a key component in the forklift driver practical intelligent examination system based on machine vision, which is used to enhance the candidate's operating experience and training effect. The unit captures the forklift driver's practical image information and uses holographic projection technology to present virtual obstacles in the candidate's field of view, providing the candidate with a highly simulated operating environment.

[0055] First, during the capture process, the system uses high-resolution cameras combined with quantum dot technology and hyperspectral imaging devices to obtain all-round image information of the forklift driver during operation in real time. These cameras can not only capture traditional Class A information, but also capture Class B information of detailed features such as material composition and surface damage. For example, when moving goods, the system can detect whether the goods have tiny cracks or surface wear to ensure a comprehensive understanding of the operating environment. At the same time, devices such as depth sensors and lidar are also used to measure distance and spatial position to provide more accurate three-dimensional data.

[0056] Next, the captured image information is transmitted to the data processing unit for pre-processing using computer vision algorithms.

[0057] Let I be the input original image with noise, poor contrast and color deviation, and define a comprehensive energy function E. The comprehensive energy function E consists of three parts: denoising term, contrast adjustment term and color correction term. The goal is to find an optimized image that minimizes the comprehensive energy function E.

[0058] E(I enhanced )=E denoise (I enhanced )+λ 1 E contrast (I enhanced )+λ 2 E color (I enhanced )

[0059] Among them, E(I enhanced ) represents the total energy function of the processed high-quality image, E denoise represents the denoising term, I enhanced represents the processed high-quality image, λ 1 Represents the weight coefficient of the contrast adjustment term, λ 2 Represents the weight coefficient of the color correction term, E color represents the color correction term, E contrast Indicates the contrast adjustment item.

[0060] Such as denoising, contrast adjustment, color correction, etc. to improve image quality. Subsequently, motion capture technology and biomechanical models further analyze the candidate's movement characteristics, including posture changes of the hands, arms and the whole body, to ensure that every subtle movement is accurately recorded.

[0061] Once the image capture and preliminary processing are completed, the hybrid generation interaction unit comes into play, using holographic projection technology to dynamically generate and project virtual obstacles into the candidate's actual operating environment. Specifically, the holographic projection equipment generates realistic virtual obstacles based on the processed image data and pre-set work scenarios, such as shelves, ground obstacles or other potential dangerous areas in simulated warehouses. These virtual obstacles are seamlessly integrated with the actual environment through spatial mapping algorithms, allowing candidates to interact with virtual elements in a real environment, increasing the realism and challenge of the training.

[0062] In addition, in order to enhance interactivity and user experience, the system also integrates gesture control and voice command functions. Candidates can interact with virtual obstacles through simple gestures or voice commands. For example, when a candidate approaches a virtual obstacle, the system will warn the candidate through voice prompts and provide obstacle avoidance suggestions. If the candidate successfully avoids the obstacle, the system will give positive feedback to encourage him or her to keep moving forward.

[0063] The situational awareness monitoring optimization unit is used to monitor the emotional state and operating performance of candidates during the practical training phase of forklift drivers.

[0064] During the practical training phase for forklift drivers, the situational awareness monitoring and optimization unit uses a series of advanced technical means to monitor the emotional state and operating performance of the candidates in real time, thereby providing personalized guidance and support. First, before the training begins, the system will initialize a series of high-precision sensors and cameras. These devices include not only traditional cameras, but also integrated depth sensors, heart rate monitors, and facial expression recognition cameras to fully capture the candidates' operating behaviors and physiological signals. When the candidates perform actual operations, the data processing unit receives and processes the data from these sensors in real time.

[0065] Specifically, the system uses computer vision algorithms to analyze the examinee's movements and postures, combines motion capture technology and biomechanical models to accurately record the changes in the postures of the hands, arms and the whole body, and compares them with preset standard operation templates to evaluate the examinee's operation accuracy. At the same time, the emotional computing module monitors the examinee's emotional state through physiological signals such as facial expression recognition and heart rate variability. For example, when the system detects that the examinee is anxious or nervous, it will automatically adjust the training difficulty, reduce the complexity of the task or provide encouraging feedback to help the examinee relieve stress; on the contrary, if the examinee is confident and operates smoothly, the system will gradually increase challenging tasks to further improve his or her skill level.

[0066] In addition, the situational awareness monitoring and optimization unit can dynamically adjust the training content according to the candidate's operating performance. For example, during the process of moving goods, if the system finds that the candidate frequently places the goods inaccurately, it will immediately generate targeted practice tasks and display virtual guidance signs in the candidate's field of view through augmented reality technology to guide them to operate correctly. During the entire training process, the system continuously collects and analyzes the candidate's performance data, predicts future progress directions through big data analysis and machine learning algorithms, and develops personalized learning paths and improvement suggestions for each candidate. This highly intelligent situational awareness monitoring and optimization unit not only improves the effectiveness and safety of training, but also provides candidates with a more personalized and interactive learning experience.

[0067] When the system detects that the examinee is nervous or anxious, it will reduce the difficulty or provide encouraging guidance; otherwise, it will increase challenging tasks to promote skill improvement; this personalized and intelligent feedback mechanism helps to improve training results.

[0068] Predictive maintenance risk assessment unit for monitoring and health assessment of forklifts and their working environment.

[0069] With the help of big data analysis and simulation modeling technology, this unit is able to continuously monitor and evaluate the health of forklifts and their working environment, predict potential failures or safety hazards, and once an abnormal situation is detected, the system will immediately notify the candidate and provide corresponding response measures to help master the correct handling methods in emergency situations. In addition, based on historical data, it predicts possible problems in the future and formulates preventive training plans for candidates in advance.

[0070] In an embodiment of the present invention, the test image processing module includes:

[0071] The dynamic behavior analysis unit is used to evaluate whether the candidate's behavior is in compliance with the norms and identify work scenarios.

[0072] The dynamic behavior analysis unit is equipped with motion capture technology and biomechanical models. This unit can accurately analyze the candidate's movement characteristics, such as hand movements, body postures, etc. By comparing with standard operation templates, it can evaluate in real time whether the candidate's behavior complies with the specifications and identify specific work scenarios, such as assembly operations, maintenance inspections, etc.

[0073] The behavioral pattern recognition unit is used to identify and classify the behavioral patterns of candidates, including repetitive actions and sudden operational reactions. It recognizes different types of scenarios by learning historical data and predicts the direction of workflow.

[0074] Furthermore, the dynamic behavior analysis unit is one of the core components of the forklift driver practical intelligent examination system based on machine vision. It accurately analyzes the motion characteristics of candidates through motion capture technology and biomechanical models, and evaluates in real time whether these movements meet the specifications. In addition, the dynamic behavior analysis unit can also identify specific work scenarios, and identify different types of scenarios by learning historical data, and predict the direction of the work process. The specific implementation is as follows:

[0075] First, in terms of motion capture and biomechanical model analysis, the dynamic behavior analysis unit integrates a variety of high-precision sensors, including optical tracking systems, inertial measurement units and depth cameras, to capture all-round data of candidates during the operation process. For example, in the process of moving goods, the system not only records the posture changes of the hands and arms, but also monitors the whole body posture, fork position and cargo status. Using the pre-established biomechanical model, the system maps the captured data to the standard operation template for frame-by-frame comparative analysis. For fork lifting operations, the system compares the angle, speed and force in the actual operation with the preset standard values ​​to evaluate the accuracy of the action. When it is detected that the action does not meet the specifications, such as the fork lifting angle is too large or too small, the system will immediately generate a warning and display correction suggestions through voice prompts or augmented reality to help candidates correct errors in time.

[0076] Secondly, in terms of standard operation templates and scene recognition, the system has built a standard operation template library containing a variety of typical operation tasks. Each template defines the ideal operation parameters for a specific task, such as angle range, speed, force, etc., and uses convolutional neural networks and recursive neural networks to process video streams in real time, extract key action feature points, and match them with the standard template library to identify the current work scene. For example, when the system recognizes that the candidate is performing a maintenance inspection, the system automatically switches to the corresponding evaluation criteria. According to the identified work scene, the system dynamically adjusts the evaluation criteria to ensure the fairness and accuracy of the scoring. For tasks with high precision requirements, the system will strictly evaluate the subtle differences in the actions, thereby providing more accurate feedback.

[0077] Finally, in terms of behavioral pattern recognition and prediction, the system establishes a centralized historical data analysis platform to collect the operational data of each candidate in real time, conduct long-term tracking analysis, and train deep learning models, such as recurrent neural networks and long short-term memory networks, to identify and classify candidates' behavioral patterns, such as repetitive actions and sudden operational reactions. For example, the system can identify the error patterns that frequently occur in certain tasks of candidates and adjust the training content accordingly. Using reinforcement learning algorithms, the system can predict future workflow trends based on historical data and formulate personalized training plans for candidates in advance. For example, if the system predicts that a candidate may encounter difficulties in a future task, it can arrange targeted practice tasks in advance to help candidates better prepare and improve their skills.

[0078] Specifically, the ideal angle range for lifting the fork is 5° to 10°, the speed should be controlled within 0.5 meters per second, and the force needs to be maintained between 30 and 40 Newtons. If it is detected that the action does not meet the specifications, such as the lifting angle of the fork exceeds the ideal range, the system will immediately generate a warning and display correction suggestions through voice prompts or augmented reality to help candidates correct errors in time.

[0079] For assembly operations, the system's error tolerance is ±2 mm, while for maintenance inspections, higher precision is required, with an error tolerance of only ±1 mm. For tasks requiring high precision, the system will strictly evaluate subtle differences in movements to provide more accurate feedback.

[0080] Suppose the system finds that a candidate fails to accurately control the angle of the fork 60% of the time when performing cargo stacking tasks, resulting in cargo deviation. The system will adjust the training plan based on this data and add special exercises for angle control. Using reinforcement learning algorithms, the system can predict the future direction of the workflow based on historical data and formulate personalized training plans for candidates in advance. For example, if the system predicts that a candidate may encounter difficulties in a certain task in the future, it will arrange targeted practice tasks in advance to help candidates better prepare and improve their skills. For example, the system predicts that a candidate may face the challenge of emergency obstacle avoidance in the next week, so it arranges 10 simulated obstacle avoidance training for him in advance to improve his ability to cope with it.

[0081] In an embodiment of the present invention, the forklift driver practical operation intelligent feedback module includes:

[0082] Among them, the personalized scoring engine unit is used to score candidates based on their performance in different work scenarios.

[0083] The guidance interactive feedback unit is used to provide candidates with operational guidance and feedback during training or examination.

[0084] When the system detects that a candidate has made a mistake, it will give a warning through voice prompts or text instructions; at the same time, it will adjust the guidance content according to the candidate's immediate response to complete the interactive teaching between the system and the candidate.

[0085] Furthermore, the personalized scoring engine unit scores candidates based on their specific performance in different work scenarios. The personalized scoring engine unit not only considers operational accuracy, but also multiple dimensions such as efficiency and safety, thereby generating a detailed scoring report for each candidate. For example, in a cargo handling task, the system will record the time taken by the candidate from the starting point to the end point, the rationality of the path planning, and the accuracy of the cargo placement, and dynamically adjust the scoring criteria based on historical data to ensure that the evaluation results are more fair and accurate. When a candidate takes 600 seconds in a handling task, the path planning deviates from the standard path by 5%, and the cargo placement deviates by 5 cm, the system will give a comprehensive score based on these data and point out the specific aspects that need to be improved. In addition, the personalized scoring engine unit can dynamically adjust the scoring criteria based on historical data, so that the results of each training or examination can reflect the candidate's true level and development trend.

[0086] The guidance interactive feedback unit provides real-time operation guidance and feedback to candidates during their training or examination. When the system detects that the candidate has made a mistake, such as the fork lifting angle is beyond the ideal range or the cargo is placed in an inaccurate position, the system will immediately give a warning through voice prompts or text instructions. For example, if the angle deviates from the standard value by more than ±5° when the candidate is performing the fork lifting operation, the system will give a voice prompt, "Please note that the fork lifting angle has exceeded the allowable range. Please adjust it to between 5° and 10°." At the same time, the system will adjust the guidance content according to the candidate's immediate response to complete the interactive teaching between the system and the candidate. For example, when the candidate quickly corrects the error after receiving the first warning, the system will further provide more specific suggestions, such as please maintain a constant speed to avoid sudden acceleration that causes angle changes. This instant feedback mechanism can not only help candidates quickly correct errors, but also enhance their operating skills and adaptability.

[0087] In order to further improve the intelligence level of the system, the guidance interactive feedback unit also integrates emotional computing technology and situational awareness functions. The system can monitor the emotional state of the examinee, such as tension, anxiety and their operational performance, and dynamically adjust the guidance strategy based on this information. For example, when the system detects that the examinee is nervous, it will reduce the difficulty of the task or provide encouraging guidance; otherwise, it will increase challenging tasks to promote skill improvement. In addition, the system will use big data analysis and machine learning algorithms to predict future workflow trends and formulate personalized training plans for examinees in advance. For example, if the system predicts that the examinee may encounter difficulties in a future task, it can arrange targeted practice tasks in advance to help the examinee better prepare and improve his skill level. For example, the system predicts that a examinee may face the challenge of emergency obstacle avoidance in the next week, so it arranges 10 simulated obstacle avoidance training for him in advance to improve his coping ability.

[0088] In an embodiment of the present invention, the dynamic behavior analysis unit includes:

[0089] Among them, the action recording and analysis subunit is used to record the examinee's operating actions, including posture changes of hands, arms and whole body.

[0090] The motion recording and analysis subunit uses motion capture technology, such as optical tracking systems or inertial measurement units, to record the candidate's operating movements in real time, including changes in the posture of the hands, arms and the whole body. It can capture every subtle movement with millimeter-level accuracy and evaluate whether the candidate's movements meet the specifications by comparing with standard operation templates.

[0091] The action angle measurement subunit is used to consider the angle range of the candidate's operation through the preset forklift angle range.

[0092] Integrating advanced computer vision algorithms and 3D reconstruction technology, this unit focuses on measuring the key action angles of candidates when performing specific tasks, such as the angle of forklift fork lifting, the angle of steering wheel rotation, etc. By comparing with the preset ideal angle range, the accuracy of the candidate's operation can be quantified and used as one of the important bases for scoring.

[0093] The time synchronization efficiency analysis subunit is used to evaluate the time required by candidates to complete the task.

[0094] The time-synchronous efficiency analysis subunit combines time series data analysis methods to assess the time required for candidates to complete tasks and compare it with the standard time. In addition, it can also identify which links consume too much time and help candidates understand how to improve work efficiency.

[0095] Furthermore, the action angle measurement subunit integrates computer vision algorithms and 3D reconstruction technology, focusing on measuring the key action angles of candidates when performing specific tasks, such as the lifting angle of the forklift fork, the turning angle of the steering wheel, etc. In order to ensure high precision, the system uses depth cameras and lidar and other devices to obtain three-dimensional data, and uses convolutional neural networks to process the video stream in real time to extract key action feature points. For example, during the fork lifting process, the ideal fork lifting angle should be controlled between 5° and 10°, and the steering wheel rotation angle should be within ±15°. The system quantifies the accuracy of the candidate's operation by comparing it with the preset ideal angle range, and uses it as one of the important bases for scoring. When it is detected that the candidate's operation angle exceeds the ideal range, the system will immediately generate a warning and display correction suggestions through voice prompts or augmented reality to help candidates correct errors in time.

[0096] The ideal range here means that when transporting fragile items, the ideal range of the fork lifting angle may be further narrowed to 6° to 8° to ensure the absolute stability of the goods. When carrying out rapid transportation in open areas, the ideal range of the steering wheel rotation angle may be relaxed to ±20° to accommodate more flexible operating needs. Through this refined management method, the system can more accurately evaluate the candidate's operating behavior and provide personalized feedback and guidance.

[0097] The time synchronization efficiency analysis subunit combines time series data analysis methods to evaluate the time required for candidates to complete tasks and compare it with the standard time. The time synchronization efficiency analysis subunit can not only identify which links consume too much time, but also help candidates understand how to improve work efficiency. For example, in the task of moving goods, the time taken is 600 seconds. When the system finds that the candidate spent 10 seconds more in the fork lifting stage and an additional 10 seconds in the path planning stage, the system will generate a detailed report, point out the specific problems in these links, and make suggestions for improvement. In addition, the time synchronization efficiency analysis subunit can also predict the future direction of the workflow based on historical data, and formulate personalized training plans for candidates in advance. For example, when the system predicts that a candidate may encounter difficulties in a future task, it can arrange targeted practice tasks in advance to help candidates better prepare and improve their skills.

[0098] In an embodiment of the present invention, the context-aware monitoring optimization unit includes a sub-unit under a training context and a sub-unit under an examination context. The sub-unit under a training context refers to a situation in which, during an examination process, the examination rules include a situation in which there are N opportunities before the opportunity for a formal examination.

[0099] In an embodiment of the present invention, the subunits in the training scenario include:

[0100] The interactive simulation feedback sub-unit is used to point out the problem when candidates make mistakes and reproduce the moment of error through 3D reconstruction technology.

[0101] The learning progress tracking guidance sub-unit is used to predict the direction of progress through the questions where the candidates make mistakes.

[0102] Furthermore, the interactive simulation feedback sub-unit uses advanced 3D reconstruction technology and augmented reality display functions. For errors made by candidates during operation, the system will not only point out the problem, but also reproduce the moment of error through 3D reconstruction technology to help candidates intuitively understand and correct the error. For example, in the fork lifting task, when the candidate's operation causes the cargo to tilt slightly or deviate from the expected position, the system will immediately capture this moment and use 3D reconstruction technology to generate a high-precision three-dimensional model to reproduce the entire process of the error. Subsequently, the system will display this model in an augmented reality environment, allowing candidates to observe the specific manifestations of the error from multiple angles, and explain the cause of the error and the correct operation method in detail with voice prompts. When the candidate is lifting the fork, the cargo tilts slightly due to improper lifting angle. The system will display through augmented reality. Please note that the current fork lifting angle is 12°, which exceeds the ideal range of 5° to 10°. Please adjust it to 8° to ensure the stability of the cargo.

[0103] The learning progress tracking and guidance sub-unit analyzes the various errors and their frequencies made by candidates during the training process, predicts their direction of progress, and provides personalized learning plans and suggestions. The learning progress tracking and guidance sub-unit integrates a big data analysis platform and machine learning algorithms, which can collect the operation data of each candidate in real time and conduct long-term tracking and analysis. For example, the system finds that a candidate has unstable fork lifting angle when performing cargo stacking tasks many times, and fails to accurately control the fork angle in 70% of cases, resulting in cargo deviation. The system will automatically generate a detailed report, pointing out the candidate's deficiencies in fork lifting angle control and predicting his / her In order to help candidates improve on challenges that may be encountered in similar tasks in the future, the system will develop a task list of 10 special exercises based on their current level and development trends. Each exercise focuses on the fork lifting operation within a different angle range to gradually improve the candidate's skill level. In addition, the system will dynamically adjust the training content according to the candidate's progress to ensure that the learning goals of each stage are achievable and challenging. For example, when the candidate is able to stably control the fork lifting angle within the ideal range after several training sessions, the system will introduce new variables, such as increasing the weight of the cargo or a complex working environment, to further enhance the candidate's coping ability.

[0104] In the embodiment of the present invention, the sub-units in the examination context include:

[0105] The stress test emergency response assessment subunit is used to combine physiological signal monitoring, such as heart rate variability, to assess the examinee's psychological endurance and decision-making efficiency.

[0106] The standard execution scoring subunit is used to automatically adjust the scoring weights according to different types of test questions to ensure the fairness of scoring.

[0107] The post-examination comprehensive analysis sub-unit is used to provide candidates with a score report after the examination.

[0108] In an embodiment of the present invention, the action angle measurement subunit includes a forklift angle range definition calibration module and a candidate operation angle monitoring and evaluation module.

[0109] The forklift angle range definition and calibration module is used to define and calibrate the angle range of the forklift in different operating scenarios, and set the standard angle range of the forklift by combining 3D reconstruction technology.

[0110] The forklift angle range definition and calibration module is responsible for defining and calibrating the ideal angle range of the forklift in different operating scenarios. By combining 3D modeling and simulation technology, precise angle standards are set for each operating scenario (such as cargo lifting, turning, stacking, etc.).

[0111] Specific implementation:

[0112] 3D modeling and simulation technology: Use computer-aided design tools to create high-precision three-dimensional models of forklifts and their working environments, simulate various operating tasks, and determine the ideal angle range for each task. By collecting a large amount of actual operating data, the system automatically adjusts and optimizes the ideal angle range of various parts of the forklift, such as the fork lifting angle, steering wheel rotation angle, etc., and provides real-time calibration feedback. Before each training or test, the system will automatically run a calibration program to ensure that the angle standard in the current environment is optimal. Any deviation will be recorded and used for subsequent data analysis and model optimization.

[0113] In the embodiment of the present invention, the candidate's operation angle monitoring and evaluation module includes a module for monitoring the actual angle of the candidate when performing the operation, and evaluating the actual angle of the candidate when performing the operation with the angle range of the forklift in different operation scenarios.

[0114] The candidate operation angle monitoring and evaluation module focuses on real-time monitoring of the candidate's actual angle when performing the operation, and compares it with the preset ideal angle range to evaluate the candidate's operation accuracy. It uses computer vision algorithms and sensor fusion technology to ensure the high accuracy of the measurement results.

[0115] Specific implementation:

[0116] Multi-sensor fusion technology: Integrates multiple sensors, such as lidar, inertial measurement technology and cameras, to capture all-round data of the candidate's operation in real time. For example, during the cargo lifting process, the system not only records the vertical angle of the fork, but also monitors its horizontal offset. It uses computer vision algorithms and motion capture, and uses convolutional neural networks and recursive neural networks to process the video stream in real time to extract key motion feature points, such as changes in the posture of the hands, arms and the whole body. By comparing with the standard template, the system can accurately calculate the actual angle of the candidate's operation.

[0117] Once it detects that the candidate's operating angle exceeds the preset ideal range, the system will immediately generate a warning message and display correction suggestions in the candidate's field of view through voice prompts or augmented reality technology. In addition, the system will automatically generate a personalized scoring report based on the candidate's performance, pointing out specific operating errors and providing improvement suggestions.

[0118] That is, in response to the above-mentioned problem of insufficient operational accuracy, in the background technology, a significant technical problem is that it is difficult for candidates to maintain a high degree of operational accuracy during actual operation, especially in key actions such as fork lifting angle and steering wheel rotation angle. In order to solve this problem, the system uses advanced computer vision algorithms and 3D reconstruction technology to accurately measure the candidate's action angle and compare it with the preset ideal range in real time. For example, during the fork lifting process, the ideal fork lifting angle should be controlled between 5° and 10°, and the steering wheel rotation angle should be within ±15°. Otherwise, the system will immediately generate a warning and display correction suggestions through voice prompts or augmented reality to help candidates correct errors in time.

[0119] Then, in response to the problem of lack of personalized learning paths, another common technical problem is that traditional training methods lack personalized learning paths, which makes it difficult to accurately guide the learning progress and development direction of each candidate. For this reason, the system introduces a learning progress tracking and guidance sub-unit. Through the big data analysis platform and machine learning, it collects and analyzes the operation data of each candidate in real time to predict their progress direction. For example, when the system finds that a candidate has unstable fork lifting angle when performing cargo stacking tasks many times, and fails to accurately control the fork angle in 70% of cases, resulting in cargo deviation, the system will automatically generate a detailed report, pointing out the candidate's shortcomings in fork lifting angle control, and predicting the challenges they may encounter in similar tasks in the future. In order to help candidates improve, the system will formulate a task list containing 10 special exercises based on their current level and development trend. Each exercise focuses on fork lifting operations within different angle ranges, and gradually improves the candidate's skill level.

[0120] Finally, in response to the problem of insufficient real-time feedback and interactive teaching, the system integrates an interactive simulation feedback sub-unit, which uses 3D reconstruction technology and augmented reality display functions to reproduce the moment of error in the candidate's operation. For example, in the fork lifting task, when the candidate's operation causes the cargo to tilt slightly or deviate from the expected position, the system will immediately capture this moment and use 3D reconstruction technology to generate a high-precision three-dimensional model to reproduce the entire process of the error. The system will then display this model in an augmented reality environment, allowing the candidate to observe the specific manifestations of the error from multiple angles, and explain the cause of the error and the correct operation method in detail with voice prompts. In addition, the system will adjust the guidance content according to the candidate's immediate response to complete the interactive teaching between the system and the candidate. For example, when the candidate quickly corrects the error after receiving the first warning, the system will further provide more specific suggestions, such as please maintain a constant speed to avoid sudden acceleration that causes angle changes. This instant feedback mechanism not only helps candidates quickly identify and correct errors, but also enhances their operating skills and adaptability.

[0121] Example 2: According to the description of Example 1, the comparison between the present invention and the prior art is shown in Table 1 below:

[0122] Table 1 Comparison between the present invention and the prior art

[0123]

[0124] Table 1

[0125]

[0126]

[0127] It can be seen from the comparison in Table 1 that the technical solution of the present invention shows significant advantages in operation behavior capture, real-time feedback, scoring mechanism, 3D reconstruction and augmented reality feedback, big data analysis and prediction, etc., especially in improving training effects, operation accuracy and safety. These advantages make the present invention more scientific, efficient, objective and fair in the process of professional skills appraisal.

[0128] In summary, the present invention realizes real-time and accurate evaluation of examinees' operating behaviors by integrating high-precision sensors, computer vision algorithms and deep learning technologies. The system can not only automatically identify key action angles, but also provide instant feedback, and use big data analysis and personalized scoring engines to generate detailed scoring reports to help examinees quickly correct errors. In addition, the interactive simulation feedback subunit reproduces the error moment through 3D reconstruction technology and augmented reality, provides targeted improvement suggestions, and effectively improves the training effect and efficiency. Compared with traditional manual judgment methods, the present invention not only improves the objectivity and fairness of the evaluation, but also greatly enhances the safety and accuracy of the operation, providing strong support for cultivating high-quality professional skilled operators in logistics, manufacturing and other industries.

Claims

1. A machine vision-based intelligent test system for forklift drivers, characterized in that: include: Machine vision building blocks to provide candidates with real-time operational guidance and a simulated environment with virtual obstacles through integrated augmentation technology; The test image processing module is used to analyze the examinee's operating behavior and identify different work scenarios based on the examinee's operating behavior; The forklift driver practical intelligent feedback module is used to provide scoring reports based on the different operating habits of candidates in different work scenarios. It reproduces the candidates' mistakes through the integrated enhancement technology of the machine vision building module to help candidates improve their skills. Wherein, the machine vision building module includes: A forklift driver practical operation image capturing unit, used to capture the forklift driver practical operation image information by using quantum dot technology in combination with a hyperspectral imaging device, wherein the forklift driver practical operation image information includes Class A information and Class B information, wherein the Class A information refers to the image information acquired by the camera, and the Class B information refers to the image information not acquired by the camera; A hybrid generation interactive unit is used to present virtual obstacles in the examinee's field of vision through holographic projection technology based on the captured forklift driver's practical operation image information; The situational awareness monitoring and optimization unit is used to monitor the emotional state and operating performance of candidates during the practical training phase of forklift drivers through the situational awareness monitoring and optimization unit; When the system detects that the candidate is nervous or anxious, it will reduce the difficulty or provide encouraging guidance; conversely, it will increase challenging tasks to promote skill improvement; predictive maintenance risk assessment unit, which is used to monitor and conduct health assessments of forklifts and their working environment.

2. The machine vision-based intelligent test system for forklift drivers according to claim 1 is characterized in that: The test image processing module comprises: Dynamic behavior analysis unit, used to assess whether the examinee's behavior is in compliance with the specification and identify the work scenario; The behavior pattern recognition unit is used to identify and classify the behavior patterns of candidates, which include repetitive actions and sudden operational reactions. It recognizes different types of scenarios by learning historical data and predicts the direction of workflow.

3. The machine vision-based forklift driver practical intelligent examination system according to claim 2 is characterized in that: The forklift driver practical operation intelligent feedback module includes: A personalized scoring engine unit, which is used to score candidates based on their performance in different work scenarios; The guidance interactive feedback unit is used to provide operation guidance and feedback to candidates during training or examination. When the system detects that a candidate has made a mistake, it will give a warning through voice prompts or text instructions; at the same time, it will adjust the guidance content according to the candidate's immediate response to complete the interactive teaching between the system and the candidate.

4. The machine vision-based forklift driver practical intelligent examination system according to claim 2 is characterized in that: The dynamic behavior analysis unit comprises: The action recording and analysis subunit is used to record the examinee's operating actions, including the posture changes of the hands, arms and the whole body; The action angle measurement subunit is used to consider the angle range of the candidate's operation through the preset forklift angle range; The time synchronization efficiency analysis subunit is used to evaluate the time required by candidates to complete the task.

5. The machine vision-based intelligent test system for forklift drivers according to claim 1 is characterized in that: The situation-aware monitoring optimization unit includes a subunit under a training situation and a subunit under a test situation. The subunit under the training situation refers to a situation in which, during the test process, the test rules include a situation in which there are N opportunities before the opportunity for a formal test.

6. The machine vision-based intelligent test system for forklift drivers according to claim 5 is characterized in that: The subunits in the training scenario include: Interactive simulation feedback sub-unit, which is used to point out the problem when the examinee makes an error, and reproduce the moment of error through 3D reconstruction technology; The learning progress tracking guidance sub-unit is used to predict the direction of progress through the questions where the candidates make mistakes.

7. The machine vision-based intelligent test system for forklift drivers according to claim 6 is characterized in that: The sub-units of the examination scenario include: The stress test emergency response assessment subunit is used to evaluate the psychological endurance and decision-making efficiency of candidates in combination with physiological signal monitoring; The standard execution scoring subunit is used to automatically adjust the scoring weight according to different types of test questions to ensure the fairness of scoring; The post-examination comprehensive analysis sub-unit is used to provide candidates with a score report after the examination.

8. The machine vision-based intelligent test system for forklift drivers according to claim 7 is characterized in that: include: The action angle measurement subunit includes a forklift angle range definition calibration module and a candidate operation angle monitoring and evaluation module; The forklift angle range definition and calibration module is used to define and calibrate the angle range of the forklift in different operating scenarios, and set the standard angle range of the forklift by combining 3D reconstruction technology.

9. The machine vision-based intelligent test system for forklift drivers according to claim 8 is characterized in that: include: The candidate's operating angle monitoring and evaluation module includes a module for monitoring the actual angle of the candidate when performing the operation, and evaluating the actual angle of the candidate when performing the operation with the angle range of the forklift in different operating scenarios.

10. A method for intelligent practical examination of forklift drivers based on machine vision, characterized in that: include: Provide candidates with real-time operation guidance and a simulated environment of virtual obstacles through integrated augmented technology; Analyze the candidates' operating behaviors and identify different work scenarios based on their operating behaviors; Based on the different operating habits of candidates in different work scenarios, a scoring report is provided. The integrated enhancement technology of the machine vision building module is used to reproduce the candidates' mistaken moments to help them improve their skills.

Images