Intelligent man-hour measurement and quality detection system

Abstract

The invention discloses an intelligent working hour measurement and quality detection system. The system comprises a detection frame, a working platform, a display, a motion capture unit, a visual detection unit, a control unit and a storage unit. By means of the mode, the intelligent working hour measuring and calculating and quality detecting system can remind operators of assembly missing in time, effectively improve the operation quality, reduce the operation pressure and improve the working efficiency and the product yield. And the accuracy and the efficiency of working hour calculation can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial manufacturing, and in particular to an intelligent working time calculation and quality inspection system. Background Art

[0002] In recent years, with the continuous development of science and technology, more and more industries and jobs have been covered and replaced by intelligent systems.

[0003] At present, the calculation and statistics of working hours are still carried out by manual recording or punching in, which still has the following defects: 1. The statistical tools have a limited coverage and a high degree of randomness in sampling, which cannot reflect the standardized operations required by actual personnel, nor can they help operators improve the quality of their operations. 2. It cannot reflect the accurate manual operation time, has low work efficiency, and is prone to errors in investment decisions and efficiency improvements, increasing operating costs. Summary of the invention

[0004] In order to solve the above technical problems, a technical solution adopted by the present invention is: Provided is an intelligent working time calculation and quality inspection system, the steps of which include: (1) Working hours calculation: Calculate the working hours of operators according to the selected calculation mode, where the calculation mode includes a working hours calculation mode based on the area. The specific steps include: (1.1) Divide the entire work platform into several operating areas and set the order of each operating area; (1.2) Acquire or track the operator's hand image and determine whether the hand falls into the first operating area. If so, the time when the acquired hand first appears in the first operating area is used as the starting point of the work timing of the area and the timing begins; (1.3) Continuously take photos at a preset shooting frequency and determine the position of the operator's hand. When the hand is detected to fall into the second operation area for the first time, the shooting time at this time is obtained as the timing end point of the first operation area and the timing start point of the second operation area, so as to start timing the working time of the second operation area, and calculate the real-time working time of the first operation area according to the timing start point and timing end point of the first operation area; (1.4) Repeat step (1.3) in this way until the working time of all operating areas is counted. When it is detected that there is no product in the operating area, it is determined that the product has been removed and the timing of this cycle ends. (1.5) Calculate the sum of the real-time working time of all operating areas of the operator in one cycle, which is the working hours of the operator in that cycle; (2) Product testing (2.1) Determine whether to conduct product quality testing; (2.2) Obtain a real-time product image, compare the real-time product image with the preset target good product image, and display the comparison result on the display: If the product structure in the real-time product image is the same as the product structure in the target image, the inspection is qualified and the qualified product automatically flows to the next process; If the product structure in the real-time product image is different from the product structure in the target image, a modification reminder will be given and the problem location will be marked on the display.

[0005] In a preferred embodiment of the present invention, the calculation mode also includes a time-based working hours calculation mode, and the steps include: Step 1: Obtain the operator's hand image at a preset shooting frequency to determine whether the hand is in the operating area specified by the work platform; when the operator's hand appears in the operating area and the time the hand remains in the operating area exceeds the preset time threshold, obtain the shooting time when the operator's hand first appears in the operating area, and use the shooting time as the starting working time point of the operator to accumulate working hours; Step 2: Continue to shoot the specified operation area and determine whether there is a hand in the area. If there is a hand, the working time continues to accumulate. If there is no hand, the time accumulation is suspended until the next time an image of a hand appears in the specified operation area is obtained, and then the time accumulation calculation is restarted; Step 3: During the preset period, the total accumulated working time of each employee is calculated as the working hours of the employee.

[0006] In a preferred embodiment of the present invention, in step (1), when the operator clocks in, inputs a work start instruction, or detects that there is a product in the operating area, the hand image is acquired.

[0007] In a preferred embodiment of the present invention, in step (1), when it is detected that the operator's hand falls into the non-operating area, the timing of the real-time working time of the current operating area is suspended until an image of the hand appearing in the current or next operating area is obtained again, and then the timing is restarted.

[0008] In a preferred embodiment of the present invention, in step (1), when it is detected that the operation area where the hand appears again is not the current operation area, nor is it the operation area adjacent to the current operation area, timing is still not performed.

[0009] In a preferred embodiment of the present invention, the time when the hand is first detected in the non-operating area is obtained as the starting point of the wasted time timing, and the time when the hand returns to the current or next operating area is obtained as the end point of the wasted time timing, and the real-time wasted time of the operator is calculated based on the wasted time timing starting point and the wasted time timing end point; the real-time wasted time is compared with the preset wasted time threshold, and if the real-time wasted time exceeds the preset wasted time threshold, an alarm is issued, and a hand motion capture video from the wasted time timing starting point to the wasted time timing end point is automatically intercepted and saved.

[0010] In a preferred embodiment of the present invention, in step (1.3), the preset working time thresholds of each operating area are obtained, and the real-time working time of each operating area is compared with the corresponding working time threshold; if the real-time working time exceeds the working time threshold, an alarm reminder is issued, and at the same time, the operation video of the operator is automatically captured and saved according to the timing start and end points of the operating area.

[0011] In a preferred embodiment of the present invention, the waiting time is calculated and stored based on the time when the product is removed and the time when the next product is placed in the operating area.

[0012] In a preferred embodiment of the present invention, product quality inspection is performed after the working hours calculation of each operation area is completed, or product inspection is performed after the working hours calculation of a cycle is completed.

[0013] In a preferred embodiment of the present invention, in step (2), the contents of comparison include the size, shape, color, position and direction of the newly assembled parts.

[0014] The beneficial effects of the present invention are: not only can the operator be reminded of assembly omissions in a timely manner, the work quality can be effectively improved, the work pressure can be reduced, the work efficiency and product yield can be improved, and the accuracy and efficiency of the working time calculation can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work, among which: Figure 1 It is a flow chart of a preferred embodiment of an intelligent working time calculation and quality inspection system of the present invention; Figure 2 It is a structural schematic diagram of a preferred embodiment of an intelligent working time measurement and quality inspection system of the present invention. DETAILED DESCRIPTION

[0016] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] See also Figure 1-2 , the embodiment of the present invention includes: An intelligent working time calculation and quality inspection system comprises a detection frame 1, a working platform 2, a display unit 3, a motion capture unit 4, a visual inspection unit 5, a control unit 6 and a storage unit.

[0018] A work platform for operators to operate is provided on the operating table, and a display unit is provided on the inspection frame, which can be used to play operation guidance videos, real-time motion capture images, real-time working hours, product inspection results, etc. for operators to learn. The motion capture unit is installed on the operating table around and / or on the top of the working platform to capture the operator's hand movements and send the acquired image information to the control unit. The camera coverage of the motion capture unit focuses on capturing operating movements and covers the entire area of ​​the working platform. The visual inspection unit is provided above the working platform to obtain images of the assembled products.

[0019] Further preferably, in the display unit, in order to display preset information to the user in real time, QT5 (Python binding PyQt5) can be used to design the GUI. The GUI will include a video display window, a time display label and a table control. The video display window is used to display the video frame with the calibration area and hand tracking in real time; the time display label will be dynamically updated to show the time when the current finger is in a specific area; the table control will record the time when each finger enters and leaves the calibration area, as well as the total stay time.

[0020] Further preferably, the visual inspection device is movably arranged on the inspection frame to adjust the height, front and rear position and left and right position of the visual inspection device, so that the position and focus of the visual inspection device can be adjusted in real time according to the operating habits of the operator or the change of the product, so that the visual inspection device can be aligned with the product to improve the accuracy of the inspection. Among them, the visual inspection device can use CCD.

[0021] Further preferably, the motion capture unit is movably arranged on the detection frame, so that the position of the motion capture unit can be adjusted in real time according to the height of the operator and the product, so that the motion capture unit can obtain the operator's hand movements more clearly and accurately. Among them, the motion capture unit can use a high-speed camera.

[0022] Further preferably, rollers are provided at the bottom of the detection frame, so that the detection frame can be moved and transported according to more practical usage requirements.

[0023] The storage unit (or server) is used to store operation guidance videos, target hand movement operation images (or operation demonstration pictures) at preset time thresholds or preset operation stages, target (qualified) product structure images, hand operation comparison results, product inspection comparison results, operator information, working hour information of each operator, etc.

[0024] Further preferably, the storage unit saves the recorded data to a file (such as CSV or Excel format) for further analysis; Matplotib can also be used to visualize the data in the storage unit into charts, such as bar charts, line charts, etc., to intuitively display the dwell time of the finger in different areas or other relevant statistical data.

[0025] The control unit communicates and exchanges data with the motion capture unit, the visual detection unit and the storage unit respectively. Among them, the camera (motion capture unit) and the CCD camera (visual detection device) can be connected to the control unit (such as a computer, etc.) through a USB interface.

[0026] The control unit includes a motion capture control unit, a motion detection and comparison unit, a visual detection control unit, a product detection and comparison unit, and a working time control unit.

[0027] Motion capture control unit: used to start, stop, adjust and set parameters of the motion capture unit.

[0028] Motion detection and comparison unit: used to compare the received real-time hand motion image of the operator with the preset target hand motion operation image to determine whether the operator's motion meets the requirements.

[0029] Visual inspection control unit: used to control the start and stop, adjustment and parameter setting of the visual inspection unit.

[0030] Product inspection and comparison unit: used to compare the current product image assembled or processed by the operator with the preset target good product image to determine whether the product meets the requirements.

[0031] Working time control unit: used to judge and calculate the working hours of the operator based on the received time or the operator's hand image.

[0032] Further preferably, the control unit can be connected to the other three units via wired or wireless means to perform real-time data exchange.

[0033] A control method for an intelligent working time measurement and quality inspection system, the steps of which include: (1) Working hours calculation: Calculate and count employees’ working hours according to the selected calculation mode. Currently, the main working hours calculation modes used are time-based working hours calculation mode or area-based working hours calculation mode.

[0034] Further preferably, when the operator clocks in, inputs a work start instruction, or the visual inspection control unit detects that there is a product in the operation area, the hand image is acquired.

[0035] Further preferably, the VideoCapture class of OpenCV can be used to easily capture the video stream from the camera; after the control unit reads the video frame, the image format is converted (such as grayscale or color) through OpenCV, and then preprocessing (such as filtering, edge detection) is performed. Finally, specific areas based on fixed coordinates are calibrated in the frame, such as the area and coordinates of the hand and product in the image.

[0036] Further preferably, in order to track the movement of the fingers, the motion capture unit includes a hand recognition module integrated with the Mediapipe library, which can detect and track key points of the hand (such as fingertips, wrists, etc.) in real time. By comparing the coordinates of the fingers (especially the fingertips) with the coordinates of the calibration area, it can be determined when the fingers enter and leave the operation area.

[0037] Further preferably, the datetime or time module of Python is used to record the timestamp of each motion capture image, so as to calculate the time that the finger stays in each operation area, that is, the time of the decomposed action.

[0038] Time-based working hours calculation mode Step 1: The motion capture control unit captures the image of the operator's hand and determines whether the hand is in the operating area specified by the work platform; if it is determined that the operator's hand appears in the operating area, the hand image is acquired again at a preset shooting frequency. When the time the hand remains in the operating area exceeds the preset time threshold, the shooting time corresponding to the image of the operator's hand appearing in the operating area for the first time is acquired, and the shooting time is used as the starting working time point of the operator to accumulate working hours.

[0039] Step 2: The motion capture control unit continuously shoots the specified operation area at a preset shooting frequency and determines whether there is a hand in the area. If there is a hand, the time continues to accumulate. If there is no hand, the time accumulation is paused until the next time an image of a hand appearing in the specified operation area is obtained, and then the time accumulation calculation is restarted; Step 3: The total accumulated time of each employee on that day is the employee's working hours for that day, and the working hours are sent and stored in the storage unit.

[0040] Time calculation mode based on area Step 1: Divide the entire work platform into several operation areas and set the order of each operation area. For example, it can be divided into A, B, C, D, E, F, G areas, and the work order is A→B→C→D→E→F→G.

[0041] Step 2: The motion capture control unit obtains or tracks the operator's hand image, and determines whether the hand falls in the first operation area based on the coordinates of the hand in the image. If so, the time when the operator's hand first appears in the operation area is obtained as the starting point of the work timing and the timing begins.

[0042] Step 3: The motion capture control unit continuously shoots at a preset shooting frequency and determines the position of the operator's hand. When it is detected that the operator's hand falls into the second operation area for the first time, the shooting time of the image is obtained as the timing end point of the first operation area and the timing start point of the second operation area, so as to start timing the working time of the second operation area, and calculate the real-time working time of the first operation area based on the timing start point and timing end point of the first operation area.

[0043] Further preferably, when it is detected that the operator's hand falls into an area outside the operating area (non-operating area), the timing of the real-time working time of the current operating area is paused until an image of the hand appears in the current or next operating area again, and then the timing is restarted.

[0044] If the operation area where the hand appears again is neither the current operation area nor the next area of ​​the current operation area, that is, the order of the two operation areas is not continuous, then timing is still not performed.

[0045] Further preferably, the time when the hand is first detected in the non-operating area is obtained as the starting point of the wasted time timing, and the time when the hand returns to the current or next operating area is obtained as the end point of the wasted time timing, and the real-time wasted time of the operator is calculated based on the wasted time timing starting point and the wasted time timing end point; the real-time wasted time is compared with the preset wasted time threshold, if the real-time wasted time exceeds the preset wasted time threshold, an alarm is issued, and a hand motion capture video from the wasted time timing starting point to the wasted time timing end point is automatically captured, and the video is saved in a storage unit.

[0046] Step 4: Obtain the preset working time thresholds for each operating area, and compare the real-time working time of each operating area with the corresponding working time threshold: If the real-time working time is less than or equal to the working time threshold, the working information of the operator is sent to the storage unit or saved in the operator working information statistics table; wherein the working information includes operator information, working time information, etc.; If the real-time working time exceeds the working time threshold, an alarm will be issued. At the same time, the operator's operation (motion capture) video will be automatically captured based on the timing start and end points of the operation area and saved in the storage unit.

[0047] Step 5: Repeat steps 2 to 4 in this way until the timing of the working time of all operating areas is completed; when the visual inspection control unit detects that there is no product in the operating area of ​​the working platform, it is determined that the product is removed and the timing of this cycle is ended to achieve continuous production operations.

[0048] Step 6: According to the preset working time statistical cycle, calculate the total working time of each operator in all operation areas. This total is the working time of the operator in the unit working time statistical cycle.

[0049] Further preferably, the waiting time is calculated according to the time when the product is removed and the time when the next product is placed in the operation area, and sent to the storage unit.

[0050] (2) Motion detection The motion capture control unit captures the operator's hand motions at a preset frequency and sends the captured images to the control unit; The real-time hand movement image is compared with the preset target hand movement operation image to determine whether the operator's action is correct; if incorrect, a reminder will be given.

[0051] (3) Product testing Step 1: Determine whether to conduct product quality testing; Further preferably, the product (partial) inspection may be performed after the working time calculation of each operation area is completed, or the product inspection may be performed after the working time calculation of a cycle is completed.

[0052] Further preferably, the triggering conditions for product quality detection may also include: 1. After setting the material number, you can conduct product quality inspection; 2. Start the detection when it is detected that the operator's hands are out of the operating area for a time greater than the preset time threshold; 3. Start the test according to the preset product test start time.

[0053] Step 2: Acquire a real-time product image, compare it with a preset target good product image, and display the comparison result on the monitor; the main content of the comparison is the size, shape, color, position and direction of the newly assembled parts.

[0054] If the product structure in the real-time product image is the same as the product structure in the target image, the inspection is qualified and the qualified product automatically flows to the next process; If the product structure in the real-time product image is different from the product structure in the target image, a reminder will be given, and the problem location will be marked on the display, and the real-time product image will be automatically saved.

[0055] The beneficial effects of the intelligent working time calculation and quality inspection system of the present invention are: 1. Continuous time capture of employees can be achieved through cyclic operation, multiple recording and weighted averaging, thereby improving the accuracy of working time calculation, providing guidance and decision-making basis for investment and cost control, and meeting customer order requirements; 2. It can enhance and improve the working methods, and provide data analysis support for line balance, which is conducive to technical improvement; 3. By comparing preset images, operators can be reminded of omissions in the assembly process, effectively improving work quality, reducing work pressure, and improving work efficiency and product yield.

[0056] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An intelligent working time calculation and quality inspection system, characterized in that the steps include: (1) Working hours calculation: Calculate the working hours of operators according to the selected calculation mode, where the calculation mode includes a working hours calculation mode based on the area. The specific steps include: (1.1) Divide the entire work platform into several operating areas and set the order of each operating area; (1.2) Acquire or track the operator's hand image and determine whether the hand falls into the first operating area. If so, the time when the acquired hand first appears in the first operating area is used as the starting point of the work timing of the area and the timing begins; (1.3) Continuously take photos at a preset shooting frequency and determine the position of the operator's hand. When the hand is detected to fall into the second operation area for the first time, the shooting time at this time is obtained as the timing end point of the first operation area and the timing start point of the second operation area, so as to start timing the working time of the second operation area, and calculate the real-time working time of the first operation area according to the timing start point and timing end point of the first operation area; (1.4) Repeat step (1.3) in this way until the working time of all operating areas is counted. When it is detected that there is no product in the operating area, it is determined that the product has been removed and the timing of this cycle ends. (1.5) Calculate the sum of the real-time working time of all operating areas of the operator in one cycle, which is the working hours of the operator in that cycle; (2) Product testing (2.1) Determine whether to conduct product quality testing; (2.2) Obtain a real-time product image, compare the real-time product image with the preset target good product image, and display the comparison result on the display: If the product structure in the real-time product image is the same as the product structure in the target image, the inspection is qualified and the qualified product automatically flows to the next process; If the product structure in the real-time product image is different from the product structure in the target image, a modification reminder will be given and the problem location will be marked on the display.

2. The intelligent working time calculation and quality inspection system according to claim 1 is characterized in that: The calculation mode also includes a time-based working hours calculation mode, and the steps include: Step 1: Obtain the operator's hand image at a preset shooting frequency to determine whether the hand is in the operating area specified by the work platform; when the operator's hand appears in the operating area and the time the hand remains in the operating area exceeds the preset time threshold, obtain the shooting time when the operator's hand first appears in the operating area, and use the shooting time as the starting working time point of the operator to accumulate working hours; Step 2: Continue to shoot the specified operation area and determine whether there is a hand in the area. If there is a hand, the working time continues to accumulate. If there is no hand, the time accumulation is suspended until the next time an image of a hand appears in the specified operation area is obtained, and then the time accumulation calculation is restarted; Step 3: During the preset period, the total accumulated working time of each employee is calculated as the working hours of the employee.

3. The intelligent working time calculation and quality inspection system according to claim 1 is characterized in that: In step (1), when the operator clocks in, inputs a work start instruction, or detects that there is a product in the operation area, the hand image is acquired.

4. The intelligent working time calculation and quality inspection system according to claim 1 is characterized in that: In step (1), when it is detected that the operator's hand falls into the non-operating area, the timing of the real-time working time of the current operating area is suspended until an image of the hand appears in the current or next operating area again, and then the timing is restarted.

5. The intelligent working time calculation and quality inspection system according to claim 1 is characterized in that: In step (1), when it is detected that the operation area where the hand appears again is not the current operation area, nor is it an operation area adjacent to the current operation area, timing is still not performed.

6. The intelligent working time calculation and quality inspection system according to claim 1 is characterized in that: The time when the hand is first detected in the non-operating area is taken as the starting point of the wasted time timing, and the time when the hand returns to the current or next operating area is taken as the end point of the wasted time timing. The real-time wasted time of the operator is calculated based on the wasted time timing starting point and the wasted time timing end point; the real-time wasted time is compared with the preset wasted time threshold. If the real-time wasted time exceeds the preset wasted time threshold, an alarm is issued, and the hand motion capture video from the wasted time timing starting point to the wasted time timing end point is automatically captured and saved.

7. The intelligent working time calculation and quality inspection system according to claim 1 is characterized in that: In step (1.3), the preset working time thresholds of each operating area are obtained, and the real-time working time of each operating area is compared with the corresponding working time threshold; if the real-time working time exceeds the working time threshold, an alarm reminder is issued, and at the same time, the operation video of the operator is automatically captured and saved according to the timing start and end points of the operating area.

8. The intelligent working time calculation and quality inspection system according to claim 1 is characterized in that: Based on the time it takes for a product to be removed and the time it takes for the next product to be placed in the operating area, the waiting time is calculated and saved.

9. The intelligent working time calculation and quality inspection system according to claim 1, characterized in that: Product quality inspection is carried out after the working hours calculation of each operation area is completed, or product inspection is carried out after the working hours calculation of a cycle is completed.

10. The intelligent working time calculation and quality inspection system according to claim 1, characterized in that: In step (2), the comparison includes the size, shape, color, position and orientation of the newly assembled parts.

Images