A real-time monitoring system and method for production line safety

Abstract

The present invention provides a real-time monitoring system and method for production line safety, the system comprising: an AI processing unit, for performing target detection on a monitoring image to be tested of the production line, and outputting a target detection result; and when the target detection result indicates that a target object exists in a production line area, the target detection result is sent to a shielding area determination unit; the shielding area determination unit is used to determine the degree of overlap between the target detection result and a preset shielding area corresponding to the monitoring image to be tested; and based on the degree of overlap, determine whether the target object is in a safe area, and generate a monitoring result; this embodiment combines target detection with safe area judgment, and can accurately distinguish between the production line area and the safe area in a workshop environment, thereby reducing the false detection rate, and realizing accurate monitoring of production line personnel on the production line and accurate detection of their residence status.

Description

Technical Field

[0001] The present invention belongs to the technical field of production line safety detection, and in particular relates to a real-time monitoring system and method for production line safety. Background Art

[0002] Traditional target detection method is a commonly used technology for detecting and locating target objects. In terms of safety monitoring in industrial production workshops, the existing technology mainly adopts two solutions.

[0003] The first solution is manual inspection, which relies on staff to manually inspect and monitor the target. In this solution, staff are responsible for patrolling the production workshop to observe and determine whether there are people stationed in the production line. However, manual inspection has some limitations. First, due to factors such as human subjective judgment and visual fatigue, missed inspections or false inspections may occur. Second, due to human resource limitations, 24-hour continuous monitoring cannot be achieved, thus failing to meet the needs of real-time monitoring. Therefore, manual inspection has a low safety factor and low efficiency.

[0004] The second solution is the application of traditional target detection methods. This method involves the use of traditional image processing and machine learning techniques for target detection. In traditional target detection methods, commonly used algorithms include feature extraction-based methods (such as Haar features, HOG features, etc.) and machine learning-based methods (such as SVM, AdaBoost, etc.). However, traditional target detection methods have some disadvantages. First, due to certain limitations in the extraction and matching of target features by traditional methods, missed detection or false detection of targets may occur. Secondly, due to the complexity of the algorithm and the limitation of computing resources, the processing speed of traditional methods is slow and cannot meet the requirements of real-time monitoring.

[0005] The existing technical solutions have the following objective disadvantages:

[0006] (1) Low target detection accuracy: Traditional target detection algorithms have certain limitations in the detection accuracy of production line personnel. This is mainly because the target detection methods used by traditional algorithms may have certain errors in target positioning and classification. For example, methods based on sliding windows or region proposals may not accurately locate the target bounding box, resulting in reduced target detection accuracy. In addition, traditional algorithms have poor adaptability to factors such as the specific shape, posture, and lighting changes of production line personnel, which further affects the accuracy of target detection.

[0007] (2) High computational complexity: Some existing technologies require multiple predictions or complex computational operations during the target detection process, resulting in high computational complexity. For example, the sliding window method requires target detection at different locations in the image, which increases the amount of computation. At the same time, some traditional algorithms may use larger feature map sizes or complex network structures, resulting in increased consumption of computing resources. This reduces the operating efficiency of the system and may not meet the requirements of real-time monitoring.

[0008] (3) Poor real-time performance: Traditional target detection algorithms may not be able to monitor the presence of personnel in the production line in real time due to high computational complexity or cumbersome processing steps. This means that in actual applications, the system may not be able to detect the presence of personnel in the production line in a timely manner and trigger the corresponding alarm mechanism. The disadvantage of poor real-time performance may lead to delays in safety issues and the inability to take timely measures.

[0009] (4) High false detection rate in irrelevant areas: Existing technologies fail to effectively shield windows and door areas within the workshop during target detection, resulting in possible false detection problems. Since the windows and doors of the workshop usually display the external environment or people entering and leaving the workshop, it is difficult to accurately distinguish these non-target areas from the actual production line personnel. In actual applications, these irrelevant areas may be mistakenly identified as the residence of production line personnel, resulting in an increased false detection rate, which in turn wastes system resources and processing time. Such false detection problems not only reduce the reliability and efficiency of the monitoring system, but may also cause false alarms in irrelevant areas, interfering with the normal safety monitoring process of the workshop environment. Summary of the invention

[0010] In response to the above-mentioned problems existing in the prior art, an embodiment of the present invention provides a real-time monitoring system and method for production line safety; the monitoring system can accurately distinguish between production line areas and safety areas, reduce the false detection rate, and thus realize precise monitoring of production line personnel and accurate detection of their residence status.

[0011] According to the first aspect of this embodiment, the present invention provides a real-time monitoring system for production line safety, including: an AI processing unit, used to perform target detection on the monitoring image to be tested of the production line, and output a target detection result; and when the target detection result indicates that there is a target object in the production line area, the target detection result is sent to a shielding area determination unit; the shielding area determination unit is used to determine the degree of overlap between the target detection result and a preset shielding area corresponding to the monitoring image to be tested; and based on the degree of overlap, determine whether the target object is in a safe area, and generate a monitoring result; wherein the preset shielding area is used to indicate a safe area outside the glass windows located in the production line area.

[0012] Optionally, the system further includes: a camera acquisition unit, which is used to acquire the monitoring image to be tested in the production line area and send it to the AI ​​processing unit; and generate an alarm prompt message after receiving the monitoring result that the target object is not in the preset shielding area.

[0013] Optionally, the system also includes: a shared processing module and a camera acquisition module; the camera acquisition module includes several camera acquisition units and an image storage unit; the shared processing module includes several shared processing units; each of the shared processing units includes the AI ​​processing unit and the shielding area determination unit; wherein the number of camera acquisition units in the camera acquisition module is much larger than the number of shared processing units in the shared processing module; the image storage unit is used to store the monitoring images to be tested acquired in real time by all the camera acquisition units; the shared processing unit is used to obtain the monitoring images to be tested from the image storage unit, and after performing image processing on the monitoring images to be tested, send the image processing results to the camera acquisition unit corresponding to the monitoring images to be tested.

[0014] Optionally, the masked area determination unit includes: a first determination subunit, a judgment subunit, a second determination subunit, and a third determination subunit; the first determination subunit is used to obtain a target bounding box in the target detection result, and determine the degree of overlap between the target bounding box and a preset masked area; the judgment subunit is used to judge whether the degree of overlap reaches a preset IOU threshold to obtain a judgment result; the second determination subunit is used to determine that the target object is in the production line area if the judgment result indicates that the degree of overlap does not reach the preset IOU threshold; the third determination subunit is used to determine that the target object is in a safe area if the judgment result indicates that the degree of overlap reaches the preset IOU threshold.

[0015] Optionally, the AI ​​processing unit includes a model generation subunit, a model prediction subunit and a sending subunit; the model generation subunit is used to perform model training on several sample monitoring images using the YOLOv5 algorithm to generate a target detection model; the model prediction subunit is used to perform target detection on the monitoring images to be tested of the production line based on the target detection model, and output the target detection results; the sending subunit is used to send the target detection results to the shielding area determination unit.

[0016] Optionally, the model generation subunit includes: a feature fusion processing unit, a model training unit, and a generation unit; the feature fusion processing unit is used to perform multi-scale feature extraction processing on each sample monitoring image, and perform feature fusion processing on the generated multi-scale feature image, and output a feature fusion image; the model training unit is used to perform model convergence detection based on multiple feature fusion images, and output a target detection loss; the generation unit is used to generate a target detection model when the target detection loss tends to be minimized.

[0017] Optionally, the target detection loss includes: target bounding box position loss and target category classification loss; the target bounding box position loss includes: sum of square error loss and SmoothL1 loss; the sum of square error loss is used to measure the prediction deviation of the target bounding box position; the Smooth L1 loss is used to reduce the impact of outliers on the target detection loss, and to generate a smooth loss value when the prediction deviation of the target bounding box position is small and to generate a larger loss value when the prediction deviation is large; assuming that the predicted target bounding box position is (x, y, w, h) and the target bounding box position of the true label is (x', y', w', h'), the sum of square error loss function is calculated as shown in formula (1):

[0018] Formula (1);

[0019] Among them, ^2 means taking the square, and L means the loss value;

[0020] The calculation of the Smooth L1 loss function is shown in formula (2):

[0021] Formula (2);

[0022] Wherein, smoothL1(x) represents the smooth L1 function, which is used to indicate the difference between the predicted position and the true position; the calculation of smoothL1(x) is shown in formula (3):

[0023] Formula (3);

[0024] The target category classification loss is used to calculate the target category classification error through the cross entropy loss function; the cross entropy loss function is used to measure the difference between the predicted result and the true label in target classification.

[0025] Optionally, the target detection loss further includes: target confidence loss; the target confidence loss is used to measure the difference between the probability prediction of whether the target exists and the true label in target detection; the calculation of the target confidence loss function is shown in formula (5):

[0026] Formula (5);

[0027] Among them, L_conf represents the value of the target confidence loss term, ∑ represents the sum of all target boxes, y represents the true label (0 or 1), and p represents the predicted target confidence.

[0028] Optionally, the preset shielding area is obtained by drawing the glass window in the monitored image to be tested to obtain an image mask; each of the monitored images to be tested has a corresponding preset shielding area.

[0029] According to the second aspect of this embodiment, the present invention also provides a real-time monitoring method for production line safety, including: performing target detection on the monitoring image to be tested of the production line, and outputting the target detection result; if the target detection result indicates that there is a target object in the production line area, determining the degree of overlap between the target detection result and the corresponding preset shielding area; determining whether the target object is in a safe area based on the overlap degree, and generating a monitoring result; wherein the preset shielding area is used to indicate a safe area outside the glass windows located in the production line area.

[0030] The present invention provides a real-time monitoring system and method for production line safety, the system comprising: an AI processing unit, for performing target detection on the monitoring image to be tested of the production line, and outputting the target detection result; and when the target detection result indicates that there is a target object in the production line area, the target detection result is sent to a shielding area determination unit; the shielding area determination unit is used to determine the degree of overlap between the target detection result and a preset shielding area corresponding to the monitoring image to be tested; and based on the degree of overlap, determine whether the target object is in a safe area, and generate a monitoring result; wherein the preset shielding area is used to indicate a safe area outside the glass windows in the production line area. This embodiment combines target detection with safe area judgment, and can accurately distinguish between the production line area and the safe area in a workshop environment, thereby reducing the false detection rate and realizing accurate monitoring of production line personnel on the production line and accurate detection of their residence. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Hereinafter, some specific embodiments of the present invention will be described in detail in an exemplary and non-limiting manner with reference to the accompanying drawings. The same reference numerals in the accompanying drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. In the accompanying drawings:

[0032] Figure 1 A schematic diagram of the structure of a real-time monitoring system for production line safety provided by an embodiment of the present invention;

[0033] Figure 2A schematic structural diagram of a real-time monitoring system for production line safety provided by another embodiment of the present invention;

[0034] Figure 3 A schematic diagram of a monitoring image to be tested provided by an embodiment of the present invention;

[0035] Figure 4 A flowchart of a real-time monitoring method for production line safety provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0037] like Figure 1 As shown, it is a structural diagram of a real-time monitoring system for production line safety provided by an embodiment of the present invention; Figure 3 , which is a schematic diagram of a monitoring image to be tested provided by an embodiment of the present invention.

[0038] A real-time monitoring system for production line safety, the system 100 includes: an AI processing unit 101, used to perform target detection on the monitoring image to be tested of the production line and output the target detection result; and when the target detection result indicates that there is a target object in the production line area, the target detection result is sent to a shielding area determination unit 102; the shielding area determination unit 102 is used to determine the degree of overlap between the target detection result and a preset shielding area corresponding to the monitoring image to be tested; and based on the overlap degree, determine whether the target object is in a safe area and generate a monitoring result; wherein the preset shielding area is used to indicate a safe area outside the glass windows of the production line area.

[0039] In the AI ​​processing unit 101, target detection is performed on the monitoring image to be tested of the production line based on preset rules or training models, and the target detection result is output. For example: the AI ​​processing unit includes a model generation subunit, a model prediction subunit and a sending subunit; the model generation subunit is used to perform model training on several sample monitoring images using the YOLOv5 algorithm to generate a target detection model; the model prediction subunit is used to perform target detection on the monitoring image to be tested of the production line based on the target detection model, and output the target detection result; the sending subunit is used to send the target detection result to the shielding area determination unit. The present invention adopts the YOLO-V5 algorithm, which utilizes efficient convolution and pooling layer stacking, and can extract multi-scale features with increasing abstraction level layer by layer. This can reduce the consumption of computing resources while maintaining an effective representation of the target. Compared with the complex computing requirements in the prior art, the YOLO-V5 algorithm of the present invention reduces unnecessary computing steps, reduces the computational complexity, and improves the running speed of the algorithm.

[0040] In the mask area determination unit 102 , there is no limitation on the calculation method of the overlap degree, as long as the intersection and union between the target detection result and the preset mask area can be determined.

[0041] Exemplary: Based on the IOU threshold method, the degree of overlap between the target detection result and the preset shielding area corresponding to the monitored image to be tested can be determined; if the degree of overlap is less than the preset IOU threshold, it is determined that the target object detected by the target detection result is in a safe area; Figure 3 If the overlap degree is not less than the preset IOU threshold, it is determined that the target object detected by the target detection result is in the production line area.

[0042] like Figure 3 As shown, the preset shielding area varies with the different monitoring images to be tested. For example: the preset shielding area is obtained by drawing the glass windows in the monitoring image to be tested to obtain an image mask; each of the monitoring images to be tested has a corresponding preset shielding area. This embodiment draws the image mask by connecting points to surfaces, so that the customer can independently define the glass areas of windows and doors according to the actual situation and needs of the workshop, thereby limiting the scope of target detection to areas other than the glass areas of windows and doors. Such flexibility and customizability make the monitoring system of the present invention more suitable for different workshop environments and layouts.

[0043] This embodiment combines target detection and safety area judgment, and can accurately distinguish between production line areas and safety areas in a workshop environment, thereby reducing the false detection rate and achieving precise monitoring of production personnel on the production line and accurate detection of their residence status.

[0044] In a preferred implementation of this embodiment, the system also includes: a camera acquisition unit 103, which is used to collect the monitoring image to be tested in the production line area and send it to the AI ​​processing unit; and generate an alarm prompt information after receiving the monitoring result that the target object is not in the preset shielding area.

[0045] Specifically, the camera acquisition unit is connected to the AI ​​processing unit and the shielding area determination unit respectively through communication; the AI ​​processing unit and the shielding area determination unit are connected through communication.

[0046] In a preferred implementation manner of the present embodiment, the shielding area determination unit includes: a first determination subunit, a judgment subunit, a second determination subunit, and a third determination subunit; the first determination subunit is used to obtain the target bounding box in the target detection result, and determine the degree of overlap between the target bounding box and the preset shielding area; the judgment subunit is used to judge whether the degree of overlap reaches a preset IOU threshold to obtain a judgment result; the second determination subunit is used to determine that the target object is in the production line area if the judgment result indicates that the degree of overlap does not reach the preset IOU threshold; the third determination subunit is used to determine that the target object is in a safe area if the judgment result indicates that the degree of overlap reaches the preset IOU threshold.

[0047] The present invention adopts an innovative custom shielding algorithm to provide a practical solution for effective shielding in the production line area, so that customers can flexibly shield the window glass area and the door glass area in the production line area according to actual needs. Traditional target detection algorithms are difficult to solve the shielding problem of the window glass area and the door glass area in the production line area, and the algorithm implemented in this embodiment allows all camera shooting areas to be opened to customers, and customers can independently draw preset shielding areas by connecting points to surfaces. Such a custom shielding function effectively avoids the interference of irrelevant areas on target detection and improves the accuracy of the monitoring system.

[0048] In addition, the present invention also opens up the IOU threshold adjustment function, which flexibly controls the overlap between the target detection result and the preset shielding area by adjusting the IOU threshold, further optimizing the monitoring effect. Through these improvement measures, the present invention can accurately distinguish between the production line area and the safety area in the workshop environment, thereby reducing the false detection rate and realizing accurate monitoring of production line personnel and accurate detection of residence status. This function makes the algorithm of the present invention more flexible to adapt to the monitoring needs in different scenarios, and enhances the applicability and practicality of the system.

[0049] In a preferred implementation manner of this embodiment, the model generation subunit includes: a feature fusion processing unit, a model training unit, and a generation unit; the feature fusion processing unit is used to perform multi-scale feature extraction processing on each sample monitoring image, and perform feature fusion processing on the generated multi-scale feature image, and output a feature fusion image; the model training unit is used to perform model convergence detection based on multiple feature fusion images, and output a target detection loss; the generation unit is used to generate a target detection model when the target detection loss tends to be minimized.

[0050] This embodiment reduces the number of scales in the feature pyramid of the YOLO-V5 algorithm, and reduces the number of upsampling, downsampling, and feature fusion, thereby introducing the technology of parallel processing and optimization of the calculation process; and achieves real-time monitoring capability per second. Compared with the prior art, the simplified YOLO-V5 algorithm in this embodiment can simplify the calculation steps and reduce the calculation complexity by optimizing the calculation process and adjusting parameters while ensuring accuracy, and achieves rapid processing of the calculation process, so that the system can perform efficient monitoring in real-time scenarios.

[0051] Since the production line personnel in the real-time scene of this embodiment are large targets, although the YOLO-V5 algorithm is simplified, it is still possible to effectively detect the target object, thereby achieving real-time monitoring of the residence status of the production line personnel while improving the reasoning calculation speed.

[0052] The target detection loss includes: target bounding box position loss and target category classification loss; the target bounding box position loss includes: sum of square error loss and SmoothL1 loss; the sum of square error loss is used to measure the prediction deviation of the target bounding box position; the Smooth L1 loss is used to reduce the impact of outliers on the target detection loss and generate a smooth loss value when the prediction deviation of the target bounding box position is small and generate a larger loss value when the prediction deviation is large;

[0053] Assuming that the predicted target bounding box position is (x, y, w, h) and the target bounding box position of the true label is (x', y', w', h'), the squared error loss function is calculated as shown in formula (1):

[0054] Formula (1);

[0055] Among them, ^2 means taking the square, and L means the loss value;

[0056] The calculation of the Smooth L1 loss function is shown in formula (2):

[0057] Formula (2);

[0058] Wherein, smoothL1(x) represents the smooth L1 function, which is used to indicate the difference between the predicted position and the true position; the calculation of smoothL1(x) is shown in formula (3):

[0059] Formula (3);

[0060] This embodiment can reduce the impact of outliers on the loss through Smooth L1 loss, thereby improving the robustness of the target detection algorithm.

[0061] The target category classification loss is used to calculate the target category classification error through the cross entropy loss function; the cross entropy loss function is used to measure the difference between the prediction result and the true label in target classification; the calculation formula of the cross entropy loss function is shown in formula (4):

[0062] Formula (4);

[0063] Among them, L represents the loss value, ∑ represents the sum of all categories, y represents the probability distribution vector of the true label, and p represents the probability distribution vector of the predicted result.

[0064] In a preferred implementation of this embodiment, the target detection loss also includes: target confidence loss; the target confidence loss is used to measure the difference between the probability prediction of whether the target exists and the true label in target detection; the calculation of the target confidence loss function is shown in formula (5):

[0065] Formula (5);

[0066] Among them, L_conf represents the value of the target confidence loss term, ∑ represents the sum of all target boxes, y represents the true label (0 or 1), and p represents the predicted target confidence.

[0067] This embodiment introduces a target confidence loss term to predict the confidence of the existence of the target and calculate the confidence error, thereby further improving the accuracy of target positioning and detection.

[0068] By improving the YOLO-V5 algorithm, this embodiment can better capture the detailed features of the target and improve the accuracy and robustness of target detection; by optimizing the loss function of the YOLO-V5 algorithm, it can more accurately identify and locate production line personnel, thereby reducing the false detection rate and missed detection rate and improving the accuracy of target detection; thereby improving the accuracy and reliability of the system.

[0069] like Figure 2FIG. 1 is a schematic diagram of the structure of a real-time monitoring system for production line safety provided by another embodiment of the present invention.

[0070] When the production line area includes multiple camera acquisition units, in order to meet the real-time needs of on-site monitoring of production line personnel, multiple shared processing units are introduced to achieve parallel processing and efficient computing.

[0071] A real-time monitoring system for production line safety, the system 100 comprising: a shared processing module 110 and a camera acquisition module 120; the camera acquisition module 120 comprises a plurality of camera acquisition units 103 and an image storage unit 104; the shared processing module 110 comprises a plurality of shared processing units 111; each of the shared processing units 111 comprises the AI ​​processing unit 101 and the shielding area determination unit 102; wherein the number of camera acquisition units in the camera acquisition module 120 is much greater than the number of shared processing units in the shared processing module 110;

[0072] The image storage unit 104 is used to store all the monitoring images to be tested collected in real time by the camera collection unit 103;

[0073] The shared processing unit 111 is used for acquiring the monitored image to be tested from the image storage unit 104 , and after performing image processing on the monitored image to be tested, sending the image processing result to the camera acquisition unit 103 corresponding to the monitored image to be tested.

[0074] Specifically, the shared processing module divides the computing task into multiple subtasks and assigns them to multiple shared processing units for simultaneous processing. Each shared processing unit has independent computing resources and processing capabilities, and can independently process the tasks assigned to it. This parallel processing method enables the system to process multiple image frames at the same time, greatly improving the processing speed and real-time performance. A data sharing and communication mechanism is adopted between multiple shared processing units. These shared processing units can transfer data and synchronize operations through shared memory or other communication methods; thereby, the consistency of the calculation results and intermediate data between the shared processing units can be ensured, and efficient parallel computing and collaborative work can be achieved.

[0075] This embodiment introduces multiple shared processing units and implements task division, parallel processing and data sharing, so that computing resources can be better utilized in real-time monitoring, and the processing capability and real-time performance of the system can be improved. The collaborative work of multiple shared processing units enables the system to quickly process and return a large number of monitoring results per second. This makes the present invention efficient, fast and reliable in the on-site monitoring of production line personnel, and can monitor and respond to various situations in real time.

[0076] like Figure 4FIG. 1 is a flow chart of a method for real-time monitoring of production line safety provided by an embodiment of the present invention.

[0077] A real-time monitoring method for production line safety, comprising:

[0078] S401, performing target detection on the monitoring image to be tested of the production line and outputting the target detection result;

[0079] S402, if the target detection result indicates that there is a target object in the production line area, determining the degree of overlap between the target detection result and the corresponding preset shielding area;

[0080] S403, determining whether the target object is in a safe area based on the degree of overlap, and generating a monitoring result; wherein the preset shielding area is used to indicate a safe area outside the glass windows in the production line area.

[0081] In a preferred implementation manner of this embodiment, the method further includes: if the monitoring result indicates that the target object is not in a preset shielding area, generating an alarm prompt message.

[0082] In a preferred implementation manner of this embodiment, before performing target detection on the monitoring images to be tested of the production line, the method further includes: collecting the monitoring images to be tested by using a plurality of camera acquisition units, and storing the collected monitoring images to be tested in the same storage area.

[0083] In a preferred implementation manner of this embodiment, the method comprises: obtaining a target boundary box in the target detection result, and determining a degree of overlap between the target boundary box and a preset shielding area corresponding to the monitored image to be tested; and determining whether the target object is in a safe area based on the degree of overlap to generate a monitoring result; comprising: obtaining a target boundary box in the target detection result, and determining a degree of overlap between the target boundary box and a preset shielding area; judging whether the degree of overlap reaches a preset IOU threshold to obtain a judgment result; if the judgment result indicates that the degree of overlap does not reach the preset IOU threshold, then determining that the target object is in a production line area; if the judgment result indicates that the degree of overlap reaches the preset IOU threshold, then determining that the target object is in a safe area; wherein the preset shielding area is an image mask obtained by drawing a glass window in the monitored image to be tested; and each of the monitored images to be tested has a corresponding preset shielding area.

[0084] In a preferred implementation of this embodiment, the target detection is performed on the monitoring image to be tested of the production line, and the target detection result is output; including: using the YOLOv5 algorithm to perform model training on a number of sample monitoring images to generate a target detection model; based on the target detection model, the target detection is performed on the monitoring image to be tested of the production line, and the target detection result is output;

[0085] In a preferred implementation manner of this embodiment, the YOLOv5 algorithm is used to perform model training on several sample monitoring images to generate a target detection model; including: performing multi-scale feature extraction processing on each sample monitoring image, and performing feature fusion processing on the generated multi-scale feature image, and outputting a feature fusion image; performing model convergence detection based on multiple feature fusion images, and outputting a target detection loss; when the target detection loss tends to be minimized, generating a target detection model.

[0086] In a preferred implementation of this embodiment, the target detection loss includes: target bounding box position loss and target category classification loss;

[0087] The target bounding box position loss includes: a sum of square error loss and a SmoothL1 loss; the sum of square error loss is used to measure the prediction deviation of the target bounding box position; the Smooth L1 loss is used to reduce the influence of outliers on the target detection loss and generate a smooth loss value when the prediction deviation of the target bounding box position is small and generate a larger loss value when the prediction deviation is large;

[0088] Assuming that the predicted target bounding box position is (x, y, w, h) and the target bounding box position of the true label is (x', y', w', h'), the squared error loss function is calculated as shown in formula (1):

[0089] Formula (1);

[0090] Among them, ^2 means taking the square, and L means the loss value;

[0091] The calculation of the Smooth L1 loss function is shown in formula (2):

[0092] Formula (2);

[0093] Wherein, smoothL1(x) represents the smooth L1 function, which is used to indicate the difference between the predicted position and the true position; the calculation of smoothL1(x) is shown in formula (3):

[0094] Formula (3);

[0095] The target category classification loss is used to calculate the target category classification error through the cross entropy loss function; the cross entropy loss function is used to measure the difference between the predicted result and the true label in target classification.

[0096] In a preferred implementation of this embodiment, the target detection loss also includes: target confidence loss; the target confidence loss is used to measure the difference between the probability prediction of whether the target exists and the true label in target detection; the calculation of the target confidence loss function is shown in formula (5):

[0097] Formula (5);

[0098] Among them, L_conf represents the value of the target confidence loss term, ∑ represents the sum of all target boxes, y represents the true label (0 or 1), and p represents the predicted target confidence.

[0099] This embodiment combines the improved YOLO-V5 algorithm and the custom shielding algorithm in the shared processing unit, which can improve the accuracy and real-time performance of production line personnel monitoring, thereby accurately distinguishing between production line areas and safety areas, reducing the false detection rate, and achieving precise monitoring of production line personnel and accurate detection of their residence status.

[0100] The present invention also provides an electronic device, comprising: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement the monitoring method of the present invention.

[0101] In addition, an embodiment of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the method according to the following embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0102] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0103] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.

[0104] It should also be noted that in the system, method and device of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0105] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

[0106] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0107] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0108] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A real-time monitoring system for production line safety, characterized in that: include: An AI processing unit, configured to perform target detection on the monitoring image to be tested of the production line and output the target detection result; When the target detection result indicates that there is a target object in the production line area, the target detection result is sent to a shielding area determination unit; The shielding area determination unit is used to determine the degree of overlap between the target detection result and the preset shielding area corresponding to the monitored image to be detected; And based on the overlap degree, determine whether the target object is in a safe area, and generate a monitoring result; wherein the preset shielding area is used to indicate a safe area outside the glass windows in the production line area; The AI ​​processing unit includes a model generation subunit, a model prediction subunit, and a sending subunit; The model generation subunit is used to perform model training on a number of sample monitoring images using the YOLOv5 algorithm to generate a target detection model; wherein the YOLO-V5 algorithm is used to indicate an improved algorithm formed by reducing the number of scales in the feature pyramid and the number of upsampling, downsampling, and feature fusion of the existing YOLOv5 algorithm; the model generation subunit includes: a feature fusion processing unit, a model training unit, and a generation unit; the feature fusion processing unit is used to extract multi-scale feature images with increasing abstraction levels layer by layer for each sample monitoring image using a stack of convolution and pooling layers, and perform feature fusion processing on the generated multi-scale feature images to output feature fusion images; the model training unit is used to perform model convergence detection based on multiple feature fusion images and output target detection loss; the generation unit is used to generate a target detection model when the target detection loss tends to be minimized; The model prediction subunit is used to perform target detection on the monitoring image to be tested of the production line based on the target detection model, and output the target detection result; The sending subunit is used to send the target detection result to the shielding area determination unit; The shielding area determination unit includes: a first determination subunit, a judgment subunit, a second determination subunit, and a third determination subunit; the first determination subunit is used to obtain the target bounding box in the target detection result, and determine the degree of overlap between the target bounding box and the preset shielding area; the judgment subunit is used to judge whether the degree of overlap reaches a preset IOU threshold to obtain a judgment result; the second determination subunit is used to determine that the target object is in the production line area when the judgment result indicates that the degree of overlap does not reach the preset IOU threshold; the third determination subunit is used to determine that the target object is in a safe area when the judgment result indicates that the degree of overlap reaches the preset IOU threshold; wherein the IOU threshold has an adjustment function; The preset shielding area is obtained by drawing the glass window in the monitored image to be tested to obtain an image mask; each of the monitored images to be tested has a corresponding preset shielding area.

2. The system according to claim 1, characterized in that Also includes: A camera acquisition unit, used to collect the monitoring image to be tested in the production line area and send it to the AI ​​processing unit; And after receiving the monitoring result that the target object is not in the preset shielding area, an alarm prompt message is generated.

3. The system according to claim 1, characterized in that Also includes: Shared processing module and camera acquisition module; The camera acquisition module includes a plurality of camera acquisition units and an image storage unit; The shared processing module includes a plurality of shared processing units; each of the shared processing units includes the AI ​​processing unit and the shielding area determination unit; wherein the number of camera acquisition units in the camera acquisition module is much greater than the number of shared processing units in the shared processing module; The image storage unit is used to store all the monitoring images to be tested collected in real time by the camera collection units; The shared processing unit is used to obtain the monitored image to be tested from the image storage unit, and after performing image processing on the monitored image to be tested, send the image processing result to the camera acquisition unit corresponding to the monitored image to be tested.

4. The system according to claim 1, characterized in that The target detection loss includes: target bounding box position loss and target category classification loss; The target bounding box position loss includes: square sum error loss and SmoothL1 loss; Assuming that the predicted target bounding box position is (x, y, w, h) and the target bounding box position of the true label is (x', y', w', h'), the squared error loss function is calculated as shown in formula (1): Formula (1); Among them, ^2 means taking the square, and L means the loss value; The calculation of the Smooth L1 loss function is shown in formula (2): Formula (2); Wherein, smoothL1(x) represents the smooth L1 function, which is used to indicate the difference between the predicted position and the true position; the calculation of smoothL1(x) is shown in formula (3): Formula (3); The target category classification loss is used to calculate the target category classification error through the cross entropy loss function; the cross entropy loss function is used to measure the difference between the predicted result and the true label in target classification.

5. The system according to claim 4, characterized in that The target detection loss also includes: target confidence loss; The target confidence loss is used to measure the difference between the probability prediction of whether the target exists and the true label in target detection; the calculation of the target confidence loss function is shown in formula (5): Formula (5); Among them, L_conf represents the value of the target confidence loss term, ∑ represents the sum of all target boxes, y represents the true label, and p represents the predicted target confidence.

6. A method for real-time monitoring of production line safety using the real-time monitoring system according to any one of claims 1 to 5, characterized in that: include: Performing target detection on the monitoring image to be tested of the production line and outputting the target detection result; If the target detection result indicates that there is a target object in the production line area, determining the degree of overlap between the target detection result and the corresponding preset shielding area; Based on the degree of overlap, it is determined whether the target object is in a safe area, and a monitoring result is generated; wherein the preset shielding area is used to indicate a safe area outside the glass windows in the production line area.

Images