Intelligent monitoring management system for production workshop

Abstract

The invention relates to the field of production system monitoring, in particular to an intelligent monitoring management system for a production workshop, which is characterized in that through cooperative work of a monitoring data acquisition module, a collaborative analysis module, a classification module and a monitoring early warning module, image data of a target equipment area is acquired based on the monitoring data acquisition module; the collaborative analysis module calculates action collaborative characterization values of an operator and an equipment moving part and classifies the equipment based on the classification module, and then the monitoring and early warning module analyzes a movement track based on the category of the equipment by calibrating a collaborative associated action part and based on the movement track of the moving part and the action part. And whether the video segments meet the coordination action standard is judged, or a violation joint point action graph is matched, whether production is abnormal is judged, and the workshop safety and the production efficiency are effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of production system monitoring, and in particular to an intelligent monitoring and management system for a production workshop. Background Art

[0002] With the continuous advancement of global industrialization and the gradual popularization of intelligent manufacturing concepts, modern production workshops are moving towards a high degree of automation and intelligence. The intelligent monitoring and management system of production workshops is closely related not only to industrial production safety, but also to the high-quality development of the intelligent manufacturing industry. The intelligent monitoring and management system of production workshops has become an indispensable key link in realizing intelligent manufacturing and ensuring production safety.

[0003] Chinese Patent Publication No.: CN111537022A discloses a safety monitoring system for stamping and forming production workshops based on artificial intelligence, which is characterized by including a safety monitoring module, a safety information collection module and a safety management center. The safety monitoring module uses a sensor component to collect workshop environmental parameter data and transmits the collected workshop environmental parameter data to the safety information collection module. The safety information collection module forwards the received workshop environmental parameter data to the safety management center. The safety management center compares the received workshop environmental parameter data with a given safety threshold and issues an early warning when the workshop environmental parameter data is higher than the given safety threshold. The beneficial effect of this invention is that the sensor component is used to perform safety monitoring on the stamping and forming production workshop, which can ensure the safe operation of the stamping and forming production workshop.

[0004] However, the prior art still has the following problems:

[0005] In actual situations, when the equipment in the production workshop is running, due to the transmission relationship of the equipment, when the operator makes the predetermined action, the moving parts of the equipment usually move in coordination. There is a collaborative relationship. Whether the operator's action forms an effective coordination with the moving parts of the equipment has not been fully paid attention to, and it is difficult to detect the potential safety hazards caused by coordination disorder in time. The existing system lacks an effective mechanism to adjust the monitoring focus according to the coordination of equipment, resulting in low monitoring efficiency and reducing the overall safety and production efficiency of the production workshop. Summary of the Invention

[0006] To this end, the present invention provides an intelligent monitoring and management system for a production workshop, which is used to overcome the problem in the prior art that when the equipment in the production workshop is running, whether the operator's actions form effective coordination with the moving parts of the equipment is not fully paid attention to, and it is difficult to timely detect potential safety hazards due to coordination disorders, resulting in low monitoring efficiency and reduced overall safety and production efficiency of the production workshop.

[0007] To achieve the above objectives, the present invention provides an intelligent monitoring and management system for a production workshop, which includes:

[0008] A monitoring data acquisition module, which is used to collect image data of the area where several target devices are located;

[0009] a collaborative analysis module, connected to the monitoring data acquisition module, configured to extract image data containing the target device and the operator, and perform motion collaborative analysis on the image data, including determining motion trajectories of various motion parts of the operator when the moving parts of the target device perform the same motion, and calculating motion collaborative representation values ​​for the motion parts based on the motion trajectories of the various motion parts;

[0010] a classification module connected to the collaboration analysis module, configured to classify target devices based on the action collaboration representation value;

[0011] A monitoring and early warning module, which is connected to the monitoring data acquisition module, the collaborative analysis module and the classification module, is used to monitor the image data of the area where the target device is located based on the category of the target device, including:

[0012] Determining a moving part of a target device within a video segment, calibrating an action portion that is cooperatively associated with the moving part, and determining whether the video segment meets a cooperative action standard based on the motion trajectories of the moving part and the action portion to determine whether there is an anomaly;

[0013] Alternatively, the operator's joint action graphs in the video frames are extracted one by one and matched with the illegal joint action graphs in the illegal sample library to determine whether there is an abnormality.

[0014] Furthermore, the collaborative analysis module is used to perform collaborative analysis of actions, including:

[0015] A plurality of video segments for determining the same action of a single moving part of the target device;

[0016] Determine the key points of each action part in each video segment, and determine the action trajectory of each action part corresponding to the key points;

[0017] It is used to calculate the average similarity between the motion trajectories of the same motion part, and set the average similarity as the motion collaborative representation value of the motion part.

[0018] Furthermore, the classification module is used to classify the target device based on the action collaborative representation value, including:

[0019] If the action collaborative representation value of any action part is greater than or equal to the preset action collaborative representation standard value, the target device is determined to be in the collaborative relationship category;

[0020] If the motion cooperative representation value of any motion part is less than the preset motion cooperative representation standard value, the target device is determined to be in a non-cooperative relationship category.

[0021] Furthermore, the collaborative analysis module is also used to determine the collaborative association between the moving parts of the target device and the operator's action parts.

[0022] If the motion synergy characterization value of the motion part is greater than or equal to a preset motion synergy characterization standard value, it is determined that there is a synergy association between the motion part of the operator and the moving component of the target device.

[0023] Furthermore, the collaborative analysis module is further configured to record the motion trajectory of the action portion and the motion trajectory of the action component in the video segment, and store them in the motion trajectory set corresponding to the collaborative association.

[0024] Furthermore, the monitoring and warning module is used to monitor the image data of the area where the target device is located based on the category of the target device, including:

[0025] If the target device is in a collaborative relationship category, the moving parts of the target device in the video segment are determined, the action part that cooperates with the moving parts is calibrated, and based on the motion trajectories of the moving parts and the action part, whether the video segment meets the collaborative action standard is determined to determine whether there is any abnormality;

[0026] If the target device belongs to the non-cooperative relationship category, the operator's joint action graphs in the video frames are extracted one by one and matched with the illegal joint action graphs in the illegal sample library to determine whether there is an abnormality.

[0027] Furthermore, the monitoring and early warning module is used to determine the moving parts of the target device in the video segment, and to mark the action parts that are cooperatively associated with the moving parts, including:

[0028] To determine the motion part that is in a cooperative relationship with the moving part;

[0029] It is used to mark the action part and synchronously call the motion trajectory set corresponding to the collaborative association.

[0030] Furthermore, the monitoring and early warning module is used to determine whether the video segment meets the collaborative action standard based on the motion trajectory of the moving component and the action part, so as to determine whether there is an abnormality, including:

[0031] Used to extract the motion trajectory of the moving parts and the motion trajectory of the calibrated action parts respectively;

[0032] Comparing the extracted motion trajectory of the moving part with the motion trajectory of the action part in the motion trajectory set to calculate the similarity of the action part trajectory;

[0033] Comparing the extracted motion trajectory of each action part with the motion trajectory of the corresponding action part in the motion trajectory set to calculate the similarity of the action part trajectory;

[0034] If the coordinated action criteria are met, it is determined that there is no abnormality;

[0035] If the coordinated action standards are not met, it is determined to be abnormal;

[0036] The collaborative action standard is that the trajectory similarity of the action components is greater than a predetermined standard trajectory similarity threshold, and the trajectory similarity of each action part is greater than a predetermined standard trajectory similarity threshold.

[0037] Furthermore, the monitoring and early warning module is used to extract the operator's joint action diagrams in the video frames one by one, and match them with the illegal joint action diagrams in the illegal sample library to determine whether there are any abnormalities, including:

[0038] Used to extract the operator's joint coordinate information and construct a joint action graph;

[0039] Comparing the joint action graph with the action graphs of each illegal joint point in the illegal sample library to determine the action similarity;

[0040] If the action similarity between the key point action graph and any illegal joint point action graph in the illegal sample library is greater than the preset action similarity threshold, it is determined that an anomaly exists.

[0041] Furthermore, the monitoring and early warning module is also used to store video segments or video frames that are determined to be abnormal.

[0042] Compared with the prior art, the present invention sets up the collaborative work of a monitoring data acquisition module, a collaborative analysis module, a classification module and a monitoring and early warning module. The monitoring data acquisition module collects image data of the target equipment area, the collaborative analysis module calculates the motion collaborative representation value of the operator and the moving parts of the equipment, and classifies the equipment based on the classification module. Then, the monitoring and early warning module, based on the equipment category, calibrates the collaborative associated action parts, analyzes the motion trajectory based on the motion trajectory of the moving parts and the action parts, determines whether the video segment meets the collaborative action standard, or matches the illegal joint action diagram, determines whether the production is abnormal, and effectively improves the workshop monitoring efficiency while ensuring reliability.

[0043] In particular, the present invention extracts image data containing target equipment and operators, and performs motion coordination analysis on the image data. In actual situations, due to the transmission relationship of the equipment, the equipment will give certain feedback after the operator makes certain actions. In some cases, there is an obvious coordination relationship between the action parts of the equipment and the action parts of the operator. If the coordination relationship is broken, it may indicate that there is an abnormality in the equipment or the action of the operator. Based on this, the present invention considers performing motion coordination analysis on the target equipment in advance. When there are a large number of target equipment in the production workshop, the coordination relationship can be fully utilized to quickly analyze whether there are potential abnormalities when the target equipment is operated by the operator. The present invention performs motion coordination analysis in a specific way, and performs adaptive analysis on the corresponding image data for the subsequent classification of target equipment types, thereby ensuring reliability and improving the efficiency of monitoring a large number of equipment in the workshop.

[0044] In particular, the present invention classifies target devices based on action collaboration characterization values. In actual situations, due to the different types of target devices and the differences in action modes, some devices have a strong collaborative relationship with the operator when they act. Therefore, the present invention scientifically divides the equipment categories based on the action collaboration characterization values. If the target device is in the collaborative relationship category, it is necessary to focus on its collaboration with the operator's action. By calibrating the collaborative associated action parts and analyzing the motion trajectory to determine whether there is an abnormality, a rapid abnormality judgment of the target device and the operator is achieved. If the target device is in the non-collaborative relationship category, it focuses on the matching of the operator's joint point action diagram with the violation sample library, and adopts a traversal analysis method to avoid missing abnormal features. Therefore, this monitoring strategy can ensure the reasonable allocation and utilization of system resources, ensure reliability, and improve the efficiency of monitoring a large number of equipment in the workshop.

[0045] In particular, the present invention calibrates the action parts that cooperate with the moving parts, and determines whether the video segment meets the collaborative action standards based on the motion trajectory of the moving parts and the action parts. In actual situations, for target equipment of the collaborative relationship category, when it is operated, there is a collaborative relationship between the moving parts and the moving parts of the operator. The present invention identifies the action parts that are collaboratively associated with the motion parts based on the collaborative association relationship determined in advance, and then compares them. It has low computing power consumption and can quickly determine whether there are abnormalities between the target equipment and the operator based on the collaborative relationship. It can promptly discover abnormal situations in equipment of the collaborative relationship category, ensure reliability, and improve the efficiency of monitoring a large number of equipment in the workshop. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A system block diagram of an intelligent monitoring and management system for a production workshop according to an embodiment of the invention;

[0047] Figure 2A logic decision diagram for identifying the cooperation category of the current model target device according to an embodiment of the invention;

[0048] Figure 3 A logic block diagram of monitoring image data of an area where a target device is located according to a collaborative category according to an embodiment of the present invention;

[0049] Figure 4 A logic determination diagram for monitoring whether image data in an area where a target device of a collaborative relationship category is located is abnormal in an embodiment of the present invention;

[0050] Figure 5 This is a logic decision diagram for monitoring whether image data in an area where a target device of a non-cooperative relationship category is located is abnormal in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are merely used to explain the present invention and are not intended to limit the present invention.

[0052] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0053] Furthermore, it should be noted that, in the description of the present invention, unless otherwise expressly specified or limited, the term "connection" should be understood in a broad sense. For example, it can mean a fixed connection, a detachable connection, or an integral connection; it can mean a mechanical connection or an electrical connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0054] See also Figure 1 As shown, it is a system block diagram of an intelligent monitoring and management system for a production workshop according to an embodiment of the present invention. The intelligent monitoring and management system for a production workshop according to an embodiment of the present invention includes:

[0055] A monitoring data acquisition module, which is used to collect image data of the area where several target devices are located;

[0056] a collaboration analysis module, connected to the monitoring data acquisition module, configured to extract image data containing the target device and the operator, and perform motion collaboration analysis on the image data, including determining motion trajectories of various motion parts of the operator when the moving parts of the target device perform the same motion, and calculating a motion collaboration representation value for the target device based on the motion trajectories of the various motion parts;

[0057] a classification module connected to the collaboration analysis module, configured to classify target devices based on the action collaboration representation value;

[0058] A monitoring and early warning module, which is connected to the monitoring data acquisition module, the collaborative analysis module and the classification module, is used to monitor the image data of the area where the target device is located based on the category of the target device, including:

[0059] Determining a moving part of a target device within a video segment, calibrating an action portion that is cooperatively associated with the moving part, and determining whether the video segment meets a cooperative action standard based on the motion trajectories of the moving part and the action portion to determine whether there is an anomaly;

[0060] Alternatively, the operator's joint action graphs in the video frames are extracted one by one and matched with the illegal joint action graphs in the illegal sample library to determine whether there is an abnormality.

[0061] Specifically, there is no limitation on the structure of the collaborative analysis module, classification module and monitoring and warning module, which can be composed of logical components or a combination of logical components, and the logical components include field programmable processors, computers or microprocessors in computers.

[0062] Specifically, there is no limitation on the structure of the monitoring data acquisition module, which can be a photographic device. It only needs to be able to achieve accurate real-time monitoring of the area where the target device is located. This will not be elaborated here.

[0063] Specifically, the collaborative analysis module is used to perform collaborative analysis of actions, including:

[0064] A plurality of video segments for determining the same action of a single moving part of the target device;

[0065] Determine the key points of each action part in each video segment, and determine the action trajectory of each action part corresponding to the key points;

[0066] It is used to calculate the average similarity between the motion trajectories of the same motion part, and set the average similarity as the motion collaborative representation value of the motion part.

[0067] In implementation, the key point of the action part may be the center point of the action part.

[0068] In implementation, the action component is a moving component in the target device. The outlines of several components in the target device can be identified first, the motion outline can be determined, and then the action component can be determined. This will not be repeated.

[0069] In practice, the action part is the part of the operator that moves, such as arms, legs, etc. The outline of each part of the operator can be identified, the outline of the movement can be determined, and then the action part can be determined. This will not be repeated.

[0070] In implementation, there is no limitation on the method of calculating the similarity between the motion trajectories of the same motion part. The cosine similarity of two motion trajectories can be calculated to determine the cosine similarity as the similarity between the motion trajectories. Of course, other methods can also be used, which will not be repeated here.

[0071] The present invention extracts image data containing target equipment and operators, and performs motion coordination analysis on the image data. In actual situations, due to the transmission relationship of the equipment, the equipment will give certain feedback after the operator makes certain actions. In some cases, there is an obvious coordination relationship between the action parts of the equipment and the action parts of the operator. If the coordination relationship is broken, it may indicate that there is an abnormality in the equipment or the action of the operator. Based on this, the present invention considers performing motion coordination analysis on the target equipment in advance. When there are a large number of target equipment in the production workshop, the coordination relationship can be fully utilized to quickly analyze whether there are potential abnormalities when the target equipment is operated by the operator. The present invention performs motion coordination analysis in a specific way, and adaptively analyzes the corresponding image data for the subsequent classification of target equipment types to ensure reliability and improve the efficiency of monitoring a large number of equipment in the workshop.

[0072] See also Figure 2 As shown, it is a logic decision diagram for identifying the collaborative category of the current model target device according to an embodiment of the invention. Specifically, the classification module is used to classify the target device based on the action collaborative representation value, including:

[0073] If the action collaborative representation value of any action part is greater than or equal to the preset action collaborative representation standard value, the target device is determined to be in the collaborative relationship category;

[0074] If the motion cooperative representation value of any motion part is less than the preset motion cooperative representation standard value, the target device is determined to be in a non-cooperative relationship category.

[0075] The purpose of setting the standard value of action coordination characterization is to characterize the similarity of certain action parts made by operators. The standard value of action coordination characterization is predetermined, wherein:

[0076] Images of technicians in this field imitating the same action several times are recorded in advance, the motion trajectory of the action parts is determined, the average similarity between the motion trajectories is calculated, and the product of the average similarity and the precision coefficient is set as the standard value of the action collaborative representation. The precision coefficient is selected within the range of [0.85, 0.95].

[0077] Specifically, the collaborative analysis module is also used to determine the collaborative association between the moving parts of the target device and the operator's action parts.

[0078] The collaborative analysis module is also used to determine the collaborative relationship between the moving parts of the target device and the operator's action parts.

[0079] If the motion synergy characterization value of the motion part is greater than or equal to a preset motion synergy characterization standard value, it is determined that there is a synergy association between the motion part of the operator and the moving component of the target device.

[0080] Specifically, the collaborative analysis module is further used to record the motion trajectory of the action part and the motion trajectory of the action component in the video segment, and store them in the motion trajectory set corresponding to the collaborative association.

[0081] It is understandable that, for collaboratively associated moving parts and action locations, the corresponding motion trajectories are stored as samples for subsequent determination, which will not be elaborated herein.

[0082] In implementation, there is no limitation on the method of storing the motion trajectory set corresponding to the collaborative association. It can be stored through a file system or cloud storage service, etc., as long as the data integrity and accessibility requirements are met, which will not be elaborated here.

[0083] Specifically, see Figure 3 As shown, Figure 3 This is a logic block diagram of an embodiment of the invention for monitoring image data of an area where a target device is located based on a collaborative category. The monitoring and early warning module is used to monitor image data of an area where a target device is located based on the category of the target device, including:

[0084] If the target device is in a collaborative relationship category, the moving parts of the target device in the video segment are determined, the action part that cooperates with the moving parts is calibrated, and based on the motion trajectories of the moving parts and the action part, whether the video segment meets the collaborative action standard is determined to determine whether there is any abnormality;

[0085] If the target device belongs to the non-cooperative relationship category, the operator's joint action graphs in the video frames are extracted one by one and matched with the illegal joint action graphs in the illegal sample library to determine whether there is an abnormality.

[0086] The present invention classifies target devices based on action collaboration characterization values. In actual situations, due to the different types of target devices, there are differences in action modes. Some devices have a strong collaborative relationship with the operator when they act. Therefore, the present invention scientifically divides the device categories based on the action collaboration characterization values. If the target device is in the collaborative relationship category, it is necessary to focus on its collaboration with the operator's action. By calibrating the collaborative associated action parts and analyzing the motion trajectory to determine whether there is an abnormality, a rapid abnormality judgment of the target device and the operator is achieved. If the target device is in the non-collaborative relationship category, it focuses on the matching of the operator's joint point action diagram with the violation sample library, and adopts a traversal analysis method to avoid missing abnormal features. Therefore, this monitoring strategy can ensure the reasonable allocation and utilization of system resources, ensure reliability, and improve the efficiency of monitoring a large number of equipment in the workshop.

[0087] Specifically, the monitoring and warning module is used to determine the moving parts of the target device in the video segment, and to mark the action parts that are cooperatively associated with the moving parts, including:

[0088] To determine the motion part that is in a cooperative relationship with the moving part;

[0089] It is used to mark the action part and synchronously call the motion trajectory set corresponding to the collaborative association.

[0090] It is understandable that there may be multiple action parts that are cooperatively associated with the moving parts. Therefore, it is necessary to identify the categories corresponding to the action parts, for example, identify the arm action parts, leg action parts and head action parts, and then perform calibration.

[0091] There is no limitation on the method of identifying the action part category. You can use an existing open source model or train an image processing model that can achieve the corresponding function to achieve the corresponding function. This will not be repeated here.

[0092] See also Figure 4 and Figure 5 As shown, Figure 4 This is a logic determination diagram for monitoring whether there is an abnormality in the image data of the area where the target device of the collaborative relationship category is located according to an embodiment of the invention. Figure 5 This is a logic determination diagram for monitoring whether there is an abnormality in the image data of the area where the target device of the non-cooperative relationship category is located according to an embodiment of the invention. Specifically, the monitoring and early warning module is used to determine whether the video segment meets the collaborative action standard based on the motion trajectory of the moving component and the action part, so as to determine whether there is an abnormality, including:

[0093] Used to extract the motion trajectory of the moving parts and the motion trajectory of the calibrated action parts respectively;

[0094] Comparing the extracted motion trajectory of the moving part with the motion trajectory of the action part in the motion trajectory set to calculate the similarity of the action part trajectory;

[0095] Comparing the extracted motion trajectory of each action part with the motion trajectory of the corresponding action part in the motion trajectory set to calculate the similarity of the action part trajectory;

[0096] If the coordinated action criteria are met, it is determined that there is no abnormality;

[0097] If the coordinated action standards are not met, it is determined to be abnormal;

[0098] The collaborative action standard is that the trajectory similarity of the action components is greater than a predetermined standard trajectory similarity threshold, and the trajectory similarity of each action part is greater than a predetermined standard trajectory similarity threshold.

[0099] It is understandable that there may be multiple marked action parts, and the motion trajectories of the action parts need to be extracted. Similarly, the motion trajectory corresponding to each action part needs to be compared with the motion trajectory of the action parts in the motion trajectory set. This will not be repeated here.

[0100] In practice, the purpose of setting the standard trajectory similarity threshold is to characterize the situation of similar motion trajectories. The standard trajectory similarity threshold is determined based on the standard value of action collaborative representation. In actual situations, a certain error can be allowed. The standard trajectory similarity threshold is set to 0.85 times the standard value of action collaborative representation.

[0101] The present invention calibrates the action parts that cooperate with the moving parts, and determines whether the video segment meets the collaborative action standard based on the motion trajectory of the moving parts and the action parts. In actual situations, for the target equipment of the collaborative relationship category, when it is operated, there is a collaborative relationship between the moving parts and the moving parts of the operator. The present invention identifies the action parts that are collaboratively associated with the motion parts based on the collaborative association relationship determined in advance, and then compares them. It has low computing power consumption and can quickly determine whether there are abnormalities between the target equipment and the operator based on the collaborative relationship. It can timely discover abnormal situations in the equipment of the collaborative relationship category, ensure reliability, and improve the efficiency of monitoring a large number of equipment in the workshop.

[0102] Specifically, the monitoring and early warning module is used to extract the operator's joint action graphs in the video frames one by one, and match them with the illegal joint action graphs in the illegal sample library to determine whether there are any abnormalities, including:

[0103] Used to extract the operator's joint coordinate information and construct a joint action graph;

[0104] Comparing the joint action graph with the action graphs of each illegal joint point in the illegal sample library to determine the action similarity;

[0105] If the action similarity between the key point action graph and any illegal joint point action graph in the illegal sample library is greater than the preset action similarity threshold, it is determined that an anomaly exists.

[0106] In implementation, there is no limitation on the method of constructing the joint action graph. The video frames can be analyzed through the existing video processing model to identify the positions of the joints of the operator's body. After extracting their coordinate information, the joints are connected according to the human body structure model to construct the joint action graph. This is an existing technology and will not be repeated here.

[0107] Specifically, the violation sample library is pre-built, and the violation joint action graphs corresponding to several violation actions can be pre-determined, which will not be repeated here.

[0108] Specifically, there is no limitation on the method of identifying the action similarity between joint action graphs. For example, the coordinate comparison can be used to scale the joint action graph to the same scale, and the distances between the corresponding coordinate points in the two key point action graphs are compared (for example, hand joints and hand joints), and the mean of the distances is solved. The inverse of the mean distance is determined as the action similarity. Of course, other methods can also be used. For example, the cosine similarity is calculated after vectorizing the joint action graph, and the cosine similarity is determined as the action similarity between the joint action graphs.

[0109] In implementation, the preset action similarity threshold is predetermined, wherein,

[0110] The technicians in this field were asked to imitate the same action several times to obtain the joint action graph, and the average action similarity between the joint action graphs was calculated. The action similarity threshold was set to 0.75 times the action similarity average.

[0111] Specifically, the monitoring and early warning module is further used to store video segments or video frames that are determined to be abnormal.

[0112] Thus far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present invention.

Claims

1. An intelligent monitoring and management system for a production workshop, characterized in that: include: A monitoring data acquisition module, which is used to collect image data of the area where several target devices are located; a collaborative analysis module, connected to the monitoring data acquisition module, configured to extract image data containing the target device and the operator, and perform motion collaborative analysis on the image data, including determining motion trajectories of various motion parts of the operator when the moving parts of the target device perform the same motion, and calculating motion collaborative representation values ​​for the motion parts based on the motion trajectories of the various motion parts; a classification module connected to the collaboration analysis module, configured to classify target devices based on the action collaboration representation value; A monitoring and early warning module, which is connected to the monitoring data acquisition module, the collaborative analysis module and the classification module, is used to monitor the image data of the area where the target device is located based on the category of the target device, including: Determining a moving part of a target device within a video segment, calibrating an action portion that is cooperatively associated with the moving part, and determining whether the video segment meets a cooperative action standard based on the motion trajectories of the moving part and the action portion to determine whether there is an anomaly; Alternatively, the operator's joint action graphs in the video frames are extracted one by one and matched with the illegal joint action graphs in the illegal sample library to determine whether there is an abnormality.

2. The intelligent monitoring and management system for a production workshop according to claim 1 is characterized in that: The collaborative analysis module is used to perform action collaborative analysis, including: A plurality of video segments for determining the same action performed by a single moving part of the target device; Determine the key points of each action part in each video segment, and determine the action trajectory of each action part corresponding to the key points; It is used to calculate the average similarity between the motion trajectories of the same motion part, and set the average similarity as the motion collaborative representation value of the motion part.

3. The intelligent monitoring and management system for a production workshop according to claim 2, characterized in that: The classification module is used to classify the target device based on the action collaborative representation value, including: If the action collaborative representation value of any action part is greater than or equal to the preset action collaborative representation standard value, the target device is determined to be in the collaborative relationship category; If the action cooperative representation value of any action part is less than the preset action cooperative representation standard value, the target device is determined to be in a non-cooperative relationship category.

4. The intelligent monitoring and management system for a production workshop according to claim 3 is characterized in that: The collaborative analysis module is also used to determine the collaborative relationship between the moving parts of the target device and the operator's action parts. If the motion synergy characterization value of the motion part is greater than or equal to a preset motion synergy characterization standard value, it is determined that there is a synergy association between the motion part of the operator and the moving component of the target device.

5. The intelligent monitoring and management system for a production workshop according to claim 4 is characterized in that: The collaborative analysis module is further configured to record the motion trajectory of the action portion and the motion trajectory of the action component in the video segment, and store them in the motion trajectory set corresponding to the collaborative association.

6. The intelligent monitoring and management system for a production workshop according to claim 1, characterized in that: The monitoring and early warning module is used to monitor the image data of the area where the target device is located based on the category of the target device, including: If the target device is in a collaborative relationship category, the moving parts of the target device in the video segment are determined, the action part that cooperates with the moving parts is calibrated, and based on the motion trajectories of the moving parts and the action part, whether the video segment meets the collaborative action standard is determined to determine whether there is any abnormality; If the target device belongs to the non-cooperative relationship category, the operator's joint action graphs in the video frames are extracted one by one and matched with the illegal joint action graphs in the illegal sample library to determine whether there is an abnormality.

7. The intelligent monitoring and management system for a production workshop according to claim 3, characterized in that: The monitoring and early warning module is used to determine the moving parts of the target device in the video segment, and to mark the action parts that are cooperatively associated with the moving parts, including: To determine the motion part that is in a cooperative relationship with the moving part; It is used to mark the action part and synchronously call the motion trajectory set corresponding to the collaborative association.

8. The intelligent monitoring and management system for a production workshop according to claim 7, characterized in that: The monitoring and early warning module is used to determine whether the video segment meets the collaborative action standard based on the motion trajectory of the moving component and the action part, so as to determine whether there is an abnormality, including: Used to extract the motion trajectory of the moving parts and the motion trajectory of the calibrated action parts respectively; Comparing the extracted motion trajectory of the moving part with the motion trajectory of the action part in the motion trajectory set to calculate the similarity of the action part trajectory; Comparing the extracted motion trajectory of each action part with the motion trajectory of the corresponding action part in the motion trajectory set to calculate the similarity of the action part trajectory; If the coordinated action criteria are met, it is determined that there is no abnormality; If the coordinated action standards are not met, it is determined to be abnormal; The collaborative action standard is that the trajectory similarity of the action components is greater than a predetermined standard trajectory similarity threshold, and the trajectory similarity of each action part is greater than a predetermined standard trajectory similarity threshold.

9. The intelligent monitoring and management system for a production workshop according to claim 1, characterized in that: The monitoring and early warning module is used to extract the operator's joint action diagrams in the video frames one by one, and match them with the illegal joint action diagrams in the illegal sample library to determine whether there are any abnormalities, including: Used to extract the operator's joint coordinate information and construct a joint action graph; Comparing the joint action graph with the action graphs of each illegal joint point in the illegal sample library to determine the action similarity; If the action similarity between the key point action graph and any illegal joint point action graph in the illegal sample library is greater than the preset action similarity threshold, it is determined that an anomaly exists.

10. The intelligent monitoring and management system for a production workshop according to claim 1, characterized in that: The monitoring and early warning module is further used to store video segments or video frames that are determined to be abnormal.

Images