Production line safety monitoring method and system

Abstract

The application relates to a production line safety monitoring method and system, the method comprising: acquiring a monitoring picture corresponding to a station of a production line, detecting and tracking personnel appearing in the monitoring picture; determining whether a tracking loss occurs in the process of tracking a target object; if it is determined that the tracking loss occurs in the process of tracking the target object, determining whether the creation of the target object is outside a protection area; wherein the protection area is a sub-area of a station entrance and exit; if it is determined that the creation of the target object is outside the protection area, triggering a counter value to be increased by 1; under the condition that the counting value of the counter is greater than 0, triggering an alarm and prohibiting resetting of the production line; and the safety of the production line is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of production line safety protection, in particular to a production line safety monitoring method and system. BACKGROUND

[0002] In the automatic production line scene, the assembly of parts is mainly performed by mechanical devices. A work station is arranged on the production line, and the work station includes a man-machine interaction area. When the mechanical device needs to be set or needs to be repaired due to a fault, the production line will be stopped, and relevant personnel will enter the man-machine interaction area of the work station to set the parameters of the mechanical device or repair the fault. After the personnel leave the man-machine interaction area, the production line is reset and the mechanical device is started.

[0003] In order to protect the safety of personnel, safety protection measures need to be configured for the production line. Related technologies provide a personnel access monitoring and alarm system for a production line workshop. The system obtains statistical information of personnel access in the production line workshop through a binocular counting monitoring device and an internal preset personnel counting software system. According to the obtained statistical information, a personnel monitoring and early warning software system determines whether there is still personnel remaining in the production line workshop, and controls whether the production line workshop device is started and whether an alarm is sent. Specifically, when the monitoring device detects that a personnel appears in the work station picture, the counting value is increased by 1, and when the monitoring device detects that the personnel disappears from the work station picture, the counting value is decreased by 1. Finally, whether to trigger an alarm is determined according to the counting value.

[0004] However, when personnel enter the monitoring area, due to problems such as object obstruction, personnel hiding, and algorithm detection failure, the personnel disappear in the picture captured by the monitoring device, and the work station is reset and the production line is restarted, which has a great safety hazard.

[0005] At present, there is no effective solution to the problem of insufficient safety of the automatic production line. SUMMARY

[0006] Therefore, it is necessary to provide a production line safety monitoring method and system capable of improving the safety of the automatic production line to solve the above technical problems.

[0007] In a first aspect, the present application provides a production line safety monitoring method. The method comprises:

[0008] obtaining a monitoring picture corresponding to a work station of a production line, detecting and tracking personnel appearing in the monitoring picture;

[0009] determining whether a tracking loss occurs in the process of tracking the target object;

[0010] If it is determined that a tracking loss occurred during the tracking of the target object, then it is determined whether the target object was created outside the protected area; wherein, the protected area is a sub-area of ​​the workstation entrance / exit.

[0011] If it is determined that the creation of the target object is outside the protected area, the counter value is incremented by 1.

[0012] If the counter value is greater than 0, an alarm is triggered and resetting the production line is prohibited.

[0013] In some embodiments, if it is determined that tracking loss has occurred during the tracking of the target object, the method further includes:

[0014] If it is determined that the creation of the target object is within the protected zone, the current count value is maintained.

[0015] In some embodiments, the method further includes:

[0016] If it is determined that no tracking loss occurred during the tracking of the target object, then it is determined whether the target object is currently outside the protected area;

[0017] If it is determined that the target object is currently outside the protected zone, then it is determined whether the creation of the target object occurred outside the protected zone;

[0018] If it is determined that the creation of the target object is within the protected area, the counter value is decremented by 1.

[0019] In some embodiments, after triggering the decrement of the counter value by 1, the method further includes:

[0020] If the counter value is not greater than 0, the alarm is deactivated and the production line is allowed to be reset.

[0021] In some embodiments, if it is determined that no tracking loss occurred during the tracking of the target object, and the target object is currently outside the protected zone, the method further includes:

[0022] Determine whether the creation of the target object occurred outside the protected area;

[0023] If it is determined that the creation of the target object is outside the protected zone, the current count value is maintained.

[0024] In some embodiments, if it is determined that no tracking loss occurred during the tracking of the target object, and the target object is currently within the protected area, the method further includes:

[0025] maintain the current count value.

[0026] In some embodiments, determining whether the target object is outside the protection zone at the time of creation includes extracting a first attribute of the target object and determining a relative position relationship between the target object and the protection zone at the time of creation according to the first attribute.

[0027] Alternatively, determining whether the target object is currently outside the protection zone includes extracting a second attribute of the target object and determining a relative position relationship between the target object and the protection zone at the current time according to the second attribute.

[0028] In some embodiments, detecting and tracking the personnel appearing in the monitoring picture includes:

[0029] extracting position information of the target object in the corresponding detection frame for the detected target object;

[0030] if the detected target object is a new target object, setting the first attribute according to the position information;

[0031] if the detected target object is not a new target object, setting the second attribute according to the position information.

[0032] In some embodiments, before extracting the position information of the target object in the corresponding detection frame for the detected target object, the method further includes:

[0033] for the detected current detection frame, determining whether there is a historical detection frame matching the current detection frame in the database; wherein the historical detection frame corresponds to a detected historical target object;

[0034] if it is determined that there is a historical detection frame matching the current detection frame in the database, determining that the target object in the current detection frame is not a new target object, and adding the current detection frame to the data queue in which the historical detection frame is located;

[0035] if it is determined that there is no historical detection frame matching the current detection frame in the database, determining that the target object in the current detection frame is a new target object, and creating an identity for the new target object.

[0036] In a second aspect, the application also provides a production line safety monitoring system, comprising: an image acquisition module and a control module, wherein the image acquisition module and the control module are connected.

[0037] The image acquisition module is configured to acquire a monitoring picture corresponding to a station of a production line and send the monitoring picture to the control module.

[0038] The control module is used for executing the production line safety monitoring method in the first aspect.

[0039] The production line safety monitoring method and system can detect and track the monitoring picture of the production line station, combine the current state and the historical creation state of the target object, determine whether the target object belongs to the case that the personnel normally enter the protection area but abnormally disappear in the protection area, trigger an alarm if the target object belongs to the case, and prohibit resetting the production line to avoid starting the mechanical device in the case that the personnel stay but disappear in the monitoring picture, thereby improving the protection safety of the production line. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 FIG. 1 is a structural block diagram of a production line safety monitoring system in one embodiment;

[0041] Figure 2 FIG. 2 is a hardware structural block diagram of a control module in one embodiment;

[0042] Figure 3 FIG. 3 is a structural block diagram of a production line safety monitoring system in another embodiment;

[0043] Figure 4 FIG. 4 is a spatial layout plane schematic diagram of a production line station in one embodiment;

[0044] Figure 5 FIG. 5 is a flowchart of a production line safety monitoring method in one embodiment;

[0045] Figure 6 FIG. 6 is a schematic diagram of a personnel activity state in a monitoring picture in one embodiment;

[0046] Figure 7 FIG. 7 is a flowchart of a production line safety monitoring method in another embodiment;

[0047] Figure 8 FIG. 8 is an interface schematic of a second terminal in one embodiment Figure 1 ;

[0048] Figure 9 FIG. 9 is an interface schematic of a second terminal in another embodiment Figure 2 ;

[0049] Figure 10 FIG. 10 is an internal structural diagram of a computer device in one embodiment. DETAILED DESCRIPTION

[0050] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not intended to limit the present application.

[0051] Unless otherwise defined, technical terms or scientific terms used in the present application shall have the general meaning understood by a person skilled in the art to which the present application belongs. In the present application, the terms "one", "a", "an", "the", "these", and similar words do not represent a quantitative limitation, and they can be singular or plural. In the present application, the terms "include", "contain", "have" and any variants thereof are intended to cover non-exclusive inclusion; for example, a process, method and system, product or device containing a series of steps or modules (units) are not limited to the listed steps or modules (units), but can include steps or modules (units) not listed, or can include other steps or modules (units) inherent to the process, method, product or device. In the present application, the terms "connected", "connected", "coupled" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. In the present application, "multiple" refers to two or more. The association between the associated objects is described by the term "and / or", which means that there can be three relationships, for example, "A and / or B" can mean that A exists alone, A and B exist together, and B exists alone. In general, the character " / " represents an "or" relationship between the objects before and after it. In the present application, the terms "first", "second", "third" and the like are only used to distinguish similar objects, and do not represent a specific order for the objects.

[0052] In one embodiment, Figure 1 A production line safety monitoring system is provided, which comprises an image acquisition module 1 and a control module 2, and the image acquisition module 1 and the control module 2 are connected; wherein the image acquisition module 1 is used to acquire a monitoring picture corresponding to a station of a production line, and send the monitoring picture to the control module 2. The image acquisition module 1 can be a monitoring device such as a monocular camera or a binocular camera. The control module 2 is used to execute a production line safety monitoring method. The control module 2 can be a terminal, a computer or a similar computing device.

[0053] The image acquisition module 1 can refer to an image sensor, or a camera containing an image sensor. The control module 2 can refer to a processing chip, or a complete terminal containing a processing chip. The image acquisition module 1 and the control module 2 can be distributed in different devices, or integrated in the same device, which is not limited by the present embodiment.

[0054] Figure 2This is a hardware structure block diagram of control module 2 in this embodiment. For example... Figure 2 As shown, control module 2 may include one or more ( Figure 2 Only one is shown in the diagram. A processor 201 and a memory 202 for storing data are also included. The processor 201 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The control module 2 may also include a transmission device 203 for communication functions and an input / output device 204. Those skilled in the art will understand that… Figure 2 The structure shown is for illustrative purposes only and does not limit the structure of the control module 2 described above. For example, the control module 2 may also include components that are more... Figure 2 The more or fewer components shown, or having the same Figure 2 The different configurations shown are illustrated.

[0055] The memory 202 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the production line safety monitoring method in this embodiment. The processor 201 executes various functional applications and data processing by running the computer program stored in the memory 202, thereby implementing the aforementioned method. The memory 202 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 202 may further include memory remotely located relative to the processor 201, and these remote memories can be connected to the control module 2 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0056] The transmission device 203 is used to receive or send data via a network. This network includes a wireless network provided by the communication provider of the control module 2. In one example, the transmission device 203 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 203 can be a Radio Frequency (RF) module for wireless communication with the Internet.

[0057] In some embodiments, Figure 3Another schematic diagram of a production line safety monitoring system is provided, which comprises a monitoring device 10, a first terminal 21, a second terminal 22 and a programmable logic controller 30 (PLC). The output of the monitoring device 10 is connected to the input of the first terminal 21 and the input of the second terminal 22, the output of the first terminal 21 is connected to the input of the second terminal 22 and the input of the programmable logic controller 30.

[0058] The monitoring device 10 is configured to collect monitoring pictures corresponding to the stations of the production line and push the encoded stream to the first terminal 21 and the second terminal 22. The first terminal 21 receives the stream output by the monitoring device, decodes the stream and executes an image processing algorithm to output the result of whether to trigger an alarm. The second terminal 22 receives the stream output by the monitoring device, decodes the stream and outputs a video. The second terminal 22 is also configured to receive the result of the image processing algorithm of the first terminal 21 and output an alarm signal according to the result of the image processing algorithm, where the alarm signal can be a visual alarm, for example, a light or a sound. Figure 8 Or Figure 9 The programmable logic controller 30 is controlled by the second terminal 22 and is configured to start, stop or reset the production line. The devices can be communicatively or electrically connected, which is not limited in the embodiment.

[0059] Figure 4 The space layout of the stations of the production line in an embodiment of the application is shown in the schematic diagram. As viewed from the monitoring pictures, an entrance is arranged in the station area, the station is semi-closed and has one or more entrances. The number of the monitoring devices 10 is determined according to the number of the entrances of the stations, as long as the monitoring devices can cover the key areas of the entrances of the stations. A pre-defined protection zone is arranged in the entrance of the station, which is a man-machine interaction area and is used for operating a mechanical device (for example, a mechanical arm). A person must pass through the protection zone to enter the inside of the station. When the mechanical device needs to be set or needs to be repaired due to a fault, the control module 2 sends a signal to the PLC to stop the production line, a relevant person enters the protection zone of the station to set the parameters of the mechanical device or repair the fault, and after the person leaves the protection zone, the control module 2 sends a signal to the PLC to reset the production line and start the mechanical device.

[0060] In the conventional safety protection measure, a monitoring device is installed above the station of the production line to detect the station by machine vision. When the monitoring device detects a person in the picture of the station, the count value is increased by 1, and when the monitoring device detects that the person disappears from the picture of the station, the count value is decreased by 1. Finally, whether to trigger an alarm is determined according to the count value. However, when a person enters the monitoring area, the person disappears in the picture captured by the monitoring device due to object blocking, person hiding and algorithm detection failure, and thus the production line is reset due to the false judgment that the station is empty, which has a great safety hazard.

[0061] To address the above problems, in one embodiment, such as Figure 5 As shown, a production line safety monitoring method is provided. This method can be applied to the control module 2 in the above embodiments or the production line safety monitoring system including the control module 2, and includes the following steps:

[0062] Step S101: Obtain the monitoring screen corresponding to the workstation on the production line, and detect and track the personnel appearing on the monitoring screen.

[0063] Image acquisition module 1 is deployed above the workstation entrance / exit to monitor the area and obtain surveillance footage. Control module 2 receives the surveillance video acquired by image acquisition module 1 and detects and tracks personnel appearing in the surveillance footage. Control module 2 has a built-in detection and tracking algorithm. After the detector identifies the surveillance footage, it outputs detection boxes. Each detection box represents an instance of a target object (i.e., a person) and carries the target object's location information. Based on this location information, the relative positional relationship between the target object and the protected area can be determined.

[0064] The tracker receives detection boxes output by the detector and tracks the target object. For each detected detection box, the tracker checks if a matching historical detection box exists in the database; each historical detection box corresponds to a previously detected target object. If a matching historical detection box exists in the database, the target object in the current detection box is determined not to be a new target object, and the current detection box is added to the data queue of historical detection boxes. Conversely, if no matching historical detection box exists in the database, the target object in the current detection box is determined to be a new target object, and an identity is created for the new target object.

[0065] Step S102: Determine whether tracking loss has occurred during the tracking of the target object.

[0066] Step S103: If it is determined that a tracking loss has occurred during the tracking of the target object, then determine whether the creation of the target object is outside the protection zone; where the protection zone is a sub-area of ​​the workstation entrance and exit.

[0067] When a certain target object is detected in the monitoring picture, if the target object is not identified within a preset time or a preset number of video frames, it is determined that the target object tracking is lost. The target object tracking loss corresponds to two cases: the target object may have left the workstation; or the target object may still be in the workstation, but due to problems such as object occlusion, personnel hiding, algorithm detection failure, etc., the personnel disappear in the picture captured by the monitoring device. In order to identify the two cases, the embodiment further determines whether the creation of the target object is outside the protection area, that is, whether the target object normally enters from the entrance and exit of the workstation.

[0068] In step S104, if it is determined that the creation of the target object is outside the protection area, the counter value is triggered to increase by 1.

[0069] If the creation of the target object is outside the protection area, and subsequent tracking loss occurs, it corresponds to the case that the personnel normally enter the protection area but abnormally disappear in the protection area. For this case, the number of people needs to be reported, and the counter value is increased by 1.

[0070] In step S105, if the counter value is greater than 0, an alarm is triggered, and the production line is prohibited from being reset.

[0071] The counter value is determined to be greater than 0. If the final counter value is greater than 0, it indicates that there is personnel retention in the protection area. At this time, triggering an alarm can prompt the relevant personnel that the production line is not allowed to be reset, and the start of the mechanical device is prohibited.

[0072] In the above steps S101 to S106, by detecting and tracking the monitoring picture of the workstation of the production line, in combination with the current state and historical creation state of the target object, it is determined whether the target object belongs to the case that the personnel normally enter the protection area but abnormally disappear in the protection area. If it is determined that the target object belongs to this case, an alarm is triggered, the production line is prohibited from being reset, and the mechanical device is started in the case of personnel retention but disappearance in the monitoring picture, thereby improving the protection safety of the production line.

[0073] The alarm strategy adopted by the related art is that when a target object is detected in a monitoring picture, an alarm is triggered. However, this method has problems of resource waste and insufficient alarm effectiveness in specific application scenarios. Specifically, in a production line safety monitoring system, a monitoring platform is usually configured to play monitoring pictures of each workstation in real time. When the personnel can be clearly observed by the monitoring picture, the on-site state can be judged autonomously and misoperation can be avoided. At this time, if an alarm is triggered due to the detection of personnel, the alarm signal may be redundant, causing unnecessary consumption of system resources.

[0074] On the contrary, when there is actually a person staying at the station, but the person is not visible in the monitoring picture due to being in a monitoring blind area or being blocked, etc., the worker may misjudge that the station is unattended, and then reset the production line, causing a safety hazard. In view of this scenario, the embodiment identifies the missing state of the target object in the monitoring picture, and combines the preset logic to determine whether the alarm condition is met, and then triggers an alarm prompt when necessary.

[0075] The alarm strategy of the embodiment focuses on the safety risks caused by the lack of monitoring visibility, can provide effective warning at key links, avoid safety accidents caused by misoperation, and has higher safety and resource utilization compared with the way of taking the appearance of the target as the alarm triggering condition in the related art.

[0076] In addition to the above-mentioned object blocking, personnel hiding, algorithm detection failure, which causes the person to disappear in the picture captured by the monitoring device, when the protective area borders the edge of the monitoring field of view of the image acquisition module 1, it will also cause the person to disappear in the monitoring picture. This also conforms to the case that the person normally enters the protective area, but abnormally disappears in the protective area, and the production line safety monitoring method of the embodiment can be used to solve it.

[0077] In order to make the production line safety monitoring method of the present application more clear and easy to understand, in one embodiment, Figure 6 A schematic diagram of personnel activity state in a monitoring picture is provided, which shows a plurality of personnel activity states and corresponding counting actions, as follows:

[0078] Personnel activity state 1: When the personnel normally enter and exit the protective area, the number of people is not reported, and the current count value is maintained. Among them, the personnel normally enter and exit the protective area means that the personnel enter the protective area from the entrance and exit of the station, and then leave the protective area.

[0079] Personnel activity state 2: When the personnel normally enter the protective area, but abnormally disappear in the protective area, the number of people is reported, and the count value is +1. For example, due to object blocking, personnel hiding, algorithm detection failure, which causes the person to disappear in the picture captured by the monitoring device. Among them, the personnel abnormally disappear in the protective area means that the personnel enter the protective area from the entrance and exit of the station, but then disappear in the monitoring picture.

[0080] Personnel activity state 3: When the personnel normally enter the protective area, but abnormally disappear in the protective area, the number of people is reported, and the count value is +1. For example, when the protective area borders the edge of the monitoring field of view of the image acquisition module 1, it will also cause the person to disappear in the monitoring picture.

[0081] Personnel activity state 4: when the personnel abnormally appears in the protection area and normally leaves the protection area, the number of people is reported, and the counter value is -1. Wherein, the personnel abnormally appears in the protection area refers to that the position of the personnel first appearing in the monitoring picture is in the protection area. In some scenes, the other end of the work station (far away from the entrance and exit of the work station) is connected to other workshops, and personnel can enter the work station from other workshops, so the personnel abnormally appears in the protection area.

[0082] Personnel activity state 5: when the personnel abnormally appears in the protection area and then abnormally disappears in the protection area, the number of people is not reported, and the current counter value is maintained.

[0083] Personnel activity state 6: when the personnel appears in the monitoring picture and does not leave the protection area, the number of people is not reported, and the current counter value is maintained.

[0084] In an embodiment, in the step S103 described above, if it is determined that tracking loss occurs in the process of tracking the target object, the method further comprises: if it is determined that the creation of the target object is within the protection area, the current counter value is maintained.

[0085] Reference Figure 6 If the creation of the target object is within the protection area and subsequent tracking loss occurs, it corresponds to the personnel activity state 5 described above, that is, the personnel abnormally appears in the protection area and then abnormally disappears in the protection area. For this kind of situation, it is possible that the personnel shuttle in and out of the edge of the protection area, at this time, reporting the number of people will make the system fall into a dead loop or even crash, therefore, in this kind of situation, the number of people is not reported, and the current counter value is maintained.

[0086] In an embodiment, after the step S102 described above, the method further comprises: if it is determined that tracking loss does not occur in the process of tracking the target object, it is determined whether the target object is currently outside the protection area; if it is determined that the target object is currently outside the protection area, it is determined whether the creation of the target object is outside the protection area; if it is determined that the creation of the target object is within the protection area, the counter value is triggered to be reduced by 1.

[0087] Reference Figure 6 If the creation of the target object is within the protection area, and the current target object does not appear tracking loss (that is, the target object appears in the monitoring picture), but is outside the protection area (that is, the target object leaves the protection area), it corresponds to the personnel activity state 4 described above, that is, when the personnel abnormally appears in the protection area and normally leaves the protection area, the number of people is reported, and the counter value is -1.

[0088] Further, after triggering the counter value to be reduced by 1, the method further comprises: in the case that the counter value is not greater than 0, the alarm is removed, and the production line is allowed to be reset.

[0089] In the embodiment, it is judged whether the count value of the counter is greater than 0. If the count value of the counter is equal to 0 finally, it indicates that no personnel stays in the protection area, and the alarm can be released to prompt the relevant personnel and allow the production line to be reset and the mechanical device to be released.

[0090] In one embodiment, after step S102, in the case that it is determined that no tracking loss occurs in the process of tracking the target object and the target object is currently outside the protection area, the method further comprises: judging whether the creation of the target object is outside the protection area; and if it is determined that the creation of the target object is outside the protection area, maintaining the current count value.

[0091] Reference Figure 6 If the creation of the target object is outside the protection area and the current target object does not occur tracking loss but is outside the protection area, it corresponds to the personnel activity state 1, i.e., when the personnel normally enters and exits the protection area, the number of people is not reported, and the current count value is maintained.

[0092] In one embodiment, after step S102, in the case that it is determined that no tracking loss occurs in the process of tracking the target object and the target object is currently inside the protection area, the method further comprises: maintaining the current count value.

[0093] Reference Figure 6 If the target object does not occur tracking loss but is outside the protection area, it corresponds to the personnel activity state 6, i.e., when the personnel is displayed on the monitoring screen and does not leave the protection area, the number of people is not reported, and the current count value is maintained.

[0094] In some embodiments, step S101 further comprises: for the detected target object, extracting position information of the target object in the corresponding detection frame; if the detected target object is a new target object, setting a first attribute according to the position information; and if the detected target object is not a new target object, setting a second attribute according to the position information. The first attribute is used to represent the relative position relationship between the target object at the creation moment and the protection area, and the second attribute is used to represent the relative position relationship between the target object at any moment after the creation and the protection area.

[0095] After the attribute of the target object is set, the first attribute of the target object can be extracted subsequently, the relative position relationship between the target object at the creation moment and the protection area is determined according to the first attribute, and the judgment of whether the creation of the target object is outside the protection area is realized. Alternatively, the second attribute of the target object is extracted, the relative position relationship between the target object at the current moment and the protection area is determined according to the second attribute, and the judgment of whether the target object is currently outside the protection area is realized.

[0096] In the embodiment, the detector receives the monitoring picture, and outputs a detection frame after identifying the monitoring picture, each detection frame representing an instance of a target object (i.e., a person), and the detection frame carries position information of the target object, based on which the relative position relationship between the target object and the protection zone can be determined. The tracker receives the detection frame output by the detector, and tracks the target object.

[0097] The position information carried by the detection frame when the target object appears for the first time can be used to determine whether the target object is created inside or outside the protection zone. If the target object is created outside the protection zone, it belongs to the case that the person normally enters the protection zone; if the target object is created inside the protection zone, it belongs to the case that the person abnormally appears in the protection zone.

[0098] The position information carried by the detection frame when the target object appears for the first time and the last time can be used to determine whether the target object abnormally disappears in the protection zone. If the target object is created outside the protection zone and then lost in the protection zone, it belongs to the case that the person abnormally disappears in the protection zone.

[0099] It should be understood that, although each step in the flowchart involved in each embodiment as described above is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.

[0100] In one embodiment, Figure 7 Another flowchart of a production line safety monitoring method is provided, which includes the following steps:

[0101] In step S201, the detector receives the monitoring video, and detects the monitoring picture. Specifically, the detector identifies the monitoring picture and outputs a detection frame, each detection frame representing an instance of a target object (i.e., a person), and the detection frame carries position information of the target object, based on which the relative position relationship between the target object and the protection zone can be determined.

[0102] Step S202, the tracker receives the detection frame output by the detector, and tracks the target object. Specifically, the tracker determines whether there is a historical detection frame matching the current detection frame in the database; wherein the historical detection frame corresponds to a historical target object that has been detected. If it is determined that there is a historical detection frame matching the current detection frame in the database, it is determined that the target object in the current detection frame is not a new target object, and the current detection frame is added to the data queue in which the historical detection frame is located. On the contrary, if it is determined that there is no historical detection frame matching the current detection frame in the database, it is determined that the target object in the current detection frame is a new target object, and an identity is created for the new target object. According to the position information carried by each queue detection frame, a personnel movement trajectory can be generated.

[0103] Step S203, the tracker processes each tracked target object. For the detected target object, the position information of the target object is extracted in the corresponding detection frame; if the detected target object is a new target object, a first attribute is set according to the position information; if the detected target object is not a new target object, a second attribute is set according to the position information. Wherein, the first attribute is used to represent the relative position relationship between the target object at the creation time and the protection area, and the second attribute is used to represent the relative position relationship between the target object at any time after the creation and the protection area.

[0104] Step S204, it is determined whether the target object loss and the situation of about to be destroyed appear. If it is determined that yes, go to step S205; if it is determined that no, go to step S210.

[0105] Step S205, it is determined whether the target object is created outside the protection area. If it is determined that yes, go to step S206; if it is determined that no, end the current process. Wherein, the first attribute of the target object can be extracted, and the relative position relationship between the target object at the creation time and the protection area is determined according to the first attribute, so as to realize the judgment of whether the creation of the target object is outside the protection area.

[0106] Step S206, the number of people is reported +1.

[0107] Step S207, the area counter counts.

[0108] Step S208, the PLC controls the operation of the mechanical arm according to the counting value of the area counter. If the counting value of the area counter is greater than 0, the alarm is triggered, and the mechanical arm is prohibited to be reset. If the counting value of the area counter is equal to 0, the alarm is released, and the mechanical arm is allowed to be reset.

[0109] Step S209, the mechanical arm starts or stops according to the signal output by the PLC.

[0110] Step S210, determine whether the target object is currently outside the protection area. If the determination is yes, go to step S211; if the determination is no, end the current process. In this embodiment, the second attribute of the target object can be extracted, and the relative position relationship between the target object and the protection area at the current time can be determined according to the second attribute, so as to determine whether the target object is currently outside the protection area.

[0111] Step S211, determine whether the target object is outside the protection area when it is created. If the determination is yes, end the current process; if the determination is no, go to step S212.

[0112] Step S212, report the number of people -1, and go to step S207.

[0113] Figure 8 Interface diagram of the second terminal 22 in the embodiment Figure 1 Corresponding to the personnel activity state 6, when the personnel is displayed on the monitoring screen and does not leave the protection area, no alarm is triggered. Figure 9 Interface diagram of the second terminal 22 in the embodiment Figure 2 Corresponding to the personnel activity state 2 or the personnel activity state 3, when the personnel normally enters the protection area but abnormally disappears in the protection area, an alarm is triggered.

[0114] In this embodiment, the system independently judges and processes each person entering the protection area through the detector and the tracker. First, the system determines whether a person has entered the pre-set protection area. Once it is detected that a person enters the protection area, the system will determine whether the initial position of the target object is outside the protection area according to the time point when the target object is created (i.e. the time when the system first sees the person). If the target object is created outside the protection area, it means that the person normally enters the protection area.

[0115] As the tracking process proceeds, the system gradually accumulates the historical trajectory information of the person, forming a record of the moving path. Based on this information, the system can determine whether the person was outside the protection area when first seen. If the determination is yes, the person is assigned the corresponding attribute (i.e. the first attribute, such as “initially outside the protection area”), and the tracking continues.

[0116] In the subsequent tracking process, if the person temporarily “disappears” due to obstruction or leaving the field of view, but is still in the protection area, the system will not immediately alarm or report; only when the system observes that the person leaves the protection area in the state of continuous tracking, the “person leaves” event is triggered, and the number of people in the area is reduced by one.

[0117] On the contrary, if a person is outside the protected area when the system "first sees" him, and then is tracked into the protected area and disappears within the protected area (e.g. is obscured or stationary), the system will determine that a "person entered" and the abnormal disappearance within the protected area will add one to the number of people in the area, triggering an alarm.

[0118] Regarding the alarm mechanism, the general protected area alarm mode triggers an alarm as soon as someone enters the protected area. The mode adopted in this embodiment is triggered when "person abnormal disappearance" occurs. For example, a person enters the protected area and suddenly disappears from the field of view without being observed to leave, the system will determine that it is abnormal and thus issue an alarm. When all entering personnel are normally tracked and confirmed to leave, and the number of people in the area is zero, the system will automatically disarm the alarm.

[0119] Therefore, in the current mode, the system does not alarm for normal personnel activities within the protected area, and only triggers an alarm when unexplained disappearance behavior (i.e. abnormal stay or disappearance) occurs, thereby improving the accuracy and practicality of the alarm.

[0120] In one embodiment, a computer device, which can be a terminal, can have an internal structure diagram as shown in Figure 10 The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the computer device is used for wired or wireless communication with external terminals. Wireless communication can be achieved through WIFI, mobile cellular network, NFC (Near Field Communication), or other technologies. The computer program is executed by the processor to implement a production line safety monitoring method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball, or touchpad provided on the computer device housing. It can also be an external keyboard, touchpad, or mouse, etc.

[0121] Those skilled in the art can understand that Figure 10 the structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. A specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0122] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties.

[0123] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiments. Any reference to memory, database or other medium used in the embodiments provided by the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided by the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided by the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0124] The technical features of the above embodiments can be combined in any way. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present disclosure.

[0125] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A production line safety monitoring method characterized by, The method comprises: acquiring a monitoring picture corresponding to a station of a production line, detecting and tracking personnel appearing on the monitoring picture; judging whether a tracking loss occurs in the process of tracking a target object; if it is judged that a tracking loss occurs in the process of tracking the target object, judging whether the creation of the target object is outside a protection area; the protection area is a sub-area of a station entrance and exit; if it is judged that the creation of the target object is outside the protection area, triggering a counter value to increase by 1; if the counter value is greater than 0, triggering an alarm and prohibiting resetting the production line; wherein the detection and tracking of personnel appearing on the monitoring picture comprises: for the detected target object, extracting position information of the target object in a corresponding detection box; if the detected target object is a new target object, setting a first attribute according to the position information, the first attribute being used to represent the relative position relationship between the target object at the creation moment and the protection area; if the detected target object is not a new target object, setting a second attribute according to the position information, the second attribute being used to represent the relative position relationship between the target object at any moment after the creation and the protection area; wherein the judgment of whether the creation of the target object is outside the protection area comprises: extracting the first attribute of the target object, and determining the relative position relationship between the target object at the creation moment and the protection area according to the first attribute.

2. The production line safety monitoring method according to claim 1, characterized by, In the case where it is judged that a tracking loss occurs in the process of tracking the target object, the method further comprises: if it is judged that the creation of the target object is inside the protection area, maintaining the current counter value.

3. The manufacturing line safety monitoring method according to claim 1, characterized by, The method further comprises: if it is judged that a tracking loss does not occur in the process of tracking the target object, judging whether the target object is currently outside the protection area; if it is judged that the target object is currently outside the protection area, judging whether the creation of the target object is outside the protection area; if it is judged that the creation of the target object is inside the protection area, triggering the counter value to decrease by 1.

4. The production line safety monitoring method according to claim 3, characterized by, After triggering the counter value to decrease by 1, the method further comprises: if the counter value is not greater than 0, canceling the alarm and allowing resetting the production line.

5. The manufacturing line safety monitoring method according to claim 3, characterized by, In the case where it is judged that a tracking loss does not occur in the process of tracking the target object and the target object is currently inside the protection area, the method further comprises: judging whether the creation of the target object is outside the protection area; if it is judged that the creation of the target object is outside the protection area, maintaining the current counter value.

6. The manufacturing line safety monitoring method according to claim 3, characterized by, In the case where it is judged that a tracking loss does not occur in the process of tracking the target object and the target object is currently inside the protection area, the method further comprises: maintaining the current counter value.

7. The manufacturing line safety monitoring method according to claim 3, characterized by, The judgment of whether the target object is currently outside the protection area comprises: extracting the second attribute of the target object, and determining the relative position relationship between the target object and the protection zone at the current time according to the second attribute.

8. The manufacturing line safety monitoring method according to claim 1, characterized by, Before extracting the position information of the target object in the corresponding detection frame for the detected target object, the method further comprises: for the detected current detection frame, determining whether there is a historical detection frame matching the current detection frame in the database; wherein the historical detection frame corresponds to a detected historical target object; if it is determined that there is a historical detection frame matching the current detection frame in the database, it is determined that the target object in the current detection frame is not a new target object, and the current detection frame is added to the data queue where the historical detection frame is located; if it is determined that there is no historical detection frame matching the current detection frame in the database, it is determined that the target object in the current detection frame is a new target object, and an identity is created for the new target object.

9. A production line safety monitoring system characterized by, comprise: an image acquisition module and a control module, the image acquisition module and the control module being connected; wherein the image acquisition module is used to acquire a monitoring picture corresponding to a station of a production line, and send the monitoring picture to the control module; the control module is used to execute the production line safety monitoring method in any one of claims 1 to 8.

Images