Production line safety monitoring method and system

Abstract

The invention relates to a production line safety monitoring method and system, and the method comprises the steps: obtaining a monitoring image corresponding to a station of a production line, and carrying out the detection and tracking of a person appearing in the monitoring image; judging whether tracking loss occurs in the process of tracking the target object; if it is judged that tracking loss occurs in the process of tracking the target object, whether creation of the target object is outside a protection area is judged; wherein the protection area is a sub-area of a station entrance / exit; if it is judged that the creation of the target object is outside the protection area, a counter is triggered to add 1 to the numerical value; when the count value of the counter is greater than 0, triggering an alarm, and prohibiting resetting of the production line; and the protection safety of the production line is improved.

Description

Technical Field

[0001] This application relates to the field of production line safety protection technology, and in particular to a production line safety monitoring method and system. Background Technology

[0002] In automated production line scenarios, component assembly is primarily performed by mechanical devices. Workstations are set up on the production line, each containing a human-machine interface (HMI) area. When a mechanical device needs parameter settings or requires maintenance due to a malfunction, the production line stops, and relevant personnel enter the HMI area of ​​the workstation to set parameters or perform maintenance. After the personnel leave the HMI area, the production line is reset, and the mechanical device is restarted.

[0003] To ensure personnel safety, safety protection measures are required for the production line. Related technology provides a personnel entry / exit monitoring and alarm system for the production line workshop. This system uses binocular counting monitoring equipment and pre-installed personnel counting software to acquire statistical information on personnel entering and exiting the production line / workshop. Based on this statistical information, the personnel monitoring and early warning software system determines whether any personnel remain on the production line / workshop and controls whether the production line equipment starts and whether an alarm is triggered. Specifically, when the monitoring equipment detects personnel on the workstation screen, the count value is incremented by 1; when the monitoring equipment detects that the personnel disappear from the workstation screen, the count value is decremented by 1. Finally, the alarm is triggered based on the count value.

[0004] However, when personnel enter the monitored area, due to issues such as object obstruction, personnel hiding, and algorithm detection failure, personnel may disappear from the images captured by the monitoring equipment, leading to a misjudgment that the workstation is unoccupied and the production line being reset, posing a significant safety hazard.

[0005] Currently, no effective solution has been proposed to address the issue of insufficient safety in automated production lines. Summary of the Invention

[0006] Therefore, it is necessary to provide a production line safety monitoring method and system that can improve the safety of automated production lines, addressing the aforementioned technical problems.

[0007] Firstly, this application provides a method for monitoring safety on a production line. The method includes:

[0008] Acquire monitoring images of workstations corresponding to the production line, and detect and track personnel appearing in the monitoring images;

[0009] Determine whether tracking loss has occurred during the tracking of the target object;

[0010] If it is determined that a tracking loss occurred during the tracking of the target object, then it is determined whether the target object was created outside the protected area; wherein, the protected area is a sub-area of ​​the workstation entrance / exit.

[0011] If it is determined that the creation of the target object is outside the protected area, the counter value is incremented by 1.

[0012] If the counter value is greater than 0, an alarm is triggered and resetting the production line is prohibited.

[0013] In some embodiments, if it is determined that tracking loss has occurred during the tracking of the target object, the method further includes:

[0014] If it is determined that the creation of the target object is within the protected zone, the current count value is maintained.

[0015] In some embodiments, the method further includes:

[0016] If it is determined that no tracking loss occurred during the tracking of the target object, then it is determined whether the target object is currently outside the protected area;

[0017] If it is determined that the target object is currently outside the protected zone, then it is determined whether the creation of the target object occurred outside the protected zone;

[0018] If it is determined that the creation of the target object is within the protected area, the counter value is decremented by 1.

[0019] In some embodiments, after triggering the decrement of the counter value by 1, the method further includes:

[0020] If the counter value is not greater than 0, the alarm is deactivated and the production line is allowed to be reset.

[0021] In some embodiments, if it is determined that no tracking loss occurred during the tracking of the target object, and the target object is currently outside the protected zone, the method further includes:

[0022] Determine whether the creation of the target object occurred outside the protected area;

[0023] If it is determined that the creation of the target object is outside the protected zone, the current count value is maintained.

[0024] In some embodiments, if it is determined that no tracking loss occurred during the tracking of the target object, and the target object is currently within the protected area, the method further includes:

[0025] Maintain the current count value.

[0026] In some embodiments, determining whether the creation of the target object is outside the protection zone includes: extracting a first attribute of the target object, and determining the relative positional relationship between the target object and the protection zone at the time of creation based on the first attribute;

[0027] Alternatively, determining whether the target object is currently outside the protected zone includes: extracting a second attribute of the target object and determining the relative position of the target object to the protected zone at the current moment based on the second attribute.

[0028] In some embodiments, the detection and tracking of personnel appearing on the surveillance screen includes:

[0029] For the detected target object, extract the position information of the target object in the corresponding detection box;

[0030] If the detected target object is a new target object, then the first attribute is set according to the location setting information;

[0031] If the detected target object is not a new target object, then the second attribute is set according to the location information.

[0032] In some embodiments, before extracting the location information of the detected target object from the corresponding detection box, the method further includes:

[0033] For the currently detected bounding box, determine whether there is a matching historical bounding box in the database; wherein, the historical bounding box corresponds to a previously detected historical target object;

[0034] If it is determined that a matching historical detection box exists in the database, then the target object in the current detection box is determined not to be a new target object, and the current detection box is added to the data queue where the historical detection box is located;

[0035] If it is determined that there is no matching historical detection box in the database, then the target object in the current detection box is determined to be a new target object, and an identity identifier is created for the new target object.

[0036] Secondly, this application also provides a production line safety monitoring system, including: an image acquisition module and a control module, wherein the image acquisition module and the control module are connected;

[0037] The image acquisition module is used to acquire monitoring images of workstations corresponding to the production line and send the monitoring images to the control module.

[0038] The control module is used to execute the production line safety monitoring method described in the first aspect above.

[0039] The aforementioned production line safety monitoring method and system detects and tracks the monitoring screens of the production line workstations, and combines the current status and historical creation status of the target object to determine whether the target object belongs to the situation where personnel normally enter the protected area but abnormally disappear within the protected area. If the target object is determined to belong to this situation, an alarm is triggered, and the production line is prohibited from being reset. This avoids starting mechanical devices when personnel are left behind but disappear from the monitoring screen, thereby improving the safety of the production line. Attached Figure Description

[0040] Figure 1 This is a structural block diagram of a production line safety monitoring system in one embodiment;

[0041] Figure 2 This is a hardware structure block diagram of the control module in one embodiment;

[0042] Figure 3 This is a structural block diagram of a production line safety monitoring system in another embodiment;

[0043] Figure 4 This is a schematic plan view of the spatial layout of the production line workstations in one embodiment;

[0044] Figure 5 This is a flowchart illustrating a production line safety monitoring method in one embodiment;

[0045] Figure 6 This is a schematic diagram of the activity status of personnel in a monitoring screen in one embodiment;

[0046] Figure 7 This is a flowchart illustrating a production line safety monitoring method in another embodiment;

[0047] Figure 8 This is a schematic diagram of the interface of the second terminal in one embodiment. Figure 1 ;

[0048] Figure 9 This is a schematic diagram of the interface of the second terminal in one embodiment. Figure 2 ;

[0049] Figure 10 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0051] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.

[0052] In one embodiment, Figure 1 A production line safety monitoring system is provided, comprising: an image acquisition module 1 and a control module 2, which are connected together. The image acquisition module 1 is used to acquire monitoring images corresponding to workstations on the production line and send these images to the control module 2. The image acquisition module 1 can be a monitoring device such as a monocular camera or a binocular camera. The control module 2 is used to execute production line safety monitoring methods. The control module 2 can be a terminal, a computer, or a similar computing device.

[0053] Image acquisition module 1 can refer to an image sensor or a camera containing an image sensor. Control module 2 can refer to a processing chip or a complete terminal containing a processing chip. Image acquisition module 1 and control module 2 can be distributed in different devices or integrated in the same device; this embodiment does not impose any restrictions.

[0054] Figure 2This is a hardware structure block diagram of control module 2 in this embodiment. For example... Figure 2 As shown, control module 2 may include one or more ( Figure 2 Only one is shown in the diagram. A processor 201 and a memory 202 for storing data are also included. The processor 201 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The control module 2 may also include a transmission device 203 for communication functions and an input / output device 204. Those skilled in the art will understand that… Figure 2 The structure shown is for illustrative purposes only and does not limit the structure of the control module 2 described above. For example, the control module 2 may also include components that are more... Figure 2 The more or fewer components shown, or having the same Figure 2 The different configurations shown are illustrated.

[0055] The memory 202 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the production line safety monitoring method in this embodiment. The processor 201 executes various functional applications and data processing by running the computer program stored in the memory 202, thereby implementing the aforementioned method. The memory 202 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 202 may further include memory remotely located relative to the processor 201, and these remote memories can be connected to the control module 2 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0056] The transmission device 203 is used to receive or send data via a network. This network includes a wireless network provided by the communication provider of the control module 2. In one example, the transmission device 203 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 203 can be a Radio Frequency (RF) module for wireless communication with the Internet.

[0057] In some embodiments, Figure 3A schematic diagram of another production line safety monitoring system is provided. This system includes: a monitoring device 10, a first terminal 21, a second terminal 22, and a programmable logic controller (PLC) 30. The output of the monitoring device 10 is connected to the input of the first terminal 21 and the input of the second terminal 22. The output of the first terminal 21 is connected to the input of the second terminal 22 and the input of the programmable logic controller 30.

[0058] The monitoring device 10 is used to collect monitoring images of the workstations corresponding to the production line and stream them to the first terminal 21 and the second terminal 22 after encoding. The first terminal 21 receives the bitstream output by the monitoring device, decodes it, executes an image processing algorithm, and outputs a result indicating whether an alarm has been triggered. The second terminal 22 receives the bitstream output by the monitoring device, decodes it, and outputs video. The second terminal 22 is also used to receive the image processing algorithm result from the first terminal 21 and outputs an alarm signal based on the result. The alarm signal can be a visual alarm, as shown in the reference... Figure 8 or Figure 9 The programmable logic controller 30 is controlled by the second terminal 22 and is used to start, stop, or reset the production line. The devices can be connected via communication or electrical connections; this embodiment does not impose any restrictions.

[0059] Figure 4 This is a schematic plan view of the spatial layout of the production line workstations in one embodiment of this application. From the monitoring screen, entrances and exits are set up in the workstation area. The workstations are semi-enclosed and have one or more entrances and exits. The number of monitoring devices (10) depends on the number of entrances and exits, as long as they can cover the key areas of the workstation entrances and exits. A pre-defined protective zone is set up within the workstation entrances and exits. This protective zone is a human-machine interaction area used to operate mechanical devices (such as robotic arms). Personnel must pass through the protective zone to enter the workstation. When the mechanical device needs parameter setting or requires maintenance due to a malfunction, the control module 2 sends a signal to the PLC to stop the production line. Relevant personnel will enter the protective zone of the workstation to set parameters or perform maintenance on the mechanical device. After the personnel leave the protective zone, the control module 2 sends a signal to the PLC to reset the production line and restart the mechanical device.

[0060] Traditional safety measures involve installing monitoring equipment above workstations on the production line, using machine vision to detect personnel. When the monitoring equipment detects a person in the workstation's view, a counter is incremented; when the person disappears from the view, the counter is decremented. The final alarm is triggered based on the count. However, when personnel enter the monitored area, issues such as object obstruction, personnel hiding, and algorithm failure can cause them to disappear from the monitoring equipment's view. This can lead to a false alarm, causing the production line to be reset, posing a significant safety hazard.

[0061] To address the above problems, in one embodiment, such as Figure 5 As shown, a production line safety monitoring method is provided. This method can be applied to the control module 2 in the above embodiments or the production line safety monitoring system including the control module 2, and includes the following steps:

[0062] Step S101: Obtain the monitoring screen corresponding to the workstation on the production line, and detect and track the personnel appearing on the monitoring screen.

[0063] Image acquisition module 1 is deployed above the workstation entrance / exit to monitor the area and obtain surveillance footage. Control module 2 receives the surveillance video acquired by image acquisition module 1 and detects and tracks personnel appearing in the surveillance footage. Control module 2 has a built-in detection and tracking algorithm. After the detector identifies the surveillance footage, it outputs detection boxes. Each detection box represents an instance of a target object (i.e., a person) and carries the target object's location information. Based on this location information, the relative positional relationship between the target object and the protected area can be determined.

[0064] The tracker receives detection boxes output by the detector and tracks the target object. For each detected detection box, the tracker checks if a matching historical detection box exists in the database; each historical detection box corresponds to a previously detected target object. If a matching historical detection box exists in the database, the target object in the current detection box is determined not to be a new target object, and the current detection box is added to the data queue of historical detection boxes. Conversely, if no matching historical detection box exists in the database, the target object in the current detection box is determined to be a new target object, and an identity is created for the new target object.

[0065] Step S102: Determine whether tracking loss has occurred during the tracking of the target object.

[0066] Step S103: If it is determined that a tracking loss occurred during the tracking of the target object, then determine whether the creation of the target object was outside the protected area; where the protected area is a sub-area of ​​the workstation entrance and exit.

[0067] Once a target object is detected in the monitoring screen, if it is not identified within a preset time or a preset number of video frames, it is determined that the target object has been lost. Target object tracking loss corresponds to two scenarios: the target object may have left the workstation; or the target object may still be at the workstation, but due to object obstruction, personnel hiding, or algorithm detection failure, it disappears from the monitoring equipment's view. To differentiate between these two scenarios, this embodiment further determines whether the target object was created outside the protected area, i.e., whether the target object entered normally from the workstation entrance / exit.

[0068] Step S104: If it is determined that the creation of the target object is outside the protected area, the counter value is incremented by 1.

[0069] If the target object was created outside the protected area and subsequent tracking is lost, it indicates that the corresponding personnel entered the protected area normally but abnormally disappeared within it. In this case, the number of people involved needs to be reported, and the count value incremented by 1.

[0070] In step S105, if the counter value is greater than 0, an alarm is triggered and the production line is prevented from being reset.

[0071] Determine if the counter value is greater than 0. If the final counter value is greater than 0, it indicates that personnel are lingering in the protected area. At this time, triggering an alarm can alert the relevant personnel, prohibit the production line from being reset, and prohibit the start of mechanical devices.

[0072] In steps S101 to S106 above, by monitoring and tracking the workstations on the production line, and combining the current status and historical creation status of the target object, it is determined whether the target object belongs to the situation where personnel normally enter the protected area but abnormally disappear within the protected area. If it is determined that the target object belongs to this situation, an alarm is triggered, and the production line is prohibited from being reset. This avoids starting mechanical devices when personnel are left behind but disappear from the monitoring screen, thereby improving the safety of the production line.

[0073] The alarm strategy employed by the relevant technology is to trigger an alarm when a target object is detected in the monitoring screen. However, this method suffers from resource waste and insufficient alarm effectiveness in certain application scenarios. Specifically, in production line safety monitoring systems, a monitoring platform is typically configured to display real-time monitoring footage from each workstation. When workers can clearly observe the presence of personnel through the monitoring screen, they can independently assess the on-site situation and avoid misoperation. If an alarm is triggered due to the detection of personnel, it may lead to alarm signal redundancy, causing unnecessary consumption of system resources.

[0074] Conversely, if a person is actually present at a workstation but is not visible in the monitoring footage due to being in a blind spot or obstructed, staff may mistakenly assume the workstation is unoccupied and reset the production line, creating a safety hazard. To address this scenario, this embodiment identifies the missing state of the target object in the monitoring footage and, based on preset logic, determines whether alarm conditions are met, triggering an alarm when necessary.

[0075] The alarm strategy in this embodiment focuses on the security risks caused by the lack of monitoring visibility. It can provide effective warnings at critical stages and avoid security accidents caused by misoperation. Compared with the related technologies that only use the appearance of the target as the alarm trigger condition, it has higher security and resource utilization.

[0076] Besides the aforementioned issues of object obstruction, personnel hiding, and algorithm detection failure causing personnel to disappear from the monitoring equipment's view, the disappearance of personnel from the monitoring screen can also occur when the protected area borders the edge of the monitoring field of view of image acquisition module 1. This also applies to situations where personnel normally enter the protected area but abnormally disappear within it, and can be resolved using the production line safety monitoring method of this embodiment.

[0077] To make the production line safety monitoring method of this application clearer and easier to understand, in one embodiment, Figure 6 A schematic diagram of personnel activity status in a monitoring screen is provided. The diagram shows various personnel activity statuses and their corresponding counting actions, as follows:

[0078] Personnel Activity Status 1: When personnel enter and exit the protected area normally, the number of people is not reported, and the current count value is maintained. Normal entry and exit of the protected area refers to personnel entering the protected area from the workstation entrance / exit and then leaving the protected area.

[0079] Personnel Activity Status 2: When personnel enter the protected area normally but abnormally disappear within the protected area, a personnel count is reported and incremented by 1. For example, personnel may disappear from the monitoring equipment's view due to object obstruction, personnel hiding, or algorithm detection failure. Abnormally disappearing within the protected area refers to personnel entering the protected area from the workstation entrance / exit but subsequently disappearing from the monitoring screen.

[0080] Personnel Activity Status 3: When personnel enter the protected area normally but abnormally disappear within the protected area, a personnel count is reported and incremented by 1. For example, when the protected area borders the edge of the monitoring field of view of image acquisition module 1, it will also cause personnel to disappear from the monitoring screen.

[0081] Personnel Activity Status 4: When a person abnormally appears within the protected area and then leaves the protected area normally, a personnel count is reported, and the count value is decremented by 1. "Personnel abnormally appearing within the protected area" means that the person's first appearance on the monitoring screen is within the protected area. In some scenarios, the other end of the workstation (the end furthest from the workstation entrance / exit) connects to other workshops, allowing personnel to enter the workstation from other workshops, thus resulting in situations where personnel abnormally appear within the protected area.

[0082] Personnel Activity Status 5: If a person appears abnormally in the protected area and then disappears abnormally from the protected area, the number of people will not be reported, and the current count value will be maintained.

[0083] Personnel Activity Status 6: When personnel are displayed on the monitoring screen and have not left the protected area, the number of people will not be reported and the current count value will be maintained.

[0084] In one embodiment, in step S103 above, if it is determined that a tracking loss has occurred during the tracking of the target object, the method further includes: if it is determined that the creation of the target object is within the protected area, then maintain the current count value.

[0085] refer to Figure 6 If the target object is created within the protected zone and subsequent tracking is lost, it corresponds to personnel activity state 5 above, meaning the person abnormally appears within the protected zone and then abnormally disappears from it. In this case, it's possible the person is moving back and forth along the edge of the protected zone. Reporting the number of people in this situation would cause the system to enter an infinite loop or even crash. Therefore, in this case, the number of people is not reported, and the current count is maintained.

[0086] In one embodiment, after step S102, the method further includes: if it is determined that no tracking loss has occurred during the tracking of the target object, then it is determined whether the target object is currently outside the protection zone; if it is determined that the target object is currently outside the protection zone, then it is determined whether the creation of the target object is outside the protection zone; if it is determined that the creation of the target object is within the protection zone, then the counter value is decremented by 1.

[0087] refer to Figure 6 If the target object is created within the protected area and the target object is not lost (i.e., the target object is displayed in the monitoring screen), but is outside the protected area (i.e., the target object has left the protected area), then the above-mentioned personnel activity status 4 is applied. That is, when a person abnormally appears in the protected area and leaves the protected area normally, the number of people is reported and the count value is reduced by 1.

[0088] Furthermore, after triggering the counter value to decrement by 1, the method also includes: deactivating the alarm and allowing the production line to be reset if the counter value is not greater than 0.

[0089] In this embodiment, it is determined whether the counter value is greater than 0. If the final counter value is equal to 0, it means that no personnel are staying in the protected area, the alarm can be deactivated, and relevant personnel can be notified to allow the production line to be reset and the mechanical device to be started.

[0090] In one embodiment, after step S102 above, if it is determined that no tracking loss occurred during the tracking of the target object and the target object is currently outside the protection zone, the method further includes: determining whether the creation of the target object is outside the protection zone; if it is determined that the creation of the target object is outside the protection zone, then maintaining the current count value.

[0091] refer to Figure 6 If the target object is created outside the protected area, and the current target object has not been lost in tracking, but is outside the protected area, then the above-mentioned personnel activity state 1 is used, that is, when personnel enter and exit the protected area normally, the number of people is not reported, and the current count value is maintained.

[0092] In one embodiment, after step S102 above, if it is determined that no tracking loss has occurred during the tracking of the target object and the target object is currently within the protected area, the method further includes: maintaining the current count value.

[0093] refer to Figure 6 If the target object is not lost to tracking but is outside the protected area, then the corresponding personnel activity status 6 above applies, that is, when the personnel are displayed on the monitoring screen and have not left the protected area, the number of people will not be reported and the current count value will be maintained.

[0094] In some embodiments, step S101 further includes: extracting the position information of the detected target object in the corresponding detection frame; if the detected target object is a new target object, setting a first attribute based on the position information; if the detected target object is not a new target object, setting a second attribute based on the position information. The first attribute characterizes the relative positional relationship between the target object and the protected area at the time of creation, and the second attribute characterizes the relative positional relationship between the target object and the protected area at any time after creation.

[0095] After setting the attributes of the target object, you can extract its first attribute and use it to determine the target object's relative position to the protected zone at the time of its creation, thus determining whether the target object was created outside the protected zone. Alternatively, you can extract the target object's second attribute and use it to determine the target object's relative position to the protected zone at the current time, thus determining whether the target object is currently outside the protected zone.

[0096] In this embodiment, the detector receives the monitoring screen, identifies the screen, and outputs detection frames. Each detection frame represents an instance of a target object (i.e., a person). The detection frame carries the target object's location information, which determines the relative position of the target object to the protected area. The tracker receives the detection frames output by the detector and tracks the target object.

[0097] By using the location information carried by the detection box when the target object first appears, it can be determined whether the target object was created inside or outside the protected area. If the target object is created outside the protected area, it indicates that the person entered the protected area normally; if the target object is created inside the protected area, it indicates that the person appeared abnormally inside the protected area.

[0098] By using the location information carried by the detection boxes when the target object first appears and last appears, it can be determined whether the target object has abnormally disappeared from the protected area. If the target object was created outside the protected area and subsequently lost within the protected area, it is considered a case of abnormal disappearance of a person from the protected area.

[0099] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0100] In one embodiment, Figure 7 A flowchart of another method for production line safety monitoring is provided, which includes the following steps:

[0101] In step S201, the detector receives the monitoring video and performs detection on the monitoring screen. Specifically, after recognizing the monitoring screen, the detector outputs detection frames. Each detection frame represents an instance of a target object (i.e., a person). The detection frame carries the location information of the target object, and based on this location information, the relative positional relationship between the target object and the protected area can be determined.

[0102] In step S202, the tracker receives the detection boxes output by the detector and tracks the target object. Specifically, for the detected current detection box, the tracker determines whether a matching historical detection box exists in the database; where a historical detection box corresponds to a previously detected target object. If a matching historical detection box exists in the database, the target object in the current detection box is determined not to be a new target object, and the current detection box is added to the data queue of historical detection boxes. Conversely, if a matching historical detection box does not exist in the database, the target object in the current detection box is determined to be a new target object, and an identity identifier is created for the new target object. Based on the location information carried by the detection boxes in each queue, a personnel movement trajectory can be generated.

[0103] In step S203, the tracker processes each tracked target object. For each detected target object, its position information is extracted from the corresponding detection box. If the detected target object is a new target object, a first attribute is set based on the position information; if the detected target object is not a new target object, a second attribute is set based on the position information. The first attribute characterizes the relative position of the target object to the protected area at the time of its creation, and the second attribute characterizes the relative position of the target object to the protected area at any time after its creation.

[0104] Step S204: Determine if the target object is lost and about to be destroyed. If yes, proceed to step S205; if no, proceed to step S210.

[0105] Step S205: Determine whether the target object was created outside the protected area. If yes, proceed to step S206; if no, end the current process. Specifically, the first attribute of the target object can be extracted, and its relative position to the protected area at the time of creation can be determined based on this attribute, thus determining whether the target object was created outside the protected area.

[0106] Step S206, report the number of people +1.

[0107] In step S207, the area counter performs counting.

[0108] In step S208, the PLC controls the operation of the robotic arm based on the count value of the area counter. If the count value of the area counter is greater than 0, an alarm is triggered, and resetting the robotic arm is prohibited. If the count value of the area counter is equal to 0, the alarm is deactivated, and resetting the robotic arm is allowed.

[0109] In step S209, the robotic arm starts or stops according to the signal output by the PLC.

[0110] Step S210: Determine whether the target object is currently outside the protected area. If yes, proceed to step S211; if no, end the current process. Specifically, the second attribute of the target object can be extracted, and its relative position to the protected area at the current moment can be determined based on this attribute, thus determining whether the target object is currently outside the protected area.

[0111] Step S211: Determine whether the target object was created outside the protected area. If yes, end the current process; if no, proceed to step S212.

[0112] Step S212: Report the number of people -1, and proceed to step S207.

[0113] Figure 8 This is a schematic diagram of the interface of the second terminal 22 in this embodiment. Figure 1 For personnel activity status 6, when a person is displayed on the monitoring screen and has not left the protected area, no alarm will be triggered. Figure 9 This is a schematic diagram of the interface of the second terminal 22 in this embodiment. Figure 2 This corresponds to personnel activity status 2 or personnel activity status 3. When a person enters the protected area normally but disappears abnormally within the protected area, an alarm is triggered.

[0114] In this embodiment, the system uses detectors and trackers to independently judge and process each person entering the protected area. First, the system determines whether a person has entered a pre-set protected area. Once a person is detected entering the protected area, the system will determine whether their initial location is outside the protected area based on the time point when the target object was created (i.e., the time when the system "first sees" the person). If the target object was created outside the protected area, it means that the person has entered the protected area normally.

[0115] As the tracking process progresses, the system gradually accumulates the individual's historical trajectory information, forming a record of their movement path. Based on this information, the system can determine whether the individual was already outside the protected area when they first became visible. If the determination is "yes," the system assigns the individual the corresponding attribute (i.e., the first attribute, such as "initially outside the protected area") and continues tracking.

[0116] During subsequent tracking, if the person temporarily "disappears" due to obstruction or leaving the field of vision, but is still within the protected area, the system will not immediately alarm or report; only when the system observes the person leaving the protected area while continuously tracking will it trigger the "person leaving" event and reduce the number of people in the area by one.

[0117] Conversely, if someone is outside the protected area when the system first becomes visible, and is subsequently tracked to enter the protected area and disappear within it (e.g., being obscured or stationary), the system will determine that "personnel entered" and abnormally disappeared within the protected area, incrementing the number of people in the area and triggering an alarm.

[0118] Regarding the alarm mechanism, in the normal protected area alarm mode, the alarm is triggered immediately whenever someone enters the protected area. The mode used in this embodiment is triggered when "abnormal personnel disappearance" occurs. For example, if someone enters the protected area and suddenly disappears from sight without being observed leaving, the system will determine this as abnormal and issue an alarm. Once all personnel who entered have been properly tracked and confirmed to have left, and the number of people in the area is reduced to zero, the system will automatically deactivate the alarm.

[0119] Therefore, under the current mode, the system will not trigger alarms for normal personnel activities within the protected area, but will only trigger alarms when there are unexplained disappearance behaviors (i.e., abnormal lingering or disappearance), thereby improving the accuracy and practicality of alarms.

[0120] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a production line safety monitoring method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0121] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0122] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0123] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0124] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0125] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for safety monitoring of a production line, characterized in that, include: Acquire monitoring images of workstations corresponding to the production line, and detect and track personnel appearing in the monitoring images; Determine whether tracking loss has occurred during the tracking of the target object; If it is determined that a tracking loss occurred during the tracking of the target object, then it is determined whether the target object was created outside the protected area; wherein, the protected area is a sub-area of ​​the workstation entrance / exit. If it is determined that the creation of the target object is outside the protected area, the counter value is incremented by 1. If the counter value is greater than 0, an alarm is triggered and resetting the production line is prohibited.

2. The production line safety monitoring method according to claim 1, characterized in that, If it is determined that tracking loss has occurred during the tracking of the target object, the method further includes: If it is determined that the creation of the target object is within the protected zone, the current count value is maintained.

3. The production line safety monitoring method according to claim 1, characterized in that, The method further includes: If it is determined that no tracking loss occurred during the tracking of the target object, then it is determined whether the target object is currently outside the protected area; If it is determined that the target object is currently outside the protected zone, then it is determined whether the creation of the target object occurred outside the protected zone; If it is determined that the creation of the target object is within the protected area, the counter value is decremented by 1.

4. The production line safety monitoring method according to claim 3, characterized in that, After triggering the decrement of the counter value by 1, the method further includes: If the counter value is not greater than 0, the alarm is deactivated and the production line is allowed to be reset.

5. The production line safety monitoring method according to claim 3, characterized in that, If it is determined that no tracking loss occurred during the tracking of the target object, and the target object is currently outside the protected area, the method further includes: Determine whether the creation of the target object occurred outside the protected area; If it is determined that the creation of the target object is outside the protected zone, the current count value is maintained.

6. The production line safety monitoring method according to claim 3, characterized in that, If it is determined that no tracking loss occurred during the tracking of the target object, and the target object is currently within the protected area, the method further includes: Maintain the current count value.

7. The production line safety monitoring method according to any one of claims 1 to 6, characterized in that, Determining whether the creation of the target object occurred outside the protected zone includes: extracting a first attribute of the target object, and determining the relative position of the target object with respect to the protected zone at the time of its creation based on the first attribute; or... Determining whether the target object is currently outside the protected zone includes: extracting a second attribute of the target object and determining the relative position of the target object with respect to the protected zone at the current moment based on the second attribute.

8. The production line safety monitoring method according to claim 7, characterized in that, Detecting and tracking personnel appearing in the surveillance footage includes: For the detected target object, extract the position information of the target object in the corresponding detection box; If the detected target object is a new target object, then the first attribute is set according to the location setting information; If the detected target object is not a new target object, then the second attribute is set according to the location information.

9. The production line safety monitoring method according to claim 8, characterized in that, Before extracting the location information of the detected target object from the corresponding detection box, the method further includes: For the currently detected bounding box, determine whether there is a matching historical bounding box in the database; wherein, the historical bounding box corresponds to a previously detected historical target object; If it is determined that a matching historical detection box exists in the database, then the target object in the current detection box is determined not to be a new target object, and the current detection box is added to the data queue where the historical detection box is located; If it is determined that there is no matching historical detection box in the database, then the target object in the current detection box is determined to be a new target object, and an identity identifier is created for the new target object.

10. A production line safety monitoring system, characterized in that, include: An image acquisition module and a control module are connected; wherein... The image acquisition module is used to acquire monitoring images corresponding to the workstations on the production line and send the monitoring images to the control module; The control module is used to execute the production line safety monitoring method according to any one of claims 1 to 9.

Images