A station personnel identity recognition and alarm system based on multi-modal perception

Abstract

This invention discloses a workstation personnel identification and alarm system based on multimodal perception, belonging to the field of intelligent security and office information technology. It includes: a management terminal and several monitoring terminals. The management terminal is electrically connected to each of the monitoring terminals. The management terminal is used to generate an authorized list based on permission-related data and to dynamically adjust the authorized list of the monitoring terminals in real time based on the target task. The monitoring terminals include: a data acquisition module for determining whether personnel are present at a specific workstation and acquiring image information when personnel are present; a control module for obtaining authorization information of personnel at a specific workstation based on the image data and the acquired authorized list, and generating alarm information based on the authorization information and duration; and an audible and visual alarm module for triggering an alarm based on the alarm information.

Description

Technical Field

[0001] This invention belongs to the field of intelligent security and office information technology, specifically relating to a workstation personnel identification and alarm system based on multimodal perception. Background Technology

[0002] In today's era of accelerated digital transformation, refined management of office environments has become a core requirement for enterprises to improve operational efficiency and ensure information security. As the core unit of office activities, the precise control of personnel presence at workstations is of paramount importance. Whether in confidential industries such as finance and technology, or in the daily management of ordinary enterprises, the legitimate use of workstations is directly related to the protection of trade secrets, data security, and human resource optimization. Unauthorized personnel occupying critical workstations could lead to risks such as the leakage of core data and disruption of business processes. Simultaneously, precise workstation personnel monitoring can provide data support for attendance management and workstation utilization analysis, helping enterprises reduce costs and increase efficiency. Especially in the new model of integrated remote and offline work, the dynamic nature of workstation use has increased, and traditional management methods are no longer sufficient to meet the requirements of real-time and precise management. Developing efficient and reliable workstation personnel identification and monitoring technology has become a key issue that urgently needs to be addressed in the field of office management. Currently, industry solutions for monitoring employee presence at workstations mainly fall into three categories, but all have significant technical shortcomings, failing to achieve end-to-end control of "identity recognition - status verification - risk warning." The first is a pure video surveillance solution. This solution deploys network cameras in the office area to collect images of workstation areas through continuous recording or timed snapshots, relying primarily on visual imaging technology for scene coverage. It only provides image recording functionality, which can be used for post-event behavior tracing, but cannot intelligently determine the identity of personnel in the footage or the legality of workstation usage. A more significant problem is that continuous recording generates massive amounts of redundant data, not only consuming significant storage resources and increasing device power consumption, but also seriously infringing on personal privacy due to indiscriminate collection of employee behavior, leading to employee resistance and legal risks. The second is an access control card swipe solution. This solution, based on radio frequency identification or password verification technology, completes personnel identity verification at the office entrance, allowing only authorized personnel to enter the area. Its control scope is limited to "area access" and cannot be extended to "workstation-level" precise management. When authorized personnel enter the office area, the unauthorized occupation of workstations by unauthorized personnel (such as visitors or other employees) is completely undetectable and cannot be intervened in, resulting in a lack of guarantee for the legality of workstation use. Thirdly, there is the single-sensor solution. This solution deploys pressure sensors or infrared sensors on workstation chairs, desktops, etc., using pressure changes or infrared signals from personnel to determine the presence of people at the workstation. Its core reliance is on the sensing capability of a single physical signal. It can only output a binary "person / no one" status, failing to achieve personnel identification. Furthermore, due to the single sensing dimension, it cannot distinguish between scenarios such as brief passage, temporary leaning, and prolonged sitting, easily generating false alarms and disrupting normal office order. Therefore, there is an urgent need for a multimodal perception-based system for identifying and alarming personnel at workstations to solve the problems existing in current technologies. Summary of the Invention

[0003] In view of this, the present invention provides a workstation personnel identification and alarm system based on multimodal perception, which is used to solve the problem that the existing technology cannot perform real-time and automatic identity verification of personnel sitting at specific workstations, and to provide on-site immediate alarm for unauthorized long-term sitting behavior.

[0004] To achieve the above objectives, the present invention provides a workstation personnel identification and alarm system based on multimodal perception, comprising: The system includes a management terminal and several monitoring terminals. The management terminal is electrically connected to several monitoring terminals. The management terminal is used to generate an authorized list based on permission-related data and to adjust the authorized list of the monitoring terminals in real time based on the dynamic authorized list sequence generated according to the target task. The monitoring terminal includes: The data acquisition module is used to determine whether there are personnel at a specific workstation and to acquire image information when personnel are present. The control module is used to obtain the authorization information of personnel at a specific workstation based on image data and the acquired authorization list, and to generate alarm information based on the authorization information and duration. The audible and visual alarm module is used to trigger an alarm based on alarm information.

[0005] As an embodiment of the present invention, the data acquisition module includes: An infrared sensing unit is used to determine whether there are personnel at the workstation; if not, it will re-determine after a preset time; if there are personnel, it will generate an image acquisition signal. The camera unit is used to receive image acquisition signals and acquire image information of the workstation.

[0006] As an embodiment of the present invention, the control module performs the following operations: The identity information of personnel at a specific workstation is obtained by identifying image information; The authorization information of personnel at a specific workstation is determined based on their identity information and the authorized list; the authorization information includes authorized personnel and unauthorized personnel. If the personnel are unauthorized or the identity information of the personnel at a specific workstation cannot be identified, the specific workstation is determined to be in an unauthorized state, and an alarm message is generated after the duration of the unauthorized state exceeds the preset duration.

[0007] As an embodiment of the present invention, the monitoring terminal further includes: The communication module, electrically connected to the control module, is used to upload alarm information to the management terminal and obtain updated authorization lists.

[0008] As an embodiment of the present invention, the management terminal performs the following operations: Construct a map of specific workstations and determine the workstation number for each specific workstation; Periodically retrieve permission-related data; the permission-related data includes: the latest workstation allocation data and temporary permission data; Randomly select any specific workstation as the target workstation, and determine the four-dimensional mapping relationship of permissions for the target workstation based on the permission association data; wherein, the four dimensions of the four-dimensional mapping relationship of permissions are workstation number, personnel identity information, authorization information, and authorization time; Repeat the above steps until you obtain the four-dimensional mapping relationship of permissions for all specific workstations; The authorized list is obtained based on the four-dimensional mapping relationship of permissions for all specific workstations.

[0009] As an embodiment of the present invention, the management terminal also performs the following operations: Obtain the identity information of all personnel at a specific workstation at the same sampling time to obtain an identity information set; Determine if any two pieces of identity information are identical in the identity information set; if so, generate an alarm message.

[0010] As an embodiment of the present invention, the management terminal also performs the following operations: Obtain target task and corresponding personnel status data, and parse the target task to obtain key entity information and intent information; By using a pre-built task step association graph, a task execution sequence is obtained based on key entity information and intent information; the task step association graph includes step operation nodes and workstation monitoring nodes that are associated in a graph structure. A task execution space is constructed based on personnel status data, and a reinforcement learning agent is constructed based on the task execution space; Based on the task execution sequence, a dynamic authorization list sequence is generated through a reinforcement learning agent; The dynamic authorization list sequence is sent to the monitoring terminal according to the workstation number, so that the monitoring terminal can monitor and alarm the personnel moving to a specific workstation according to the dynamic authorization list sequence.

[0011] As one embodiment of the present invention, personnel status data is acquired, a task execution space is constructed based on the personnel status data, and a reinforcement learning agent is constructed based on the task execution space, including: Acquire personnel status data, determine the task execution operation for each person based on the personnel status data, and add an operation identifier to each task execution operation; the operation identifier includes: estimated execution time, personnel identity information, and personnel information; Construct a task execution space based on the task execution operations of all personnel; A linear reward function is constructed, and a reinforcement learning agent architecture is initialized based on a deep neural network. The task execution space is then input into the reinforcement learning agent architecture to obtain the reinforcement learning agent.

[0012] As an embodiment of the present invention, a dynamic authorization list sequence is generated by a reinforcement learning agent based on the task execution sequence, including: Input the task execution sequence into the trained reinforcement learning agent; By using the policy network in the reinforcement learning agent, the fit score between each element in the task execution sequence and each task execution operation in the task execution space is calculated. The task execution operation with the highest fit score is determined as the candidate task execution operation, and the candidate task execution operation sequence is obtained. The authorized list sequence is obtained based on the candidate task execution operation sequence, task execution sequence, and operation identifier; among them, the dynamic authorized list sequence is a sequence composed of multiple dynamic authorized lists, which include: workstation number, identity information, permissions, and minimum on-duty time.

[0013] The authorized list sequence is obtained based on the candidate task execution operation sequence, the task execution sequence, and the operation identifier; the authorized list sequence is a sequence composed of multiple dynamic lists, including: workstation number, identity information, permissions, and minimum on-duty time.

[0014] The beneficial effects of this invention are as follows: the data acquisition module and the control module respectively monitor the personnel at specific workstations in real time and identify their identity and authorization information. When the authorization of the personnel at the specific workstation is not approved and the person stays for more than the preset duration, an alarm is issued through the sound and light alarm module until the personnel leave (the infrared sensing signal disappears) or the administrator intervenes. This solves the problem that the existing technology cannot perform real-time and automatic identity verification of personnel sitting at specific workstations and can provide on-site immediate alarm for unauthorized long-term sitting behavior.

[0015] Other advantages, objectives, and features of the invention will be set forth in the following description and will be apparent to those skilled in the art in some respects, or may be learned by practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0016] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the following figures are provided for illustration: Figure 1 This is a schematic diagram of the system of the present invention; Figure 2 This is a schematic diagram of the monitoring terminal module of the present invention; Figure 3 This is a flowchart illustrating the control module of the present invention. Detailed Implementation

[0017] like Figures 1-3 As shown, this invention provides a workstation personnel identification and alarm system based on multimodal perception, comprising: The system includes a management terminal and several monitoring terminals. The management terminal is electrically connected to several monitoring terminals. The management terminal is used to generate an authorized list based on permission-related data and to adjust the authorized list of the monitoring terminals in real time based on the dynamic authorized list sequence generated according to the target task. The monitoring terminals include: The data acquisition module is used to determine whether there are personnel at a specific workstation and to acquire image information when personnel are present. The control module is used to obtain the authorization information of personnel at a specific workstation based on image data and the acquired authorization list, and to generate alarm information based on the authorization information and duration. The audible and visual alarm module is used to trigger an alarm based on alarm information.

[0018] The working principle of the above technical solution is as follows: In the actual monitoring process, a single monitoring terminal monitors a specific workstation; multiple monitoring terminals monitor all workstations. During the monitoring process, a management terminal stores and manages the data generated by the monitoring terminals. For example, the management terminal generates an authorized list and sends it to the corresponding monitoring terminals, enabling the monitoring terminals to monitor personnel at specific workstations based on the authorized list. During the monitoring of a specific workstation, the data acquisition module determines in real time whether personnel are present at the specific workstation and acquires image information of the specific workstation when personnel are present. Then, the control module obtains the authorization information of the personnel at the specific workstation based on the image data and the authorized list, and generates alarm information based on the authorization information and duration. Finally, the audible and visual alarm module triggers an alarm based on the alarm information. The audible and visual alarm module includes LED lights and a buzzer, which can achieve immediate deterrence and on-site intervention for violations, transforming passive tracing into proactive protection and improving security response efficiency. The beneficial effects of the above technical solution are as follows: Through the above technical solution, the data acquisition module and the control module respectively perform real-time monitoring and identification of personnel at specific workstations, and when the authorization of personnel at a specific workstation is not approved and the stay exceeds the preset time, the sound and light alarm module will issue an alarm until the personnel leave (the infrared sensing signal disappears) or the administrator intervenes; This solves the problem that the existing technology cannot perform real-time and automatic identity verification of personnel sitting at specific workstations, and cannot provide on-site immediate alarm for unauthorized long-term sitting behavior.

[0019] In one embodiment, the data acquisition module includes: The data acquisition module includes: An infrared sensing unit is used to determine whether there are personnel at the workstation; if not, it will re-determine after a preset time; if there are personnel, it will generate an image acquisition signal. The camera unit is used to receive image acquisition signals and acquire image information of the workstation.

[0020] The working principle and beneficial effects of the above technical solution are as follows: The infrared sensing unit, based on the pyroelectric infrared (PIR) principle and / or infrared ranging principle, can effectively distinguish between people and objects and sense the seating status of people. Its output signal (image acquisition signal) serves as the trigger condition for the camera unit to collect data. The camera unit is in a sleep state when it does not receive an image acquisition signal. Through the collaborative work of the infrared sensing unit and the camera unit, an intelligent triggering mechanism of "shooting only when there is someone present, not when there is no one present" is realized, which can significantly reduce system power consumption and data redundancy, and protect employee privacy. In one embodiment, the control module performs the following operations: The identity information of personnel at a specific workstation is obtained by identifying image information; The authorization information of personnel at a specific workstation is determined based on their identity information and the authorized list; the authorization information includes authorized personnel and unauthorized personnel. If the personnel are unauthorized or the identity information of the personnel at a specific workstation cannot be identified, the specific workstation is determined to be in an unauthorized state, and an alarm message is generated after the duration of the unauthorized state exceeds the preset duration.

[0021] The working principle and beneficial effects of the above technical solution are as follows: After receiving image information, the main control module performs face recognition or employee badge recognition on the captured image information (face recognition and employee badge recognition are existing technologies and will not be elaborated on here) to obtain the personnel's identity information. It then compares the workstation number with the authorized list to obtain the personnel's authorization information. If the personnel are identified as authorized, the system resets and does not perform any operation. If the personnel are unauthorized or their identity information cannot be identified, the specific workstation is determined to be in an unauthorized state, indicating that the personnel at the workstation have violated regulations. The duration of the unauthorized state is recorded, and an alarm message is generated after the duration of the unauthorized state exceeds the preset duration (preferably 30 seconds). The control module adopts the logic of "identity recognition + delayed judgment," which can effectively avoid false alarms caused by personnel passing by briefly or engaging in temporary conversations. An alarm is only triggered when an unauthorized personnel remain seated for more than the preset duration, thus improving the accuracy of the alarm.

[0022] In one embodiment, the monitoring terminal further includes: The communication module, electrically connected to the control module, is used to upload alarm information to the management terminal and obtain updated authorization lists.

[0023] The working principle and beneficial effects of the above technical solution: The communication module is electrically connected to the control module, and it can be any one of the following: Wi-Fi, Ethernet module and IoT card that can be integrated. It is used to upload alarm information to the management terminal or receive the latest authorized list issued by the management terminal.

[0024] In one embodiment, the management terminal performs the following operations: Construct a map of specific workstations and determine the workstation number for each specific workstation; Periodically retrieve permission-related data; the permission-related data includes: the latest workstation allocation data and temporary permission data; Randomly select any specific workstation as the target workstation, and determine the four-dimensional mapping relationship of permissions for the target workstation based on the permission association data; wherein, the four dimensions of the four-dimensional mapping relationship of permissions are workstation number, personnel identity information, authorization information, and authorization time; Repeat the above steps until you obtain the four-dimensional mapping relationship of permissions for all specific workstations; The authorized list is obtained based on the four-dimensional mapping relationship of permissions for all specific workstations.

[0025] The management terminal also performs the following operations: Obtain the identity information of all personnel at a specific workstation at the same sampling time to obtain an identity information set; Determine if any two pieces of identity information are identical in the identity information set; if so, generate an alarm message.

[0026] The working principle and beneficial effects of the above technical solution are as follows: In actual use, a specific workstation map (the monitoring area where monitoring terminals are installed) is constructed to determine the location and number of each specific workstation; then, the permission-related data uploaded by the administrator is periodically retrieved (generally one day per cycle, either manually initiated by the administrator or updated by an "event-triggered synchronization" mechanism, such as when a workstation adjustment occurs, triggering an update of the authorization list); the permission-related data includes: the latest workstation allocation data and temporary permission data. The latest workstation allocation data is the data after internal employees undergo workstation adjustments, employee job transfers, etc. Newly assigned location information, and temporary permission data, are manually entered by the administrator when external personnel enter the internal collaborative work area. This data includes the assigned workstation, permissions, and authorization time. By randomly selecting a specific workstation, a four-dimensional permission mapping relationship for the target workstation is constructed based on permission association data. Then, the four-dimensional permission mapping relationships for all specific workstations are aggregated into a table to obtain the authorized list. Finally, the management terminal distributes the authorized list to each monitoring terminal, overwriting and updating the existing authorized list. This allows the monitoring terminal to determine whether the personnel at a specific workstation are authorized when it detects personnel at that workstation, based on the authorized list. Meanwhile, after the monitoring terminal completes the identification of personnel at a specific workstation, the management terminal will aggregate the identity information of personnel at all characteristic workstations into an identity information set, and determine whether the identity information of any two workstations in the identity information set is the same. If they are the same, the management terminal will generate an alarm message and send it to the corresponding monitoring terminal to trigger an alarm, thus preventing identity information from being stolen.

[0027] In one embodiment, the management terminal also performs the following operations: Obtain target task and corresponding personnel status data, and parse the target task to obtain key entity information and intent information; By using a pre-built task step association graph, a task execution sequence is obtained based on key entity information and intent information; the task step association graph includes step operation nodes and workstation monitoring nodes that are associated in a graph structure. A task execution space is constructed based on personnel status data, and a reinforcement learning agent is constructed based on the task execution space; Based on the task execution sequence, a dynamic authorization list sequence is generated through a reinforcement learning agent; The dynamic authorization list sequence is sent to the monitoring terminal according to the workstation number, so that the monitoring terminal can monitor and alarm the personnel moving to a specific workstation according to the dynamic authorization list sequence.

[0028] This includes acquiring personnel status data, constructing a task execution space based on the personnel status data, and constructing a reinforcement learning agent based on the task execution space, including: Acquire personnel status data, determine the task execution operation for each person based on the personnel status data, and add an operation identifier to each task execution operation; the operation identifier includes: estimated execution time, personnel identity information, and personnel information; Construct a task execution space based on the task execution operations of all personnel; A linear reward function is constructed, and a reinforcement learning agent architecture is initialized based on a deep neural network. The task execution space is then input into the reinforcement learning agent architecture to obtain the reinforcement learning agent.

[0029] Based on the task execution sequence, a dynamic authorization list sequence is generated through a reinforcement learning agent, including: Input the task execution sequence into the trained reinforcement learning agent; By using the policy network in the reinforcement learning agent, the fit score between each element in the task execution sequence and each task execution operation in the task execution space is calculated. The task execution operation with the highest fit score is determined as the candidate task execution operation, and the candidate task execution operation sequence is obtained. The authorized list sequence is obtained based on the candidate task execution operation sequence, the task execution sequence, and the operation identifier; the authorized list sequence is a sequence composed of multiple dynamic lists, including: workstation number, identity information, permissions, and minimum on-duty time.

[0030] The working principle of the above technical solution is as follows: In the production process of large machinery, the machinery is fixed on the installation mechanism. The installation mechanism is driven by the transmission mechanism to pass through the installation personnel on both sides in sequence. After each installation personnel completes an operation step, the transmission mechanism drives the machinery to the next step, resulting in a large flow between the machinery and the personnel. However, if monitoring is carried out directly through the above method using a monitoring terminal, i.e., installing the monitoring terminal at the personnel's work position to form a specific monitoring workstation, it can only identify whether the employee is at the specific workstation, but cannot identify whether the personnel at the workstation have performed production operations on the machinery. At the same time, the monitoring terminal needs to be constantly turned on. Therefore, this technical solution also proposes the following monitoring method: by setting the monitoring terminal on the transmission mechanism... The system allows the monitoring terminal to move relative to personnel, meaning specific workstations move relative to personnel. This ensures that operators performing specific tasks must be present at their designated workstations and remain there for a certain period; otherwise, the monitoring terminal will issue an alarm. When installing the monitoring terminal, its camera area (specific workstation) corresponds to the location where personnel need to operate during equipment assembly. Specifically, after receiving the target task input by the administrator (tasks are typically expressed as natural language text, such as "installing doors on a XXX model car"), the management terminal uses an existing language analysis model to parse the natural language text, extracting key entity information (e.g., XXX model car) and intent information (e.g., "for..."). (Car door installation); then, through a pre-built task step association graph, based on key entity information and intent information, a task execution sequence is obtained (e.g., 1. Connect the door frame to the frame; 2. Install the window transmission mechanism; 3. Install other components; 4. Organize the wiring; 5. Install the sealing strip, etc.). The task execution sequence consists of the steps required to complete the target task, with each step corresponding to an element in the sequence. Each element contains specific operation steps and the shortest time required for the operation. The task step association graph is constructed by identifying and obtaining workstation entities and step entities from standard document data in the device manufacturing process. Specifically, step entities are extracted by scanning process documents during the manufacturing process, such as… The system identifies steps involving personnel operations, such as "Vehicle Model XXX," "Installing Doors," "Installing Transmission Mechanisms," and "Installing Sealing Strips" (along with records of the specific locations of these operations and the minimum time required to complete them according to standard specifications). Monitoring entities are obtained by analyzing the installation locations of monitoring terminals and the corresponding workstation maps. The relationships between these entities are analyzed, including monitoring relationships, sequential relationships, and attribution relationships. Based on the workstation and step entities, they are transformed into nodes in a graph, creating a unique node for each entity. Connecting edges are added between nodes according to the relationship types, thus obtaining the task step relationship graph. Finally, the complete structure of the nodes and edges is stored in a graph database for querying. A task execution space is constructed based on personnel status data, and a reinforcement learning agent is built based on this space. Personnel status data includes: personnel identity information, action content (referring to the specific operations that can be performed to complete the task), and the time required to complete the action content. Specifically, all executable operations for completing the task (the operation process for personnel to complete the task) are read from a pre-set storage database. Then, an operation identifier is added to each operation based on the personnel status data to obtain the task execution operations. These operations are then aggregated to obtain the task execution space. The operation identifier includes: estimated execution time (obtained from the time required for personnel to complete the action content), identity information, and personnel information (long-term personnel). (Regarding staff or temporary personnel, authorization time, etc.); then construct a linear reward function, initialize the reinforcement learning agent architecture based on a deep neural network, input the task execution space into the reinforcement learning agent architecture, and obtain the reinforcement learning agent; specifically: construct a linear reward function, initialize the reinforcement learning agent architecture based on a deep neural network, then add the linear reward function and task execution space to the reinforcement learning agent architecture, and then train to obtain the reinforcement learning agent; wherein, the training process is to construct an experience pool through structured experience data converted from historical production logs and test scenarios generated by the system simulator; advance the training in an iterative manner, that is, in each iteration, the agent is based on the current task step, through - A greedy strategy selects task actions and executes them in the test environment, obtaining the new state and reward value after execution. The current state, action, reward, and new state are stored in the experience pool. The process is iterated continuously. After the number of iterations reaches a preset number, training stops and the parameters of the optimal model are saved, resulting in a reinforcement learning agent. Based on the task execution sequence, a dynamic authorization list sequence is generated through a reinforcement learning agent, specifically as follows: Input the task execution sequence into the trained reinforcement learning agent; The policy network (based on Transformer or CNN) in the reinforcement learning agent calculates the fit score between each element in the task execution sequence and each task execution operation in the task execution space. Specifically, the task steps in the task execution sequence are sequentially input into the reinforcement learning agent for forward computation. The policy network outputs a fit score for each task execution operation in the task execution space, determining the task execution operation with the highest fit score as a candidate task execution operation, thus obtaining a candidate task execution operation sequence. A higher fit score indicates a more suitable task execution operation for the current task step. After determining the candidate task execution operations for all task steps, all candidate task execution operations are sorted according to the order of the task execution sequence, resulting in a candidate task execution sequence. Finally, the candidate task execution sequence, the task execution sequence, and the operation identifiers form an authorization list sequence. The dynamic authorization list sequence is composed of multiple dynamic authorization lists, each corresponding to a task step in the task execution sequence. Each dynamic authorization list includes: workstation number, identity information, permissions, and minimum on-duty time. The dynamic authorization list sequence is distributed to the monitoring terminal according to the workstation number. This allows the monitoring terminal to monitor and issue alarms for personnel moving between specific workstations based on the dynamic authorization list sequence. Specifically: before production, the authorization list is distributed to the corresponding personnel for preparation; during production, the management terminal distributes the dynamic authorization list to the corresponding monitoring terminal according to the workstation number in the order of the dynamic authorization list sequence. The monitoring terminal determines whether the personnel at the specific workstation corresponding to the monitoring terminal are on duty based on the identity information, permissions, and minimum on-duty time in the received dynamic authorization list. If not on duty, an alarm is generated; if on duty, it checks whether the personnel's on-duty time (i.e., the duration of the authorized state) is greater than the minimum on-duty time; if less, an alarm is issued; if greater, the monitoring task for the current dynamic authorization list is completed, and the system enters a dormant state, waiting to receive the next dynamic authorization list for further evaluation. The beneficial effects of the above technical solution are as follows: By generating a dynamic authorized list sequence based on the target task, when executing a task step, the personnel performing the corresponding task step must be present at the corresponding workstation; otherwise, an alarm will be issued. This avoids omissions, errors, or unauthorized personnel substituting for tasks, ensuring that production is strictly executed according to specifications. Furthermore, by querying the number of dynamically authorized list sequences issued, the specific step in production can be determined, facilitating administrator control of production progress. Simultaneously, the monitoring terminal will also determine the dwell time of personnel performing tasks at their workstations, thereby ensuring the integrity of personnel operations.

[0031] In addition, the present invention also provides a workstation personnel identification and alarm device based on multimodal perception, comprising: a device housing and a clamping device, wherein the monitoring terminal of the above-mentioned workstation personnel identification and alarm system based on multimodal perception is installed inside the device housing; The working principle and beneficial effects of the above technical solution are as follows: Through the designed clamping structure, it can be clamped on the top of the workstation computer display screen, so that the camera and infrared sensor can obtain the best monitoring angle. It can also be placed directly on the workstation desktop to realize the monitoring of personnel at specific workstations. At the same time, this invention adopts an integrated embedded box design, which can realize rapid and flexible workstation-level deployment without complex wiring, thus reducing implementation costs.

[0032] Meanwhile, a preferred embodiment of the monitoring terminal that can be put into actual production is also provided, such as: the main control chip adopts the RK3506 processor, the infrared sensor adopts the Anxinke RD-03 sensor, the camera adopts the OmniVision 9732 Camera, and the sound and light alarm module adopts a 5V active buzzer and an RGB LED.

[0033] The work process is as follows: The device is clamped to the top edge of the user's workstation display screen and powered on.

[0034] The sensor monitors a fan-shaped area with a radius of 5 meters and a radius of approximately 120 degrees in front of it. When a person sits down on the workstation chair, the sensor outputs a high-level signal to the GPIO port of the microcontroller.

[0035] After the micro-host detects a high-level signal on the GPIO port, it immediately calls the camera program to capture a picture of the scene.

[0036] The micro-host runs a locally deployed facial recognition program that compares facial features in images with those on a pre-defined authorized list. This pre-defined authorized list can be entered and updated in advance through the backend system.

[0037] Scenario A: Recognition successful, and an authorized person (e.g., Zhang San) matched. System status reset to zero, awaiting the next trigger.

[0038] Scenario B: Identification failed or the user was identified as an unauthorized person (e.g., Li Si). The micro-host starts timing.

[0039] When the preset 30-second threshold was reached, Li Si still hadn't left. The microcontroller sent commands to the GPIO ports controlling the buzzer and LED, causing the buzzer to emit a "beep beep" alarm sound and the LED to flash red.

[0040] The alarm continues until personnel are detected leaving (output low level), at which point the system automatically stops the alarm and resets.

[0041] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made to it in form and detail without departing from the scope defined by the claims of the present invention.

Claims

1. A multi-modal perception based station personnel identification and alarm system, characterized in that, include: The system includes a management terminal and several monitoring terminals. The management terminal is electrically connected to several monitoring terminals. The management terminal is used to generate an authorized list based on permission-related data and to adjust the authorized list of the monitoring terminals in real time based on the dynamic authorized list sequence generated according to the target task. The monitoring terminals include: The data acquisition module is used to determine whether there are personnel at a specific workstation and to acquire image information when personnel are present. The control module is used to obtain the authorization information of personnel at a specific workstation based on image data and the acquired authorization list, and to generate alarm information based on the authorization information and duration. The audible and visual alarm module is used to trigger an alarm based on alarm information.

2. The workstation personnel identification and alarm system based on multimodal perception according to claim 1, characterized in that, The data acquisition module includes: An infrared sensing unit is used to determine whether there are personnel at the workstation; if not, it will re-determine after a preset time; if there are personnel, it will generate an image acquisition signal. The camera unit is used to receive image acquisition signals and acquire image information of the workstation.

3. The workstation personnel identification and alarm system based on multimodal perception according to claim 1, characterized in that, The control module performs the following operations: The identity information of personnel at a specific workstation is obtained by identifying image information; The authorization information of personnel at a specific workstation is determined based on their identity information and the authorized list; the authorization information includes authorized personnel and unauthorized personnel. If the personnel are unauthorized or the identity information of the personnel at a specific workstation cannot be identified, the specific workstation is determined to be in an unauthorized state, and an alarm message is generated after the duration of the unauthorized state exceeds the preset duration.

4. The workstation personnel identification and alarm system based on multimodal perception according to claim 1, characterized in that, The monitoring terminal also includes: The communication module, electrically connected to the control module, is used to upload alarm information to the management terminal and obtain updated authorization lists.

5. The workstation personnel identification and alarm system based on multimodal perception according to claim 1, characterized in that, The management terminal performs the following operations: Construct a map of specific workstations and determine the workstation number for each specific workstation; Periodically retrieve permission-related data; the permission-related data includes: the latest workstation allocation data and temporary permission data; Randomly select any specific workstation as the target workstation, and determine the four-dimensional mapping relationship of permissions for the target workstation based on the permission association data; wherein, the four dimensions of the four-dimensional mapping relationship of permissions are workstation number, personnel identity information, authorization information, and authorization time; Repeat the above steps until you obtain the four-dimensional mapping relationship of permissions for all specific workstations; The authorized list is obtained based on the four-dimensional mapping relationship of permissions for all specific workstations.

6. The workstation personnel identification and alarm system based on multimodal perception according to claim 1, characterized in that, The management terminal also performs the following operations: Obtain the identity information of all personnel at a specific workstation at the same sampling time to obtain an identity information set; Determine if any two pieces of identity information are identical in the identity information set; if so, generate an alarm message.

7. The workstation personnel identification and alarm system based on multimodal perception according to claim 1, characterized in that, The management terminal also performs the following operations: Obtain target task and corresponding personnel status data, and parse the target task to obtain key entity information and intent information; By using a pre-built task step association graph, a task execution sequence is obtained based on key entity information and intent information; the task step association graph includes step operation nodes and workstation monitoring nodes that are associated in a graph structure. A task execution space is constructed based on personnel status data, and a reinforcement learning agent is constructed based on the task execution space; Based on the task execution sequence, a dynamic authorization list sequence is generated through a reinforcement learning agent; The dynamic authorization list sequence is sent to the monitoring terminal according to the workstation number, so that the monitoring terminal can monitor and alarm the personnel moving to a specific workstation according to the dynamic authorization list sequence.

8. A workstation personnel identification and alarm system based on multimodal perception according to claim 7, characterized in that, A task execution space is constructed based on personnel status data, and a reinforcement learning agent is built based on the task execution space, including: Acquire personnel status data, determine the task execution operation for each person based on the personnel status data, and add an operation identifier to each task execution operation; the operation identifier includes: estimated execution time, personnel identity information, and personnel information; Construct a task execution space based on the task execution operations of all personnel; A linear reward function is constructed, and a reinforcement learning agent architecture is initialized based on a deep neural network. The task execution space is then input into the reinforcement learning agent architecture to obtain the reinforcement learning agent.

9. A workstation personnel identification and alarm system based on multimodal perception according to claim 7, characterized in that, Based on the task execution sequence, a dynamic authorization list sequence is generated through a reinforcement learning agent, including: Input the task execution sequence into the trained reinforcement learning agent; By using the policy network in the reinforcement learning agent, the fit score between each element in the task execution sequence and each task execution operation in the task execution space is calculated. The task execution operation with the highest fit score is determined as the candidate task execution operation, and the candidate task execution operation sequence is obtained. The authorized list sequence is obtained based on the candidate task execution operation sequence, task execution sequence, and operation identifier; among them, the dynamic authorized list sequence is a sequence composed of multiple dynamic authorized lists, which include: workstation number, identity information, permissions, and minimum on-duty time.

Images