Medicine production safety robot inspection method and intelligent system

Abstract

The invention discloses a drug production safety robot inspection method, which comprises the following steps: acquiring multi-source heterogeneous data of a drug production safety state in real time, including environment sensor data, intelligent video data and equipment operation parameters; based on the multi-source heterogeneous data, constructing and dynamically updating a production safety risk assessment model; according to the production safety risk assessment model, a self-adaptive inspection task is generated and adjusted, and the task defines an inspection target, a path, a period and an execution maneuvering terminal; scheduling the maneuvering terminal to execute the inspection task, driving the multi-mode AI potential safety hazard identification engine by the returned safety state data and outputting an identification result; and an identification result is fed back to the production safety risk assessment model to form closed-loop control. The invention also discloses an intelligent inspection system for pharmaceutical production safety. According to the invention, prospective risk early warning, self-adaptive task scheduling and closed-loop self-optimization can be realized, so that higher requirements of the modern pharmaceutical industry on personnel safety, public health and compliance production can be met.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent management and control of drug production safety, and specifically provides a drug production safety robot inspection method and intelligent system. Background Art

[0002] The pharmaceutical manufacturing industry has extremely stringent requirements for safety, standardization, and cleanliness, requiring strict adherence to a range of regulations, including Good Manufacturing Practice (GMP). Any minor oversight during the production process, such as equipment malfunctions, environmental deviations, or improper human operation, can lead to drug contamination, batch rejection, or even serious safety accidents. Therefore, efficient and accurate safety inspections are crucial for ensuring drug quality and production safety.

[0003] Traditional pharmaceutical production safety inspections rely primarily on manual, scheduled, and fixed-point inspections. While this approach offers a degree of flexibility, its drawbacks are also significant. First, inspection quality is highly dependent on the inspector's experience and commitment, leading to inconsistent standards, subjectivity, and the risk of missed inspections. Second, manually recorded data is often in paper form or scattered spreadsheets, making systematic data analysis and trend forecasting difficult. Furthermore, frequent personnel access to high-risk areas or those requiring extremely high cleanliness levels presents a potential source of contamination and safety risks.

[0004] To overcome the shortcomings of manual inspections, some attempts at automated monitoring have emerged in existing technologies. For example, fixed cameras, temperature and humidity sensors, and differential pressure sensors are installed within production workshops to provide 24-hour, uninterrupted monitoring of key locations. Furthermore, automated inspection devices or drones have been introduced, performing mobile inspections along pre-set routes and at fixed intervals. These drones transmit video feeds via onboard cameras, replacing some manual labor and improving the objectivity of data collection. However, even with the existing technology employing inspection robots, the following technical issues and inherent shortcomings remain, requiring urgent resolution.

[0005] (1) Rigid inspection strategies and lack of adaptability: Existing robot inspections generally adopt a rigid strategy of "fixed route, fixed cycle". Regardless of how production tasks change or which area has a higher risk level, the inspection frequency and path remain unchanged. This "one-size-fits-all" model leads to irrational allocation of inspection resources. Repeated inspections of idle low-risk areas result in a waste of resources, while there may be insufficient inspection efforts in high-load, high-risk key areas.

[0006] (2) Weak data fusion capabilities and serious information silos: Existing systems typically treat video data collected by robots, environmental sensor data, and the equipment's own operating data (such as SCADA system data) as independent and fragmented information. The system lacks an effective multi-source data fusion model and is unable to correlate and analyze subtle changes in the environment, abnormal vibrations of the equipment, and abnormal phenomena in the video, thus missing out on a large number of opportunities to discover early, hidden fault signs through data correlation.

[0007] (3) Passive risk identification and lack of forward-looking warning: Most systems remain at the level of "post-event discovery," such as identifying leaks that have already occurred or instrument readings that have clearly exceeded standards. They are unable to establish a forward-looking risk assessment model based on the dynamic evolution trends of historical and real-time data to predict "which areas may have increased risk levels within the next hour," thereby achieving an upgrade from "passive response" to "active prevention."

[0008] (4) The system workflow is open-loop and lacks closed-loop feedback optimization: When a hidden danger is discovered, the system workflow is interrupted and manual processing is required. Valuable information such as the discovery, type, and cause of the hidden danger is not automatically learned by the system and used to optimize future inspection strategies. The system lacks a closed-loop feedback mechanism and cannot dynamically and intelligently adjust its risk model and inspection plan based on inspection results, resulting in the system not being able to "get smarter with use."

[0009] Therefore, how to overcome the above-mentioned defects of the existing technology and provide a drug production safety inspection solution that can intelligently perceive risks, dynamically adjust strategies, and achieve closed-loop control is a major challenge currently facing this field. Summary of the Invention

[0010] In view of this, the purpose of the present invention is to provide a pharmaceutical production safety robot inspection method and intelligent system, which can realize forward-looking risk warning, adaptive task scheduling and closed-loop self-optimization to meet the higher requirements of modern pharmaceutical production for extreme safety and efficient compliance.

[0011] In order to achieve the above object, the present invention provides the following technical solutions: The present invention first proposes a pharmaceutical production safety robot inspection method, comprising the following steps: Step 1: Acquire multi-source heterogeneous data on drug production safety status in real time. The multi-source heterogeneous data includes environmental sensor data, equipment operation data, visual / audio data, personnel / material location data, and manually entered data; and pre-process the multi-source heterogeneous data, including data cleaning, data format conversion, timestamp alignment, and preliminary feature extraction. Step 2: Based on the multi-source heterogeneous data, a dynamically updated production safety risk assessment model is constructed. The production safety risk assessment model calculates the position by the following formula: At the moment Comprehensive risk index: in: is the static risk value; is the dynamic risk value; is the event risk value, 、 and is the adjustable weight coefficient; is the risk mapping function; Step 3: Generate adaptive inspection tasks based on the comprehensive risk index. Exceeding the preset attention threshold When a patrol task is triggered, the patrol task includes the target location, risk type and detection items; Step 4: Dispatching the mobile inspection terminal to perform the inspection task, wherein the dispatching is optimized based on the location of the mobile inspection terminal, the matching degree of sensor capabilities and the current load; Step 5: Identify safety hazards in the data returned by the mobile inspection terminal using a multimodal fusion AI safety hazard identification engine, which integrates the output results of the visual analysis model and the time series analysis model. Step 6: Feedback the safety hazard identification results to the production safety risk assessment model to update the static risk value And optimize the multimodal fusion AI safety hazard identification engine to form a closed-loop control.

[0012] Furthermore, in step 1: Environmental sensor data includes: Air pollution detection data is used to collect air pollution factors including physical factors, chemical factors, biological factors and radioactive factors; Water pollution detection data, used to collect water pollution indicators including microbiological indicators, toxicological indicators, sensory properties and general chemical indicators, as well as radioactive indicators; Dust monitoring data, used to detect total dust or respirable dust concentrations in environments where flammable, explosive, or combustible gas mixtures are present; Combustible gas concentration detection data, used to detect the concentration of combustible gas in the air; Thermal infrared imaging sensor data is used to detect the temperature of equipment sealing ports, equipment electrical cabinets, and other locations; Radiation detection data, used to detect residual radioactive doses produced by radiation sterilization; Equipment operation data is read from the equipment control system DCS, including the parameter status of key hazardous sources; Visual / audio data is collected by fixed cameras and mobile inspection terminals; Personnel / material positioning data is obtained through access control, RFID tags, or Beidou / UWB indoor positioning systems to obtain the real-time location and movement trajectory of authorized personnel and key materials; Manually entered data includes safety hazard information reported by inspection personnel that is difficult to detect automatically.

[0013] Furthermore, in step 2: The static risk value Initialize settings based on equipment importance, regional functions and historical accident data; Dynamic Value at Risk It is calculated by the weighted sum of multiple normalized data deviations: in: For and location The number of associated sensors or data sources; For the The weight of the data source, which is the current production process stage The function reflects the changes in focus at different stages; For the Data sources at time The normalized deviation of Event Risk Value Exponential decay when a high-risk event is triggered: in: The moment when the event occurred; is the peak event risk; is the decay constant.

[0014] Further, in step three: When the comprehensive risk index Exceeding the emergency threshold Mark as the highest priority task; The detection items are dynamically specified according to the risk type, including infrared temperature measurement, high-definition photography and gas sampling.

[0015] Furthermore, in step 4, a greedy algorithm is used to calculate the task allocation cost: in: Mobile inspection terminal Move to a task from your current location Estimated time to target location; Mobile inspection terminal Ability matching; Mobile inspection terminal The current load; 、 and is the cost weight.

[0016] Further, in step five: The visual analysis model uses the YOLOv7 object detection network trained on an industrial defect dataset; The time series analysis model uses LSTM network; Multimodal fusion uses the XGBoost classifier as the fusion decision model, and the input features include visual confidence, equipment operating status, temperature difference anomaly, gas concentration change rate and pressure fluctuation variance. Furthermore, step six includes: After confirming the safety hazards, increase the static risk value of the corresponding location ; Manually confirmed safety hazard samples are added to the training library, and the YOLOv7 target detection network and fusion decision model are regularly fine-tuned.

[0017] The present invention also proposes an intelligent inspection system for pharmaceutical production safety, which is used to implement the above-mentioned pharmaceutical production safety robot inspection method, including an application layer, a cognitive layer, a data transmission layer, and a perception layer; The perception layer includes: an environmental sensor array for collecting environmental sensor data; Equipment status perception unit, used to collect equipment operation data; Visual / audio acquisition module, used to collect visual / audio data; Mobile inspection terminals are used to perform inspection tasks and collect multi-source heterogeneous data through the equipment they carry; The cognitive layer includes: A data preprocessing module, used to preprocess the multi-source heterogeneous data collected by the perception layer; Dynamic risk assessment module, used to carry production safety risk assessment model to calculate location At the moment Comprehensive risk index; Adaptive task planning module, used to trigger the generation of inspection tasks based on the comprehensive risk index; The scheduling module is used to schedule the mobile inspection terminal to perform inspection tasks; The AI ​​safety hazard identification module is used to carry a multimodal fusion AI safety hazard identification engine and integrate the output results of the visual analysis model and the time series analysis model; The application layer includes a central control screen, a mobile app, and an alarm terminal. The alarm terminal issues an alarm based on the output of the AI ​​safety hazard identification module. The perception layer is connected to the cognitive layer through the data transmission layer, and the output of the AI ​​safety hazard identification engine is fed back to the dynamic risk assessment module.

[0018] Furthermore, the mobile inspection terminal includes an inspection automation device and / or an inspection drone.

[0019] The beneficial effects of the present invention are: The pharmaceutical production safety robot inspection method of the present invention has the following technical effects: 1. The inspection is predictable and proactive, significantly improving safety assurance capabilities: the comprehensive risk index is calculated in real time through the production safety risk assessment model, and based on the attention threshold Intelligently generate inspection tasks, and the production safety risk assessment model converts static risk values , Dynamic Value at Risk , event risk value It is integrated into a comprehensive risk index to replace manual experience-based decision-making, so that resource scheduling and risk levels are matched in real time, which realizes the predictability and initiative of inspections and significantly improves safety assurance capabilities. The present invention no longer waits for accidents to occur before responding, but through quantitative calculation and prediction of risks, it realizes "forward-looking management" and "active prevention" of safety hazards, nipping safety issues in the bud and fundamentally improving the safety of drug production.

[0020] 2. Dynamic optimization of inspection resources has been achieved, which has greatly improved inspection efficiency: Adaptive task planning based on risk drive has completely replaced the rigid fixed-cycle inspection mode. Valuable resources such as mobile inspection terminals are accurately deployed to the places where they are most needed, avoiding ineffective inspections in low-risk areas, maximizing the input-output ratio of inspection work, and improving the overall inspection efficiency several times. Including (1) Precise resource deployment: Mobile inspection terminals are only deployed in high-risk areas ( ) to trigger the task, avoiding ineffective inspections in low-risk areas; (2) Response speed increased: to respond to sudden high-risk events (such as (3) Coverage efficiency is doubled: the average daily coverage area of ​​a single mobile inspection terminal is increased, and labor costs are reduced.

[0021] 3. Break down data silos, overcome the bottleneck of identifying complex safety hazards, and greatly enhance the accuracy and breadth of investigations: Build a multimodal fusion AI safety hazard identification engine, and simultaneously process visual data, time series data, and environmental parameters to achieve in-depth identification of hidden and complex safety hazards, greatly enhancing the accuracy and breadth of investigations: Through multimodal data fusion analysis technology, different types of data can verify and supplement each other, and can discover complex and early faults hidden behind data correlations that cannot be revealed by a single data source, greatly expanding the depth and breadth of inspections, improving the accuracy of safety hazard investigations, and reducing the missed detection rate.

[0022] 4. Build a closed-loop self-evolution system to achieve continuous performance improvement: Feedback the manually confirmed safety hazard identification results into the production safety risk assessment model to update the static risk value We have also optimized the multimodal fusion AI safety hazard identification engine, enabling closed-loop self-optimization and enabling the system to continuously evolve. Through a closed-loop feedback mechanism, every inspection and every safety hazard identification becomes an opportunity for the system to learn and grow. The production safety risk assessment model and multimodal fusion AI safety hazard identification engine are capable of continuous self-iteration and optimization, making the system increasingly accurate and intelligent with use, ensuring its long-term advancement and effectiveness.

[0023] 5. Strengthen compliance management and reduce audit risks: By fully recording the digital trajectory of the entire process from data collection, task generation to safety hazard disposal, the compliance and traceability of safety management are improved: the entire inspection, discovery, confirmation, and disposal process are automatically, objectively, and completely recorded by the system, forming an unalterable digital chain of evidence, which greatly facilitates GMP and other compliance audits and improves the company's safety management level and transparency. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to make the purpose, technical solutions and beneficial effects of the present invention more clear, the present invention provides the following drawings for illustration: Figure 1 This is a flow chart of an embodiment of the pharmaceutical production safety robot inspection method of the present invention; Figure 2 This is a framework diagram of an embodiment of the intelligent inspection system for pharmaceutical production safety of the present invention. DETAILED DESCRIPTION

[0025] The present invention will be further described below with reference to the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention.

[0026] like Figure 1As shown, the pharmaceutical production safety robot inspection method of this embodiment includes the following steps.

[0027] Step 1: Acquire multi-source heterogeneous data on the safety status of drug production in real time. Multi-source heterogeneous data includes environmental sensor data, equipment operation data, visual / audio data, personnel / material location data, and manually entered data; perform preprocessing on the multi-source heterogeneous data, including data cleaning, data format conversion, timestamp alignment, and preliminary feature extraction.

[0028] 1.1 Multi-source heterogeneous data (1) Environmental sensor data: collected by IoT sensor arrays deployed in key areas (such as clean areas, hazardous materials warehouses, and high-voltage equipment areas), including but not limited to temperature, humidity, air cleanliness (particle count), pressure difference, concentration of toxic and hazardous gases (such as O3, VHP), infrared thermal imaging data, etc.

[0029] Specifically, environmental sensor data includes: Air pollution detection data is used to collect air pollution factors including physical factors, chemical factors, biological factors and radioactive factors. Specifically, physical factors include: temperature, humidity, air flow, radiation, lighting, illumination, noise, vibration, etc.; chemical factors include: particulate matter (dust, smoke, fog), ozone, nitrogen dioxide, sulfur dioxide, carbon dioxide, carbon monoxide, ammonia, formaldehyde, benzene, toluene, xylene, total volatile organic compounds, trichloroethylene, tetrachloroethylene, benzopyrene, odor, synthetic detergents, disinfectants, etc.; biological factors include pathogenic bacteria, viruses, fungi, vectors, etc.; (4) radioactive factors include radon and its progeny, etc. Specifically, the relevant index requirements of air pollution factors can refer to the standard document "Indoor Air Quality Standard" (GB / T 18883-2022).

[0030] Water pollution testing data is used to collect water pollution indicators, including microbiological indicators, toxicological indicators, sensory properties, general chemical indicators, and radioactive indicators. Specifically, microbiological indicators include concentrations of total coliforms, Escherichia coli, and total bacterial counts; toxicological indicators include concentrations of arsenic, pyrene, chromium, lead, mercury, cyanide, fluoride, nitrate, potassium chloride, monobromodibromomethane, dichlorobromomethane, tribromoform, trihalomethanes, dichloroacetic acid, trichloroacetic acid, bromate, chlorite, and chlorate; sensory properties and general chemical indicators include color, turbidity, odor and taste, visible matter, pH, aluminum, iron, manganese, copper, zinc, chloride, sulfate, total dissolved solids, total hardness, permanganate index, and ammonia; radioactive indicators include total alpha radioactivity and total beta radioactivity. For specific water pollution indicators, please refer to the standard document "Standard for Drinking Water Quality GB5749-2022."

[0031] Dust monitoring data, used to detect total dust or respirable dust concentrations in environments where flammable, explosive, or combustible gas mixtures are present; Combustible gas concentration detection data, used to detect the concentration of combustible gas in the air; Thermal infrared imaging sensor data is used to detect the temperature of equipment sealing ports, equipment electrical cabinets, and other locations; Radiation detection data, used to detect residual radioactive doses produced by radiation sterilization; (2) Equipment operation data: Real-time operation parameters are read directly from the control system DCS of production equipment (such as reactors, freeze dryers, and filling machines) through industrial Ethernet, OPC UA protocol or dedicated interface, including the parameter status of key hazardous sources, i.e. pressure and temperature of pressure vessels.

[0032] (3) Visual / audio data: collected by fixed cameras and mobile inspection terminals. Specifically, real-time video and audio streams are collected by deployed fixed smart cameras and high-definition cameras, infrared thermal imagers, and microphones installed on mobile inspection terminals.

[0033] (4) Personnel / material positioning data: Through facial recognition access control, RFID tags or Beidou / UWB indoor positioning systems deployed in key channels, the real-time location and movement trajectory of authorized personnel and key material vehicles within the factory are obtained.

[0034] (5) Manually entered data: This includes safety hazard information that is difficult to detect automatically and reported by inspection personnel. Specifically, this includes safety hazard information that is difficult for machines to detect automatically and reported by inspection personnel through handheld mobile terminals (APP), such as subjective descriptions of odors, abnormal noises, and minor corrosion on the surface of equipment.

[0035] (6) Data Transmission: Data from all sensors and devices is aggregated through the network and transport layers. Specifically, environmental sensors publish data via the MQTT protocol to the EMQ X IoT messaging middleware deployed on the server cluster. Data from device control systems is collected via the OPC UA protocol. Video streams are transmitted via the RTSP protocol. All of these data streams are ultimately fed into an Apache Kafka distributed message queue, serving as the system's data bus for consumption by various modules in the cognitive layer.

[0036] 1.2 Data Preprocessing The collected raw data comes in various formats and of varying quality, requiring standardized preprocessing. This includes data cleaning (noise removal and missing value filling), data format conversion (converting to a standard format such as JSON), timestamp alignment (solving time asynchrony issues between different systems), and preliminary feature extraction (for example, extracting frequency domain features from continuous vibration signals).

[0037] (1) Data format conversion The data preprocessing module consumes raw data from Apache Kafka and converts it into a structured JSON format. For example, a piece of data from a reactor pressure sensor has the following format after conversion: { "event_id": "uuid-v4-generated-string", "timestamp_utc": "2025-06-16T02:34:21.123Z", "source_type": "sensor", "source_id": "Reactor_3_Pressure_Sensor_01", "location_id": "Area_C_Reactor_3", "data_schema_version": "1.0", "payload": { "value": 1.52, "unit": "MPa", "status": "normal"}} This structuring ensures that all data has a unified timestamp, source identification, and location information, laying the foundation for subsequent fusion analysis.

[0038] (2) Data cleaning For time series data (such as pressure, temperature), 3-Sigma based on moving window is used ( ) rule to remove outliers. For data points within a time window , calculate its mean and standard deviation .like , then the point is marked as an outlier and replaced with the median of the data in the window.

[0039] (3) Missing value filling For short-term missing single data points, linear interpolation is used to fill them in. The formula is: in: is the missing data value; and are the values ​​at the time point after and before the missing data, respectively; is the missing time; and They are the time point after and before the missing time, respectively.

[0040] For long periods of data loss, the system will generate a "low data quality" event and increase the dynamic risk of the area accordingly.

[0041] Step 2: Build a dynamically updated production safety risk assessment model The production safety risk assessment model is the core of decision-making. Its design concept is that the safety risk of a region is not static, but a dynamic quantity determined by its inherent attributes, real-time status and emergencies. To this end, this embodiment designs a multi-dimensional dynamic weighted production safety risk assessment model. The core concept of the model is that any location At the moment Comprehensive risk index It is defined as a function that integrates the three dimensions of static risk, dynamic risk and event risk. This embodiment builds a dynamically updated production safety risk assessment model based on multi-source heterogeneous data. The production safety risk assessment model calculates the position through the following formula At the moment Comprehensive risk index: in: is the comprehensive risk index, a dimensionless floating point number with a range of ; It is a static risk value, reflecting the inherent risk of the unit; It is a dynamic risk value, reflecting the degree of deviation of the real-time operating status of the unit; is the event risk value, which reflects the instantaneous risk triggered by discrete, high-risk events; is the risk mapping function, and this embodiment adopts a weighted sum function; 、 and is an adjustable weight coefficient, which is set by the security management personnel according to the management policy. In this embodiment, it is set to , , , indicating greater focus on dynamic and event risks.

[0042] 2.1 Static Value at Risk Static risk ( ) refers to the inherent risk attributes of the target object that do not change easily over time. During the system initialization phase, safety experts will quantitatively assign static risk levels to each monitoring area and key equipment based on GMP requirements, equipment importance, historical accident data, and area functions (such as solvent storage and office areas). In this embodiment, the static risk value Initialize settings based on equipment importance, area function, and historical accident data. For example, a warehouse storing flammable organic solvents may have The value is much higher than that in the finished product packaging area. This value is not permanent and will be adjusted upward by the closed-loop feedback mechanism after a major safety incident, forming a "risk memory."

[0043] Experts assign scores when the system is initialized and store them in a static risk database (a PostgreSQL database table). For example: Solvent storage room : (out of 100); Reactor area : ; Ordinary corridor : .

[0044] 2.2 Dynamic Value at Risk Dynamic Risk This value reflects the degree to which the target object's current operating status deviates from the normal baseline. The system continuously monitors various real-time data streams acquired in step 1. For each data type (such as temperature, pressure differential, and vibration amplitude), the system maintains a "safe baseline model" of normal operation (which can be a simple threshold range or a complex model derived through machine learning). The dynamic risk value is a weighted sum of the deviations from these multiple data points.

[0045] Dynamic Value at Risk It is calculated by the weighted sum of multiple normalized data deviations: in: For and location The number of associated sensors or data sources; For the The weight of the data source, which is the current production process stage The function reflects the changes in focus at different stages; for example: in the "heating reaction" stage , the weight of the temperature sensor ; In the "constant temperature maintenance" stage , its weight may be reduced to , and the weight of the pressure sensor increases. These weight configurations are stored in a configuration file that is linked to the production batch management system (MES). For the Data sources at time The normalized deviation of is calculated as: in: For the sensor at the moment The real-time value of It is the safety baseline value, which is also the production process stage function; for example, the normal pressure of the reactor is different at different stages. As a normalization factor, the width of the normal operating range of the parameter (i.e., the difference between the upper and lower allowed limits) is usually taken to ensure that all deviations are scaled to a comparable range.

[0046] 2.3 Event Risk Value Event Risk The instantaneous risk increase is triggered by a specific, high-risk discrete event. Trigger conditions include but are not limited to: a high-level alarm directly issued by the equipment control system, a manual press of the "Emergency Help" button through the APP, the system intelligently identifying a clear violation (such as running in a clean area), sensor data instantly breaking the "hard red line" threshold, etc. Once an event is triggered, the event risk It will be assigned an extremely high value in a short period of time (for example, 5 minutes) and included in the comprehensive risk index by multiplication or high-weight addition. , ensuring that this area receives the highest priority attention.

[0047] Event risk is handled by the event risk trigger module. When a specific high-risk event is received (such as manual pressing of the emergency button, or the key equipment itself issuing a level 3 alarm), the module will trigger a Assign an instantaneous high event risk value and make it decay exponentially over time, that is, the event risk value Exponential decay when a high-risk event is triggered: in: The moment when the event occurred; The event risk peak is a preset higher value, such as 100.0; is the decay constant, which determines the speed at which the risk dissipates; in this embodiment, , indicating that the risk value will decay to about 60% of the peak value after about 5 minutes.

[0048] Step 3: Generate adaptive inspection tasks based on comprehensive risk index. Exceeding the preset attention threshold The inspection task is triggered when the target location, risk type and detection items are included.

[0049] The "real-time risk map" generated in step 2 is converted into executable inspection instructions. This completely abandons the "fixed cycle" model and adopts an "on-demand service" approach. The system's mobile inspection terminals (automated devices, drones) are always dispatched to the areas with the highest current risk and the greatest need for attention.

[0050] 3.1 Task Generation The system continuously scans the global comprehensive risk index When any position Value at Risk Exceeding the preset attention threshold In the preferred implementation of this embodiment, when the comprehensive risk index Exceeding the emergency threshold It is marked as the highest priority task.

[0051] Specifically, the input of the task generation module is the risk_map object returned by the update_global_risk_map() function. This embodiment presets two risk thresholds: the attention threshold ; Emergency threshold .

[0052] When a task is generated, risk_map is first traversed: if a position Value at Risk , the system creates an inspection task for this location in the "Task Pool". Task priority Directly set it as risk value, that is .like , the task will be additionally marked as "Urgent".

[0053] 3.2 Task Content The generated task is a structured data object containing the task ID, target location / device, current risk score, risk type (temperature anomaly or equipment vibration anomaly), recommended inspection items (such as "infrared temperature measurement," "HD photography," or "gas sampling"), and the task deadline. In this embodiment, inspection items are dynamically assigned based on the risk type, including infrared temperature measurement, HD photography, and gas sampling.

[0054] Step 4: Dispatch the mobile inspection terminal to perform the inspection task. The dispatch is optimized based on the location of the mobile inspection terminal, the matching degree of sensor capabilities and the current load.

[0055] 4.1 Scheduling Objectives The system maintains a real-time status database of all mobile inspection terminals (automated devices, drones), including their location, power level, current tasks, and the sensor modules they are equipped with. When a new task is generated, the scheduling algorithm solves a multi-objective optimization problem, including: Objective 1 (Capability Matching): Give priority to terminals that carry sensors capable of completing the "Recommended Inspection Items".

[0056] Goal 2 (Timeliness): Under the premise of meeting capability matching, select the terminal that is closest to the target location and can reach it the fastest.

[0057] Goal 3 (Load Balancing): Avoid overloading a single terminal, taking into account its remaining battery life and maintenance schedule.

[0058] The optimal "task-terminal" pairing is calculated through a scheduling algorithm (such as the "Hungarian algorithm" or the heuristic "genetic algorithm") and instructions are automatically issued.

[0059] 4.2 Dynamic Path Planning Plan the optimal path from the current location to the target location for the selected terminal, which will avoid other high-risk areas or temporary restricted areas on the map in real time.

[0060] 4.3 Task Scheduling For the tasks sorted by priority in the "task pool", it is necessary to assign appropriate mobile inspection terminals to them. This is a multi-objective "task allocation problem", and this embodiment uses a greedy algorithm based on a cost function to approximate the solution.

[0061] Specifically, in this embodiment, a greedy algorithm is used to calculate the task allocation cost: in: is the allocation cost function. Mobile inspection terminal Move to a task from your current location The estimated time to reach the target location is calculated by the A* (A-Star) pathfinding algorithm. Mobile inspection terminal Capability matching, the task will recommend inspection items (such as "infrared temperature measurement") based on the source of risk; if the terminal If there is no corresponding sensor (such as infrared thermal imager), (a very large number); if so, . Mobile inspection terminal The current load of a task is simply defined as the length of its task queue or the inverse of the remaining power. 、 and is the cost weight, in this embodiment, , , .

[0062] The dispatch results are sent to the corresponding mobile inspection terminals through the system control and dispatch module.

[0063] Step 5: Identify safety hazards in the data returned by the mobile inspection terminal through the multimodal fusion AI safety hazard identification engine. The multimodal fusion AI safety hazard identification engine integrates the output results of the visual analysis model and the time series analysis model.

[0064] Once the mobile inspection terminal arrives at the designated location and collects data, this step is responsible for accurately identifying safety hazards from the massive amount of data. This embodiment uses a "multimodal fusion AI safety hazard identification engine" rather than a single AI analysis model, including the following content.

[0065] 5.1 Visual Analysis Video and image analysis is performed using computer vision models based on Transformer or CNN architectures. These models are specifically trained to identify safety hazards unique to pharmaceutical production, such as tiny cracks on equipment surfaces, minor leaks from seals, pressure gauges with abnormal readings, damaged cleanroom clothing, and unusual powder or liquid residue on the floor. In this example, the visual analysis model uses the YOLOv7 object detection network trained on an industrial defect dataset.

[0066] 5.2 Timing Analysis Using time series models such as LSTM (Long Short-Term Memory) networks, we analyze data such as equipment vibration and current to identify periodic failures and predict mechanical and progressive failures such as bearing wear and motor aging, thereby determining the equipment's operating status. In this embodiment, the time series analysis model uses an LSTM network.

[0067] 5.3 Multimodal Fusion Decision The preliminary analysis results (confidence scores) from the visual and time series models, along with the environmental sensor data at that time, are fed into a top-level decision model (such as a gradient boosting decision tree or a small neural network). This model learns the association rules between different information sources. For example, if the three conditions of "infrared image shows localized high temperature" + "abnormal peak in the vibration spectrum" + "slight increase in device current" are simultaneously met, the fused decision model can determine with high confidence that "device overload or impending short circuit fault" exists, even if each condition alone is not sufficient to trigger an alarm.

[0068] In this embodiment, multimodal fusion uses the XGBoost classifier as the fusion decision model. Input features include visual confidence, device operating status, temperature deviation, gas concentration change rate, and pressure fluctuation variance. The pressure fluctuation variance is the standard deviation of the pressure values ​​within a set time period.

[0069] After the mobile inspection terminal completes its task and transmits data back, the multimodal fusion AI safety hazard identification engine is responsible for in-depth analysis and identification of safety hazards. For example, the identification of a "minor flange leak" on the external pipeline of the reactor includes: (1) Data input: 4K visible light video stream sent back by the robot; video stream from the FLIR A700 thermal imager; data from the VOCs concentration sensor in the air of the area.

[0070] (2) Multimodal fusion based on multimodal fusion AI safety hazard identification engine Visual Analysis: A YOLOv7 object detection model, pre-trained on the COCO dataset and fine-tuned on a self-built industrial defect dataset, was used. This model was trained to identify extremely subtle "water" or "oil" stains on flange joints caused by liquid infiltration.

[0071] Infrared analysis: A relatively simple image processing algorithm is used to analyze the infrared video stream and output an abnormal signal if the temperature of the flange is significantly lower or higher than the temperature of the pipe body (possibly due to gas throttling cooling or liquid leakage removing heat).

[0072] Fusion decision model: This embodiment uses an XGBoost (Extreme Gradient Boosting) classifier as the fusion decision model. Its input feature vector includes: Feature 1: The highest confidence score (0.0 ~ 1.0) of the "suspected water stain" object detected by the YOLOv7 model.

[0073] Feature 2: Temperature constant output by the infrared analysis model (a normalized floating point number).

[0074] Feature 3: The rate of change of the VOCs concentration in the area compared to the average value of the past hour.

[0075] Feature 4: The pressure fluctuation variance of the pressure sensor inside the pipeline.

[0076] (3) Decision-making process When the robot is inspecting, the AI ​​engine processes the above data in real time: YOLOv7 might detect a very blurry “water stain” with a confidence score of only 0.65 (below the threshold of 0.85 for a separate alarm).

[0077] At the same time, the infrared model found that the flange temperature was 0.5°C lower than the surrounding area, with an anomaly score of 0.4.

[0078] At the same time, the VOCs concentration was 10% higher than usual, with a change rate of 0.1.

[0079] The pipeline pressure is stable and the fluctuation variance is 0.01.

[0080] The eigenvector fed into the XGBoost model is [0.65, 0.4, 0.1, 0.01]. Because the model has learned during training that the combination of "low-confidence water stains" + "slight temperature difference" + "increased VOCs" is highly likely to indicate an early leak, it may ultimately output a "confirmed leak" probability of up to 0.95, immediately triggering a high-level alarm. The location is highlighted on the central control screen and an alarm message is pushed to the relevant personnel's mobile app.

[0081] Specifically, the types of security risks identified by the multimodal fusion AI security risk identification engine include: (1) Personnel-related safety hazards Operational errors: Insufficient personnel training, lack of concentration, fatigue, violation of standard operating procedures, etc. lead to incorrect feeding, incorrect parameter settings, incomplete cleaning, etc.

[0082] Personnel hygiene: Failure to strictly follow the changing and hand washing disinfection procedures, bringing in microbial contamination or foreign objects; working while sick; inappropriate behavior (such as running or talking in the clean area).

[0083] Insufficient safety awareness: Lack of understanding of potential risks, ignoring safety warnings, and improper use of personal protective equipment.

[0084] Human contamination / deliberate action: In rare cases, there may be malicious actions.

[0085] (2) Safety hazards related to equipment and facilities Equipment failure / aging: The equipment is unstable, the accuracy is inaccurate, and the components are worn or damaged (such as aging of the seal causing leakage, broken agitator blades, and damaged filters).

[0086] Design flaws: The equipment is not designed properly, resulting in blind spots for cleaning, difficulty in sterilization, and the risk of cross-contamination or confusion.

[0087] Improper maintenance: Preventive maintenance plans are missing or not implemented properly, calibration is not timely, and repairs are not adequately confirmed.

[0088] Utility system failure: Problems occur in key utility systems (such as HVAC system failure resulting in excessive clean area temperature, humidity, pressure differential, and particle count; purified water / water for injection system contamination or substandard water quality; compressed air containing oil / water / particles; steam quality not meeting standards).

[0089] Defects in factory facilities: The clean area is not tightly sealed, there are cracks or debris on the surface; the floor drain is improperly designed or maintained; unreasonable design of personnel and logistics leads to crossover.

[0090] (3) Material-related safety hazards Material contamination: Raw materials, auxiliary materials, and packaging materials themselves are contaminated by microorganisms, foreign matter (metal chips, glass, fibers, insects, etc.), and other chemical substances (such as pesticides and detergent residues).

[0091] Material confusion / errors: unclear material labeling, improper storage (such as not storing at the required temperature / humidity), errors in receiving and distributing materials, and incorrect material input (variety, quantity, batch).

[0092] Material quality issues: Suppliers are poorly managed and use materials that do not meet quality standards.

[0093] Failure of control over intermediates / semi-finished products: Insufficient control over the intermediate process allows unqualified intermediates to flow into the next process.

[0094] (4) Environmental health-related safety hazards Chemical hazards: Exposure to toxic, harmful, flammable, explosive, and corrosive chemicals (solvents, active pharmaceutical ingredients, cleaning agents, etc.) without proper protection can lead to poisoning, fire, explosion, and chemical burns.

[0095] Biological hazards: Inadequate protection when handling highly pathogenic or highly allergenic materials may lead to infection or allergies.

[0096] Physical hazards: mechanical injuries (entanglement, crushing, cutting), noise, high / low temperature, electrical safety, slips / trips / falls, etc.

[0097] Occupational exposure: Chronic health effects that may result from long-term exposure to certain substances.

[0098] Step 6: Feedback the safety hazard identification results to the production safety risk assessment model and update the static risk value And optimize the multimodal fusion AI safety hazard identification engine to form a closed-loop control.

[0099] This embodiment possesses the ability to learn and evolve, which is key to achieving "getting smarter with use." Highly suspected security risks identified by the multimodal fusion AI security risk identification engine are pushed to human agents for final confirmation. The result of the human confirmation ("security risk" or "not a security risk") is automatically fed back into the system as a high-quality label.

[0100] Specifically, the self-optimization mechanism of the model is: if a safety hazard is confirmed to be true, the system will automatically and permanently increase the static risk baseline value of the corresponding location , which will make this area receive more "natural attention" in the future.

[0101] Specifically, the AI ​​analysis model is optimized as follows: manually confirmed "positive samples" (true safety hazards) and "negative samples" (false positives) are automatically added to the training sample library of the multimodal fusion AI safety hazard identification engine. The system regularly (e.g., weekly) uses incremental new samples to fine-tune the multimodal fusion AI safety hazard identification engine, continuously improving its recognition accuracy and generalization capabilities and reducing future false positives and missed negatives. Specifically, in this embodiment, manually confirmed safety hazard samples are added to the training library, and the YOLOv7 object detection network and fusion decision model are regularly fine-tuned.

[0102] Taking the identification of "minor flange leakage" on the external pipeline of the reactor as an example, it includes: (1) Manual confirmation and high-quality data annotation When the "flange leak" alarm is triggered, an experienced engineer displays the robot's video and all relevant data on the central control screen. After verifying the situation on-site, he marks the event as "confirmed" in the system.

[0103] This simple “confirmation” action generates a high-quality, labeled training sample in the background: { "input_vector": [0.65, 0.4, 0.1, 0.01], "label": 1 (leak)} At the same time, the image detected by YOLOv7 as containing a real leak is also associated with this label and stored in the "sample library to be trained".

[0104] (2) Model self-optimization Optimization of the production safety risk assessment model: The system background service will execute a database instruction to permanently increase the static risk baseline value of the flange position: UPDATE static_risk_database SET static_risk_score = static_risk_score+ 5.0 WHERE location_id = 'Reactor_3_Flange_12'; This means that even after a repair, the system will be more "mindful" of the problem in the future.

[0105] Model Optimization: When the "training sample library" accumulates, for example, 1,000 new manually verified samples, the system automatically initiates a background retraining task. It uses these high-quality new samples to fine-tune the XGBoost fusion model and the YOLOv7 vision model for one or more epochs. Once training is complete and passes automated testing, the new model is automatically deployed online, replacing the old one.

[0106] The specific implementation of the intelligent inspection system for drug production safety of the present invention is described in detail below in conjunction with the above-mentioned drug production safety robot inspection method of this embodiment.

[0107] like Figure 2 As shown, the intelligent drug production safety inspection system of this embodiment includes an application layer, a cognitive layer, a data transmission layer and a perception layer.

[0108] (1) Perception layer The perception layer is the system's "five senses" and "limbs." In this embodiment, the perception layer includes an environmental sensor array, an equipment status perception unit, a visual / audio acquisition module, a personnel / material location system, a mobile inspection terminal, and a human interaction terminal.

[0109] Environmental sensor arrays are used to collect environmental sensor data. German-branded SICK temperature and humidity sensors, TSI's AeroTrak® 9306 laser particle counters, and Honeywell PID gas sensors for monitoring volatile organic compounds (VOCs) are deployed in key locations such as clean areas and hazardous materials storage areas.

[0110] The equipment status sensing unit collects equipment operating data. It connects directly to the Siemens SIMATIC PCS 7 process control system via the OPC UA (Unified Architecture) protocol, providing real-time readings of key parameters such as the reactor's internal pressure, jacket temperature, agitator motor current, and vibration frequency. Vibration data is collected by a PCBPiezotronics accelerometer mounted on the motor base.

[0111] The visual / audio acquisition module is used to collect visual / audio data. In this example, a Hikvision DS-2CD8A86FWD-XZS 8K ultra-high-definition network camera was deployed on the workshop ceiling. Furthermore, as a mobile inspection unit, this example uses a quadruped robot as an intelligent mobile inspection terminal equipped with a FLIR A700 thermal imager, a DJI Zenmuse P1 visible light gimbal camera, and a Sennheiser MKE 600 directional microphone.

[0112] Personnel / material positioning: Megvii Technology’s facial recognition access control system has been deployed at all key entrances and exits.

[0113] The mobile inspection terminal is used to perform inspection tasks and collect multi-source heterogeneous data through the equipment it carries. In this embodiment, the mobile inspection terminal includes an inspection automation device and / or an inspection drone.

[0114] Manual interaction terminals, on-site operators and inspectors are equipped with industrial-grade explosion-proof smartphones equipped with mobile APPs.

[0115] (2) Cognitive layer The cognitive layer is the "brain" of the system and is deployed on a local cluster consisting of three Dell PowerEdge R750xa servers. The cluster is equipped with NVIDIA A100 Tensor Core GPUs, which are specifically used for parallel training and inference of models. The software of this layer is modular. Specifically, the cognitive layer includes a data preprocessing module, a dynamic risk assessment module, an adaptive task planning module, and a scheduling module. Specifically, the data preprocessing module is used to preprocess the multi-source heterogeneous data collected by the perception layer; the dynamic risk assessment module is used to carry out the production safety risk assessment model to calculate the location At the moment The comprehensive risk index of the safety hazards is determined by the adaptive task planning module; the adaptive task planning module is used to trigger the generation of inspection tasks according to the comprehensive risk index; the scheduling module is used to schedule the mobile inspection terminals to perform inspection tasks; the AI ​​safety hazard identification module is used to carry the multimodal fusion AI safety hazard identification engine and integrate the output results of the visual analysis model and the time series analysis model.

[0116] (3) Application layer The application layer is the window for human-computer interaction, including the large LED central control screen in the central control room, the mobile app used by field personnel, and the alarm terminal. The alarm terminal issues an alarm based on the output of the AI ​​safety hazard identification module, and can provide sound, light, and other alarm methods.

[0117] In this embodiment, the perception layer is connected to the cognitive layer through the data transmission layer, and the output of the AI ​​security hazard identification engine is fed back to the dynamic risk assessment module.

[0118] The above embodiments are merely preferred embodiments for the purpose of fully illustrating the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are within the scope of protection of the present invention. The scope of protection of the present invention shall be subject to the claims.

Claims

1. A pharmaceutical production safety robot inspection method, characterized by: The steps include: Step 1: Acquire multi-source heterogeneous data on drug production safety status in real time. The multi-source heterogeneous data includes environmental sensor data, equipment operation data, visual / audio data, personnel / material location data, and manually entered data; and pre-process the multi-source heterogeneous data, including data cleaning, data format conversion, timestamp alignment, and preliminary feature extraction. Step 2: Based on the multi-source heterogeneous data, a dynamically updated production safety risk assessment model is constructed. The production safety risk assessment model calculates the position by the following formula: At the moment Comprehensive risk index: in: is the static risk value; is the dynamic risk value; is the event risk value, 、 and is the adjustable weight coefficient; is the risk mapping function; Step 3: Generate adaptive inspection tasks based on the comprehensive risk index. Exceeding the preset attention threshold When a patrol task is triggered, the patrol task includes the target location, risk type and detection items; Step 4: Dispatching the mobile inspection terminal to perform the inspection task, wherein the dispatching is optimized based on the location of the mobile inspection terminal, the matching degree of sensor capabilities and the current load; Step 5: Identify safety hazards in the data returned by the mobile inspection terminal using a multimodal fusion AI safety hazard identification engine, which integrates the output results of the visual analysis model and the time series analysis model. Step 6: Feedback the safety hazard identification results to the production safety risk assessment model to update the static risk value And optimize the multimodal fusion AI safety hazard identification engine to form a closed-loop control.

2. The pharmaceutical production safety robot inspection method according to claim 1, characterized in that: In the step 1: Environmental sensor data includes: Air pollution detection data is used to collect air pollution factors including physical factors, chemical factors, biological factors and radioactive factors; Water pollution detection data, used to collect water pollution indicators including microbiological indicators, toxicological indicators, sensory properties and general chemical indicators, as well as radioactive indicators; Dust monitoring data, used to detect total dust or respirable dust concentrations in environments where flammable, explosive, or combustible gas mixtures are present; Combustible gas concentration detection data, used to detect the concentration of combustible gas in the air; Thermal infrared imaging sensor data is used to detect the temperature of equipment sealing ports, equipment electrical cabinets, and other locations; Radiation detection data, used to detect residual radioactive doses produced by radiation sterilization; Equipment operation data is read from the equipment control system DCS, including the parameter status of key hazardous sources; Visual / audio data is collected by fixed cameras and mobile inspection terminals; Personnel / material positioning data is obtained through access control, RFID tags, or Beidou / UWB indoor positioning systems to obtain the real-time location and movement trajectory of authorized personnel and key materials; Manually entered data includes safety hazard information reported by inspection personnel that is difficult to detect automatically.

3. The pharmaceutical production safety robot inspection method according to claim 1 or 2, characterized in that: In the step 2: The static risk value Initialize settings based on equipment importance, regional functions and historical accident data; Dynamic Value at Risk It is calculated by the weighted sum of multiple normalized data deviations: in: For and location The number of associated sensors or data sources; For the The weight of the data source, which is the current production process stage The function reflects the changes in focus at different stages; For the Data sources at time The normalized deviation of Event Risk Value Exponential decay when a high-risk event is triggered: in: The moment when the event occurred; is the peak event risk; is the decay constant.

4. The pharmaceutical production safety robot inspection method according to claim 3, characterized in that: In the step three: When the comprehensive risk index Exceeding the emergency threshold Mark as the highest priority task; The detection items are dynamically specified according to the risk type, including infrared temperature measurement, high-definition photography and gas sampling.

5. The pharmaceutical production safety robot inspection method according to claim 1, characterized in that: In step 4, the task allocation cost is calculated using a greedy algorithm: in: Mobile inspection terminal Move to a task from your current location Estimated time to target location; Mobile inspection terminal Ability matching; Mobile inspection terminal The current load; 、 and is the cost weight.

6. The pharmaceutical production safety robot inspection method according to claim 1, characterized in that: In the step five: The visual analysis model uses the YOLOv7 object detection network trained on an industrial defect dataset; The time series analysis model uses LSTM network; Multimodal fusion uses the XGBoost classifier as the fusion decision model, and the input features include visual confidence, equipment operating status, temperature difference anomaly, gas concentration change rate and pressure fluctuation variance.

7. The pharmaceutical production safety robot inspection method according to claim 6, characterized in that: The step six comprises: After confirming the safety hazards, increase the static risk value of the corresponding location ; Manually confirmed safety hazard samples are added to the training library, and the YOLOv7 target detection network and fusion decision model are regularly fine-tuned.

8. A pharmaceutical production safety intelligent inspection system, for implementing the pharmaceutical production safety robot inspection method according to any one of claims 1 to 7, characterized in that: Includes application layer, cognitive layer, data transmission layer and perception layer; The perception layer includes: an environmental sensor array for collecting environmental sensor data; Equipment status perception unit, used to collect equipment operation data; Visual / audio acquisition module, used to collect visual / audio data; Mobile inspection terminals are used to perform inspection tasks and collect multi-source heterogeneous data through the equipment they carry; The cognitive layer includes: A data preprocessing module, used to preprocess the multi-source heterogeneous data collected by the perception layer; Dynamic risk assessment module, used to carry production safety risk assessment model to calculate location At the moment Comprehensive risk index; Adaptive task planning module, used to trigger the generation of inspection tasks based on the comprehensive risk index; The scheduling module is used to schedule the mobile inspection terminal to perform inspection tasks; The AI ​​safety hazard identification module is used to carry a multimodal fusion AI safety hazard identification engine and integrate the output results of the visual analysis model and the time series analysis model; The application layer includes a central control screen, a mobile app, and an alarm terminal. The alarm terminal issues an alarm based on the output of the AI ​​safety hazard identification module. The perception layer is connected to the cognitive layer through the data transmission layer, and the output of the AI ​​safety hazard identification engine is fed back to the dynamic risk assessment module.

9. The intelligent drug production safety inspection system according to claim 8, characterized in that: The mobile inspection terminal includes an inspection automation device and / or an inspection drone.

Images