Laboratory intelligent inspection method and system based on multiple data sources

Abstract

The invention discloses a laboratory intelligent inspection method and system based on multiple data sources, and solves the problems that in the prior art, a laboratory monitoring mode is single, the monitoring range is limited, and potential safety hazards cannot be found in time. Determining an inspection route; collecting laboratory environment data in real time, reading an equipment label on an inspection route, obtaining equipment basic data, and collecting equipment image and video data; potential risks of environmental changes are judged, multiple target recognition and detection algorithms are combined and applied to the same image, different types of abnormities of equipment are detected, and the detected abnormities are classified; and determining a fault type according to an anomaly classification result, and generating a fault report. The laboratory environment and all devices in the laboratory can be detected in all directions, intelligent and convenient inspection operation is achieved, and normal operation and safety of the laboratory devices are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of laboratory monitoring, and in particular to a laboratory intelligent inspection method and system based on multiple data sources. Background Art

[0002] The laboratory is a key place for scientific research, technological development, product inspection and other activities. Therefore, laboratory safety management is of vital importance. By regularly conducting comprehensive inspections and evaluations of laboratory equipment, potential safety hazards can be discovered and resolved in a timely manner, the possibility of accidents can be reduced, and the safety of employees can be guaranteed. This is especially important for laboratories in scientific research, medical care, industry and other fields, because equipment failure or improper operation in these fields may cause serious personal injury or property loss. Generally, laboratory personnel are required to patrol the laboratory at all times to understand the laboratory's equipment conditions and experimental data, but it is impossible for laboratory personnel to patrol the laboratory at all times, and the laboratory's equipment conditions and equipment data cannot be monitored in real time, which brings inconvenience to laboratory personnel to understand the laboratory situation in real time.

[0003] At present, although some laboratories have adopted some traditional monitoring systems to monitor the safety of the laboratory, such as installing cameras and access control systems, these systems have some problems. First, the monitoring range of traditional monitoring systems is limited, and it is impossible to fully monitor the personnel and equipment in the laboratory, and it is impossible to monitor a single device; secondly, the monitoring method of traditional monitoring systems is single, which can only be achieved through video monitoring, and the monitoring content is single, which may only monitor a certain part, and it is impossible to monitor the environmental factors in the laboratory in real time. Summary of the invention

[0004] The purpose of the present invention is to solve the problems in the prior art that the laboratory monitoring method is single, the monitoring range is limited, and the laboratory safety hazards cannot be discovered in time. A laboratory intelligent inspection method and system based on multiple data sources is provided to achieve efficient management and safety monitoring of materials and equipment in the laboratory, and can realize all-round detection of the laboratory environment and various equipment in the laboratory, and realize intelligent and convenient inspection operations, thereby ensuring the normal operation and safety of laboratory equipment.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions: A laboratory intelligent inspection method based on multiple data sources, characterized in that it includes the following steps: S1: Obtain the inspection plan, confirm the equipment list to be inspected and the key inspection areas, and determine the inspection route; S2: Collect laboratory environment data in real time, read equipment tags on the inspection route, obtain basic equipment data, and collect equipment images and video data; S3: Identify potential dangers of environmental changes and apply a combination of multiple target recognition and detection algorithms to the same image to detect different types of anomalies in the device and classify the detected anomalies; S4: According to the abnormal classification results, determine the fault type, generate an inspection report, and complete the inspection.

[0006] Through the method of the present invention, laboratory equipment and laboratory environment (such as temperature, humidity, sound, etc.) can be fully monitored, and a variety of target recognition and detection algorithms are used to process the collected equipment images to promptly detect abnormal situations and issue alarm prompts. At the same time, laboratory managers can also receive alarm information at any time in mobile mailboxes, cloud platforms, and mobile phones, which is convenient for administrators to check the laboratory status anytime and anywhere. The monitoring methods are diverse and wide-ranging, including not only video acquisition image monitoring and processing for personnel and equipment, but also physical sensor monitoring for laboratory environments. A variety of target recognition and detection algorithms have been introduced to further improve the accuracy of detection results.

[0007] Preferably, step S3 comprises: Pre-process the collected environmental data to identify abnormal patterns in the environmental data and determine significant changes or potential hazards in the laboratory environment; Obtain historical equipment images and video data and preprocess them, and annotate the preprocessed images for abnormal conditions to obtain a data set; Based on the data set, the neural network algorithm is used to learn the normal and abnormal status of the equipment and build a recognition model; The recognition model is used to locate all potential abnormal areas in the acquired images and classify the abnormal areas.

[0008] Preferably, the image is divided into several grids, and multiple candidate boxes are generated in each grid, each of which is provided with a confidence score; the overlap between the confidence score of each candidate box and the actual anomaly is calculated, and the candidate boxes with confidence scores higher than a preset threshold are defined as target areas.

[0009] Preferably, the classifying of abnormal areas comprises: extracting image features from each candidate box; inputting the extracted image features into a classification neural network, the classification neural network analyzes the features and assigns a category label to each area, and attaches a confidence score.

[0010] Preferably, if experimental personnel enter the laboratory during the inspection, record the area where the experimental personnel are located. If the experimental personnel enter the key inspection area, stop the inspection after completing the inspection, obtain the environmental data of the key inspection area, monitor environmental changes, and after the experimental personnel leave, inspect the equipment in the key inspection area. After the inspection is completed, continue to implement the inspection plan.

[0011] Preferably, adjacent or overlapping abnormal regions in the image are merged into one large region.

[0012] For example, if there is a crack in the equipment, it is detected as several different small areas. The merge operation will combine these small areas into a large area, so that only one crack is shown in the inspection report instead of multiple small cracks.

[0013] Preferably, the YOLO algorithm is used to detect surface defects of the device casing to determine whether there are scratches or damages on the device appearance; the SSD algorithm is used to detect the circuits and components of the device to identify aging circuits and missing parts.

[0014] Different algorithms are used to detect different fault types, with higher recognition accuracy and more accurate detection results.

[0015] Preferably, step S4 includes: generating an inspection report, feeding back the current status of the equipment and detected abnormality information, marking the category and location of each abnormal area, and providing explanations about the abnormalities and suggested treatment measures.

[0016] A laboratory intelligent inspection system based on multiple data sources, comprising: Multi-source inspection terminal, inspect the laboratory according to the inspection plan, obtain basic equipment information, collect laboratory environment data and equipment image data; The fault detection module pre-processes the image data uploaded by the multi-source inspection terminal, establishes a recognition model, locates potential abnormal areas in the image according to the confidence level, and classifies the abnormal areas; Database, which stores detailed records of each inspection and problem handling, generates equipment history archives, and records the frequency and trend of equipment failures; The inspection result feedback module generates inspection reports, records basic equipment information and current abnormal status, generates work orders and tracks them in real time.

[0017] At the same time, not only can the collected data and results be stored and shared in the cloud, making it convenient to view and download experimental data and results anytime and anywhere, but algorithms can also be used to analyze the data archived in the database to promptly discover potential safety hazards in the laboratory and other problems.

[0018] Preferably, the multi-source inspection terminal comprises: QR code and RFID identification module, reads equipment tags, obtains equipment information and records it in inspection data; Image and video acquisition module, which performs all-round image acquisition or video recording of the device to capture device dynamics; The thermal status detection module captures the thermal image of the device, obtains the surface temperature distribution of the device, and determines whether the device is overheated.

[0019] Therefore, the present invention has the following beneficial effects: 1. It can conduct comprehensive monitoring of laboratory equipment and laboratory environment (such as temperature, humidity, sound, etc.), and also uses a variety of target recognition and detection algorithms to process the collected equipment images to promptly detect abnormal situations and issue alarm prompts, thereby improving the detection accuracy and achieving efficient management and safety monitoring of materials and equipment in the laboratory. It can achieve all-round detection of the laboratory environment and various equipment in the laboratory, and realize intelligent and convenient inspection operations, thereby ensuring the normal operation and safety of laboratory equipment.

[0020] 2. Not only can the collected data and results be stored and shared in the cloud, making it convenient to view and download experimental data and results anytime and anywhere, but algorithms can also be used to analyze the data archived in the database to promptly discover potential safety hazards in the laboratory and other problems, thereby ensuring the normal operation and safety of laboratory equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 The figure is a flowchart of the overall steps of the laboratory intelligent inspection method based on multiple data sources in the present invention.

[0022] Figure 2 This is an architectural diagram of the laboratory intelligent inspection system with multiple data sources in the present invention.

[0023] In the figure: 1. Multi-source inspection terminal; 2. Fault detection module; 3. Database; 4. Inspection result feedback module; 5. Mobile terminal; 6. QR code and RFID recognition module; 7. Image and video acquisition module; 8. Thermal state detection module; 9. Data preprocessing module; 10. Model training module; 11. Detection and classification module; 12. Fault type recognition module. DETAILED DESCRIPTION

[0024] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments: Embodiment 1: This embodiment provides a laboratory intelligent inspection method based on multiple data sources, such as Figure 1 As shown in the figure, the operation process is as follows: Step 1, obtain the inspection plan, confirm the equipment list to be inspected and the key inspection areas, and determine the inspection route; Step 2, collect laboratory environmental data in real time, read the equipment labels on the inspection route, obtain basic equipment data, and collect equipment images and video data; Step 3, determine the potential dangers of environmental changes, and apply a combination of multiple target recognition and detection algorithms to the same image, detect different types of equipment anomalies, and classify the detected anomalies; Step 4, determine the fault type based on the anomaly classification results, generate an inspection report, and complete the inspection.

[0025] Multi-source data includes environmental data and image data. The laboratory intelligent inspection method based on multi-source data provided in this embodiment can conduct comprehensive monitoring of laboratory equipment and laboratory environment (such as temperature, humidity, sound, etc.), and also uses a variety of target recognition and detection algorithms to process the collected equipment images, so that the accuracy of the detection results is further improved, and abnormal situations can be discovered in time to issue alarm prompts. At the same time, laboratory managers can also receive alarm information at any time in their mobile mailboxes, cloud platforms, and mobile phones, so that administrators can check the laboratory status anytime and anywhere. The monitoring methods are diverse and wide-ranging, including not only video acquisition image monitoring and processing for personnel and equipment, but also physical sensor monitoring for laboratory environments, thereby ensuring the normal operation and safety of laboratory equipment.

[0026] The following further illustrates the technical solution and technical effects of the present invention through specific examples and specific application scenarios. The following examples are explanations of the present invention and the present invention is not limited to the following examples.

[0027] like Figure 1 As shown, the laboratory intelligent inspection method based on multiple data sources provided in this embodiment includes the following steps: Step 1: Obtain the inspection plan, confirm the equipment list to be inspected and the key inspection areas, and determine the inspection route.

[0028] First, when the equipment enters the laboratory, a QR code label or RFID label is prepared for the equipment. The label provides basic information and unique identification of the equipment, including basic information such as equipment number, equipment ID, model, manufacturer, installation date and maintenance history, as well as the location of the equipment in the laboratory. When the equipment enters the laboratory, it needs to scan the label so that the management personnel can obtain the equipment information in the laboratory in real time.

[0029] Obtain a panoramic image of the laboratory, mark the location of the equipment in the panoramic image, formulate an inspection plan, divide the laboratory into several inspection areas, and ensure that all equipment in the laboratory can be inspected in the shortest time possible.

[0030] According to the formulated inspection plan, the inspector obtains a list of equipment to be inspected (the equipment in the list is sorted according to the numbers on the equipment QR code labels) and key inspection areas.

[0031] Key inspection areas include: areas prone to temperature changes, areas sensitive to temperature changes, areas sensitive to humidity changes, areas prone to chemical leaks, and areas sensitive to light or sound, etc.

[0032] Step 2: Collect laboratory environment data in real time, read equipment tags on the inspection route, obtain basic equipment data, and collect equipment images and video data.

[0033] (1) Laboratory environment data collected include: The temperature, humidity and light data of the overall laboratory environment are collected. At the same time, in each key inspection area, the environmental data of the key inspection area is collected according to the type of the key inspection area.

[0034] For example, set up temperature sensors in areas prone to temperature changes and areas sensitive to temperature changes to collect temperature data; set up humidity sensors in areas sensitive to humidity changes in the laboratory to collect humidity data; set up light sensors in areas sensitive to light in the laboratory to collect light data; set up sound sensors in areas sensitive to sound in the laboratory to collect volume data; set up gas sensors in areas prone to chemical leaks in the laboratory. Realize multi-physical quantity monitoring of the laboratory environment to accurately capture various environmental changes in the laboratory.

[0035] In this embodiment, a voltage / current sensor may be provided between the power supply terminal of the equipment in the laboratory and the laboratory power grid to collect voltage and current information of the laboratory equipment.

[0036] (2) Image and video data collected by the equipment include: According to the inspection route, go to the equipment, scan the label on the equipment, obtain the basic information of the equipment and record it in the inspection data.

[0037] The equipment is then imaged or video recorded in all directions, including detailed images of the equipment's appearance, key components, and operating status.

[0038] In this embodiment, it also includes collecting thermal status data of the device, capturing the thermal image of the device, recording the temperature distribution on the surface of the device, and determining whether the device has abnormal overheating.

[0039] Step 3: Determine the potential danger of environmental changes and apply a combination of multiple target recognition and detection algorithms to the same image to detect different types of anomalies in the equipment and classify the detected anomalies.

[0040] Preprocess the collected environmental data, identify abnormal patterns in the environmental data, and determine significant changes or potential dangers in the laboratory environment. Alarms are issued when the environment changes: for example, when the temperature or humidity reaches a threshold, when the light intensity or decibel reaches a threshold, when leaked gas is collected, etc.

[0041] Acquire images or video data of historical equipment under different environmental conditions, and capture the video data into images. These images contain various possible equipment states and problems, such as normal, damaged, aging, rust, etc.

[0042] Preprocess the image data, including scaling, cropping and standardizing the image to meet the input requirements of the neural network. It also includes data enhancement of the image, such as flipping, rotating, and adjusting brightness, to increase the diversity of training data and improve the robustness of the model.

[0043] The processed images are deeply annotated, including fault area, fault type, timestamp, environmental conditions, etc.

[0044] The training data consists of the large number of labeled device images mentioned above. Through these data, the neural network learns the normal and abnormal states of the equipment and forms a recognition model.

[0045] During training, the recognition model parameters are continuously adjusted to enable the recognition model to adapt to the changing laboratory scenes. After the training is completed, the image that triggers the abnormal signal each time is included in the newly collected image data, and the annotation training is performed to update the iterative model.

[0046] It should be noted that, in this embodiment, the recognition model includes multiple target detection algorithms, and multiple target recognition detection algorithms are combined and applied to the same image. Each algorithm focuses on a specific fault type and detects different types of abnormalities in the equipment, thereby comprehensively covering various equipment fault types.

[0047] Specific: YOLO is used to detect appearance defects and parts problems of the device, and to identify cracks and damage on the device casing.

[0048] YOLO (You Only Look Once): YOLO is a real-time object detection algorithm that completes image classification and localization in a single neural network forward propagation. This means that YOLO can process a large number of images in a very short time, which is very suitable for real-time applications. YOLO's architecture achieves object detection by dividing the input image into multiple grids and predicting bounding boxes and class probabilities in each grid.

[0049] Use SSD to inspect the circuits and components of the equipment, such as identifying aging wires, missing parts, etc.

[0050] SSD (Single Shot MultiBox Detector): Similar to YOLO, SSD is also a real-time algorithm for object detection. SSD predicts multiple object categories and bounding boxes in an image through a single convolutional neural network. The advantage of SSD is that it can handle objects of different scales, especially with high accuracy when detecting small objects.

[0051] Using the recognition model, the device is identified, all potential abnormal areas are located in the acquired device images, and the abnormal areas are classified to determine whether they are in a normal state or a specific abnormal situation.

[0052] Specific: (1) Locate the abnormal area.

[0053] 1) Image segmentation and gridding: The image is divided into several grids, each of which is responsible for detecting whether there are abnormal objects in the image. In this way, the object detection algorithm can perform anomaly detection independently in different areas.

[0054] 2) Candidate box generation: In each grid, multiple candidate boxes are generated, which may contain potential abnormal areas. Each candidate box has a confidence score to indicate the possibility of an abnormality in the box.

[0055] 3) Box screening and positioning: By calculating the confidence score of each candidate box and the overlap with the actual anomaly, the most likely anomaly area is screened out. Candidate boxes with confidence scores higher than the preset threshold are retained, and candidate boxes with confidence scores lower than the preset threshold are excluded. These high-confidence candidate boxes are the located anomaly areas.

[0056] (2) Classification of abnormal areas.

[0057] Feature extraction: Extract image features (including color, texture, shape, etc.) from each candidate box to describe the appearance and content of the area.

[0058] Classification network processing: The extracted image features are input into a classification neural network (which has been trained with a large amount of labeled data and can identify various types of anomalies). The classification network analyzes the image features and assigns a category label to each area, such as "crack", "rust" or "leakage".

[0059] Result output: Each abnormal area is classified as a specific type and is accompanied by a confidence score indicating the reliability of the classification result.

[0060] Finally, the recognition results obtained by the recognition model are: the location of the equipment anomaly, the anomaly type and the confidence level.

[0061] Remove low-confidence test results: Filter out low-confidence test results. Low-confidence results may be false positives or inaccurate, so they need to be removed to improve the accuracy of the final result.

[0062] Merge adjacent abnormal regions: Merge adjacent or overlapping abnormal regions in the image into one large region. For example, if there is a crack in the image device and it is detected as several different small regions, the merge operation will combine these small regions into one large region, so that only one crack is shown in the report instead of multiple small cracks. This process ensures that each actual abnormality is marked and recorded only once. Through this merge, the same problem is avoided from being marked repeatedly in the report. This can more accurately reflect the actual situation of the abnormality and improve the clarity of the report.

[0063] Step 4: Determine the fault type based on the abnormal classification results, generate an inspection report, and complete the inspection.

[0064] Identify fault types including: Appearance scratches or damage: Detect surface defects of the device casing, such as cracks, scratches, dents, etc.

[0065] Line aging: Identify signs of aging such as color changes in wires and damage to insulation.

[0066] Rust condition: Check the surface rust of metal parts to determine its severity.

[0067] Missing or Loose Parts: Identify parts that are missing or improperly installed on your equipment.

[0068] Liquid leakage: Through visual identification, detect whether there is liquid leakage on the surface of the equipment.

[0069] Generate inspection report: Based on the identification results, a detailed report is generated, including all detected anomaly information. The report will mark the category and location of each abnormal area, and provide explanations about the anomaly and recommended treatment measures.

[0070] The laboratory intelligent inspection method based on multiple data sources provided in this embodiment can comprehensively improve the management capabilities of personnel, materials, equipment and safety inspections, achieve efficient management and safety monitoring of materials and equipment in the laboratory, optimize management processes, improve management efficiency, and reduce labor costs.

[0071] Embodiment 2: This embodiment provides a laboratory intelligent inspection method based on multiple data sources, which further optimizes the solution of embodiment one.

[0072] If a laboratory personnel enters during the inspection, the current inspection will be stopped and the current inspection process will be recorded. If the laboratory personnel enters the key inspection area, the inspection equipment will only perform environmental detection. At the same time, the monitoring device installed in the area (if no monitoring device is installed, the inspection equipment will come to the key area where the current laboratory personnel is located) will be used to obtain the laboratory personnel's operation image, and the YOLO algorithm will be used to determine whether the laboratory personnel's operation is correct. After the laboratory personnel finish, if the area where the laboratory personnel is located has been detected during this inspection, the equipment in the area where the laboratory personnel is located will be re-detected and the detection record will be updated, and the laboratory personnel's operation record will be recorded at the same time; if the area where the laboratory personnel is located has not been detected during this inspection, the laboratory personnel's operation record will be recorded, and the inspection will be returned to the previously recorded inspection to continue the inspection. When the laboratory personnel operate in the key inspection area, safety accidents may occur due to improper operation. Therefore, it is necessary to monitor the laboratory personnel's operation process to ensure laboratory safety.

[0073] If the experimenter is conducting an inspection in a non-key area, determine whether the inspection area where the experimenter is located has been inspected. If so, continue the current inspection. After the inspection is completed, suspend the inspection. After the experimenter finishes, re-inspect the equipment in the area where the experimenter is located and update the inspection record. At the same time, record the experimenter's operation record, and use the inspection record update time as the end time of the inspection. If the inspection is for testing, determine the inspection area where the experimenter is located, stop the inspection when the inspection reaches the previous inspection area of ​​the inspection area, and continue the inspection after the experimenter finishes, and record the experimenter's operation record.

[0074] The experimenter's operation record includes the experimenter's identity, the time the experimenter entered the laboratory, the area the experimenter was inspecting, the equipment used by the experimenter during the operation, and the time the experimenter left the laboratory.

[0075] After a patrol inspection is completed, a check is performed to determine whether all the equipment on the equipment list to be inspected has completed a patrol inspection. If so, it means that the patrol inspection is completed, and the next patrol inspection is performed after a preset time along the reverse path of the previous patrol inspection.

[0076] Embodiment three: This embodiment provides a laboratory intelligent inspection system based on multiple data sources, which is used to execute the laboratory intelligent inspection method based on multiple data sources provided in Example 1 to comprehensively improve the management capabilities of laboratory personnel, materials, equipment and safety inspections.

[0077] A laboratory intelligent inspection system based on multiple data sources includes: a multi-source inspection terminal 1, a fault detection module 2, a database 3, an inspection result feedback module 4 and a mobile terminal 5, wherein the multi-source inspection terminal is connected to the fault detection module and the database respectively, the fault detection module is connected to the database and the inspection result feedback module respectively, the database is connected to the inspection result feedback module, and the mobile terminal is connected to the database and the inspection result feedback module respectively.

[0078] The laboratory includes an electric power system, in which the condition of each device is particularly important. The condition of the equipment includes whether the appearance of the equipment is scratched or damaged, whether the line is aging, whether the equipment is rusted, whether parts are missing, whether the equipment is leaking, etc. Therefore, this embodiment uses a multi-source inventory terminal and combines visual image recognition to identify the equipment condition. Specifically, this embodiment trains images of the equipment under different conditions through target detection algorithms such as YOLO and SSD, and uses the trained algorithm to automatically detect the equipment condition. Compared with manual inspections, the visual detection method is more efficient and can detect problems in a timely manner. Some judgment rules are also loaded behind the algorithm to take corresponding measures for the recognition results, so as to truly achieve intelligent recognition.

[0079] The functions of each module in the laboratory intelligent inspection system based on multiple data sources are further explained below.

[0080] Multi-source inspection terminal is an intelligent device that integrates multiple sensors such as high-definition cameras, infrared sensors, RFID readers, etc. Multi-source inspection terminal has high-precision data collection capabilities and can quickly obtain the appearance image and operating status information of the equipment. Its portability and versatility make it suitable for inspection tasks in different environments and equipment types.

[0081] Specific: Before the inspection begins, the inspector needs to confirm the various states of the multi-source inspection terminal, including battery power, storage capacity, and sensor functions. Check the battery power of the multi-source inspection terminal to ensure that the multi-source inspection terminal can support the entire inspection process; confirm that the storage capacity of the multi-source inspection terminal is sufficient to store all data; and test the working status of the multi-source inspection terminal's camera, infrared sensor and other sensors to ensure that they can work normally.

[0082] When testing the battery power, record the current battery remaining, the power consumption of one inspection and the standby power consumption of the multi-source inspection terminal during two inspections. At the same time, reserve the power consumption required by the experimenters if they enter during the inspection (take the average value of multiple measurements) to determine whether the current battery power meets the power demand of one inspection.

[0083] During inspection, the multi-source inspection terminal can be carried into the laboratory by the inspection personnel to collect data. The personnel need to check the inspection plan, including the equipment list to be inspected and the areas of focus. It can also be mounted on the AGV car, and the inspection plan is imported into the AGV car, and the AGV car can drive along the preset path. The AGC car is equipped with a mechanical arm that can rotate at multiple angles. The multi-source inspection terminal is installed on the mechanical arm so that it can realize all-round image or video acquisition of the equipment.

[0084] Among them, the multi-source inspection terminal includes: (1) QR code and RFID identification module 6, which performs QR code and RFID identification of equipment. The inspection personnel use the RFID reader or QR code scanner on the multi-source inspection terminal to scan the label on the equipment. Each device is equipped with a unique QR code or RFID tag, which provides basic information and unique identification of the device. The basic information of the device including the device ID, model, manufacturer, installation date and maintenance history is automatically obtained through the scanning system of the multi-source inspection terminal and recorded in the inspection data.

[0085] The QR code or RFID tag of the equipment comes from when the equipment enters the laboratory. The QR code or RFID tag is used to track the equipment throughout the entire process, including the entire process of equipment entering and leaving the laboratory and equipment disposal, to ensure the authenticity and traceability of the equipment.

[0086] (2) Image and video acquisition module 7: Inspection personnel use the high-definition camera of the multi-source inspection terminal to take photos or record videos of the equipment in all directions. The images taken include detailed images of the equipment's appearance, key components, and operating status. Special attention is paid to whether the equipment's casing is damaged or parts are missing.

[0087] (3) Thermal status detection module 8. The multi-source inspection terminal includes an infrared sensor for detecting the thermal status of the equipment. By capturing thermal images, the system can record the temperature distribution on the surface of the equipment and check whether there is abnormal overheating. This is especially important for high-temperature equipment, as overheating may indicate potential failure or danger.

[0088] The QR code data, images, videos and infrared sensor data collected by the multi-source inspection terminal are uploaded to the system database for storage and uploaded to the fault detection module for preliminary processing to ensure the integrity and accuracy of the data. After the fault detection module performs preliminary processing, the processed data is analyzed for fault identification.

[0089] The fault detection module stores AI algorithms that can identify equipment conditions. It trains images of equipment in different conditions through target detection algorithms such as YOLO and SSD. The trained algorithms can automatically detect equipment conditions. Compared with manual inspections, visual inspection methods are more efficient and can detect problems in a timely manner.

[0090] At the same time, after target detection and classification, the fault detection module will post-process the results, including removing low-confidence detection results and merging adjacent abnormal areas.

[0091] Specifically, the fault detection module includes: (1) Data preprocessing module 9: After the data from the multi-source inspection terminal is input, the data preprocessing module will preprocess the collected images and videos. Data preprocessing includes image scaling, cropping, and standardization to meet the input requirements of the neural network. In addition, data enhancement is performed, such as flipping, rotation, brightness adjustment, etc., to increase the diversity of training data and improve the robustness of the model.

[0092] (2) Model training,10,The training data consists of a large number of labeled equipment images, which contain various possible equipment states and problems, such as normal, damaged, aging, rusted, etc. Through these data, the neural network learns the normal and abnormal states of the equipment and forms a recognition model.

[0093] (3) Detection and classification module 11. The detection and classification module analyzes the images and videos uploaded by the inspection terminal. First, the target detection algorithm locates all potential abnormal areas in the image. Then, the system classifies these abnormal areas to determine whether they are normal or a specific abnormal situation. The detection results include the location, type and confidence of the abnormality.

[0094] Among them, locating all potential abnormal areas includes: 1) Image segmentation and gridding: The image is segmented into several grids. Each grid is responsible for detecting whether there are abnormal objects in the image. In this way, the detection algorithm can perform anomaly detection independently in different areas.

[0095] 2) Candidate box generation: In each grid, the algorithm generates multiple candidate boxes that may contain potential abnormal areas. Each candidate box has a confidence score that indicates the possibility of an abnormality in the box.

[0096] 3) Candidate box screening and positioning: The algorithm screens out the most likely abnormal area by calculating the confidence score of each candidate box and the overlap with the actual abnormality. Boxes with high confidence are retained, and boxes with lower confidence are excluded. These high-confidence boxes are the located abnormal areas.

[0097] Classification of abnormal areas includes: 1) Feature extraction: Extract image features from each candidate box, which are used to describe the appearance and content of the region. These features include color, texture, shape, etc.

[0098] 2) Classification network processing: The extracted features are fed into a classification neural network that has been trained with a large amount of labeled data to identify various types of anomalies. The classification network analyzes the features and assigns a class label to each region, such as "crack," "rust," or "leakage."

[0099] 3) Result output: Each abnormal area is classified into a specific type and attached with a confidence score, indicating the reliability of the classification result. The final detection result includes the location, type and classification confidence of the abnormality.

[0100] (4) A fault type identification module 12 identifies various fault types based on the results of the detection and classification modules.

[0101] like: Appearance scratches or damage: Detect surface defects of the device casing, such as cracks, scratches, dents, etc.

[0102] Line aging: Identify signs of aging such as color changes in wires and insulation damage.

[0103] Rust condition: Check the surface rust of metal parts to determine its severity.

[0104] Missing or Loose Parts: Identify parts that are missing or improperly installed on your equipment.

[0105] Liquid leakage: Through visual identification, detect whether there is liquid leakage on the surface of the equipment.

[0106] The fault detection module contains multiple algorithms. In actual application, multiple algorithms are combined and applied to the same image to comprehensively detect different types of anomalies. Each algorithm focuses on a specific fault type. YOLO is responsible for detecting appearance defects and parts problems of the device, and SSD is responsible for detecting circuit aging and rust. In this way, various types of equipment faults can be fully covered.

[0107] The inspection result feedback module sends the analysis results to the inspection result feedback module after the fault detection module completes the analysis. The inspection result feedback module generates a detailed inspection report to feedback the current status of the equipment and the problems detected.

[0108] Inspection reports include basic information about the equipment, identified abnormalities, image and video evidence, temperature conditions, and recommended treatment measures. The system automatically sends inspection reports to relevant managers and maintenance teams. According to preset permissions and responsibilities, relevant personnel can view and download reports to facilitate subsequent maintenance and management.

[0109] At the same time, for abnormal situations found during the inspection process, the inspection result feedback module automatically generates a work order and records and tracks it in detail.

[0110] Specifically, the inspection result feedback module generates a work order for each abnormal situation found, recording the fault description, location, severity, recommended treatment measures, etc. The work order also contains the responsible person information to ensure timely handling of the problem. The work order information is automatically archived and entered into the system's workflow management module to track the progress of problem handling.

[0111] Mobile terminals allow managers to view equipment test results, inspection reports, and the processing progress of each work order in real time. The system provides a task status update function, such as "pending", "processing", "resolved", etc., to ensure that each problem can be effectively tracked and resolved.

[0112] After the problem is solved, the system requires relevant personnel to submit the processing results through the terminal, confirm that the problem has been completely solved, and update the maintenance record of the equipment.

[0113] Through mobile terminals, remote monitoring, fault diagnosis and automatic scheduling of equipment can be achieved.

[0114] The mobile terminal includes a display screen, a speaker and a microphone. The display screen is used to display various inspection data, alarm information, work orders, etc. during the inspection process; the speaker is used to alarm when a device failure is detected.

[0115] The database stores QR code data, images, videos and infrared sensor data collected by multi-source inspection terminals. At the same time, the database will archive detailed records of each inspection and problem handling to form a complete historical archive of the equipment. By analyzing historical data, the system can identify the frequency and trend of equipment failures and help managers develop more scientific maintenance strategies. For example, through data analysis, it can be found that a certain type of equipment is prone to a specific type of failure under specific conditions, so that preventive measures can be taken in advance.

[0116] Based on the above, the laboratory intelligent inspection system based on multiple data sources provided by this embodiment has the following beneficial effects: by integrating multiple functional modules, efficient management and safety monitoring of materials and equipment in the laboratory can be achieved, management processes can be optimized, management efficiency can be improved, and labor costs can be reduced. By combining the multi-source inventory terminal with the AI ​​algorithm, intelligent and convenient inspection operations can be achieved, thereby ensuring the normal operation and safety of laboratory equipment.

[0117] Embodiment 4: This embodiment provides a laboratory intelligent inspection system based on multiple data sources, which further optimizes the solution in the second embodiment.

[0118] On the basis of the second embodiment, the laboratory intelligent inspection system based on multiple data sources in this embodiment further includes: Equipment reservation and borrowing system: including online reservation and borrowing system, which simplifies the equipment use process and improves equipment utilization.

[0119] Laboratory quality control management system: including mobile sampling process control system, laboratory video monitoring system and quality monitoring scheduling system, which can significantly improve the authenticity of the source of laboratory items, the standardization of the sampling process and the reliability of the analysis process.

[0120] Laboratory management system: Through personnel identification devices, RFID sensors and transmitters, and central processing modules, automatic identification and management of laboratory equipment and personnel are achieved, improving the efficiency and accuracy of laboratory management.

[0121] The above-described embodiment is only a preferred solution of the present invention and does not limit the present invention in any form. There are other variations and modifications without exceeding the technical solution described in the claims.

Claims

1. A laboratory intelligent inspection method based on multiple data sources, characterized in that: include: S1: Obtain the inspection plan, confirm the equipment list to be inspected and the key inspection areas, and determine the inspection route; S2: Collect laboratory environment data in real time, read equipment tags on the inspection route, obtain basic equipment data, and collect equipment images and video data; S3: Identify potential dangers of environmental changes and apply a combination of multiple target recognition and detection algorithms to the same image to detect different types of anomalies in the device and classify the detected anomalies; S4: According to the abnormal classification results, determine the fault type, generate an inspection report, and complete the inspection.

2. According to claim 1, a laboratory intelligent inspection method based on multiple data sources is characterized in that: The step S3 comprises: Pre-process the collected environmental data to identify abnormal patterns in the environmental data and determine significant changes or potential hazards in the laboratory environment; Obtain historical equipment images and video data and preprocess them, and annotate the preprocessed images for abnormal conditions to obtain a data set; Based on the data set, the neural network algorithm is used to learn the normal and abnormal status of the equipment and build a recognition model; The recognition model is used to locate all potential abnormal areas in the acquired images and classify the abnormal areas.

3. The method for intelligent laboratory inspection based on multiple data sources according to claim 2 is characterized in that: The method of locating the abnormal area includes: dividing the image into a plurality of grids, generating a plurality of candidate frames in each grid, each candidate frame being provided with a confidence score; calculating the overlap between the confidence score of each candidate frame and the actual abnormality, and defining the candidate frame with a confidence score higher than a preset threshold as the target area.

4. A laboratory intelligent inspection method based on multiple data sources according to claim 2 or 3, characterized in that: The classification of abnormal areas includes: dividing the image into a number of grids, generating a plurality of candidate frames in each grid, and extracting image features from each candidate frame; inputting the extracted image features into a classification neural network, the classification neural network analyzes the features and assigns a category label to each area, and attaches a confidence score.

5. A laboratory intelligent inspection method based on multiple data sources according to claim 1, 2 or 3, characterized in that: If any laboratory personnel enter the laboratory during the inspection, record the area where the laboratory personnel are located. If the laboratory personnel enter the key inspection area, stop the inspection, obtain the environmental data of the key inspection area, monitor environmental changes, and monitor the laboratory personnel's operation process. After the laboratory personnel leave, inspect the equipment in the key inspection area, and continue to implement the inspection plan after the inspection is completed.

6. A laboratory intelligent inspection method based on multiple data sources according to claim 2 or 3, characterized in that: Merge adjacent or overlapping abnormal regions in the image into one large region.

7. A laboratory intelligent inspection method based on multiple data sources according to claim 1, 2 or 3, characterized in that: The YOLO algorithm is used to detect surface defects on the device casing to determine whether there are scratches or damage on the device appearance; the SSD algorithm is used to detect the circuits and components of the device to identify aging circuits and missing parts.

8. A laboratory intelligent inspection method based on multiple data sources according to claim 1, 2 or 3, characterized in that: The step S4 includes: generating an inspection report, feeding back the current status of the equipment and the detected abnormality information, marking the category and location of each abnormal area, and providing explanations about the abnormalities and suggested treatment measures.

9. A laboratory intelligent inspection system based on multiple data sources, using a laboratory intelligent inspection method based on multiple data sources as claimed in any one of claims 1 to 8, characterized in that: include: Multi-source inspection terminal, inspect the laboratory according to the inspection plan, obtain basic equipment information, collect laboratory environment data and equipment image data; The fault detection module pre-processes the image data uploaded by the multi-source inspection terminal, establishes a recognition model, locates potential abnormal areas in the image according to the confidence level, and classifies the abnormal areas; Database, which stores detailed records of each inspection and problem handling, generates equipment history archives, and records the frequency and trend of equipment failures; The inspection result feedback module generates inspection reports, records basic equipment information and current abnormal status, generates work orders and tracks them in real time.

10. The laboratory intelligent inspection system based on multiple data sources according to claim 9, characterized in that: The multi-source inspection terminal comprises: QR code and RFID identification module, reads equipment tags, obtains equipment information and records it in inspection data; Image and video acquisition module, which performs all-round image acquisition or video recording of the device to capture device dynamics; The thermal status detection module captures the thermal image of the device, obtains the surface temperature distribution of the device, and determines whether the device is overheated.

Images