Fault identification method and system of alarm lamp based on visual detection

Abstract

The invention discloses a fault identification method and system for an alarm lamp based on visual detection, and relates to the technical field of visual detection, and the method comprises the steps: determining a plurality of groups of fault contents of the alarm lamp according to a plurality of external abnormal light events, an internal abnormal light event and the overall form of the alarm lamp, and collecting a temperature distribution diagram of the alarm lamp; and the plurality of fault items of the alarm lamp are determined based on each group of fault contents, the temperature distribution diagram of the alarm lamp and the corresponding fault signals, so that the accuracy of the plurality of fault items of the alarm lamp is improved. Therefore, a plurality of sub-maintenance contents are determined based on the identification of the dynamic maintenance event, and a corresponding maintenance path is determined according to the plurality of sub-maintenance contents, the corresponding maintenance time and the work task list of the alarm lamp. And the autonomous maintenance system of the alarm lamp is determined based on the maintenance path, the maintenance part corresponding to each fault item of the alarm lamp and the working content of the alarm lamp, so that the accuracy of the autonomous maintenance system of the alarm lamp is improved.

Description

Technical Field

[0001] This invention relates to the technical field of visual inspection, and in particular to a fault identification method and system for alarm lights based on visual inspection. Background Technology

[0002] With the development of technology, alarm lights are Internet of Things (IoT) terminal devices that integrate multiple sensors, communication modules, and intelligent controllers. They are not limited to industrial scenarios. When in operation, alarm lights output corresponding alarm lights with different colors. In existing technologies, the light images of alarm lights are collected, and the abnormal light content is determined based on the recognition of the light images. The corresponding abnormal light event is inferred based on the abnormal light content. However, this abnormal light event is only an external abnormal light event, ignoring internal abnormal light events and the overall shape of the alarm light. This affects the accuracy of multiple fault items of the alarm light and results in low accuracy of the alarm light's autonomous maintenance system. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art. This invention provides a fault identification method and system for alarm lights based on visual detection.

[0004] This invention provides a visual detection-based fault identification method for alarm lights, comprising:

[0005] The light image is determined based on the visual detection of the alarm light by the camera. Multiple light areas are determined based on the recognition of the light image. The current light event is determined based on the area location, area shape and alarm light task list of each light area.

[0006] Based on the identification of the current lighting event, multiple external abnormal lighting events are identified, and internal abnormal lighting events are identified based on multiple working data of the alarm light during the lighting display process and the current lighting mode of the alarm light.

[0007] Based on multiple external abnormal light events, internal abnormal light events, and the overall shape of the alarm light, determine multiple sets of fault contents of the alarm light, collect the temperature distribution map of the alarm light, and determine multiple fault items of the alarm light based on each set of fault contents, the temperature distribution map of the alarm light, and the corresponding fault signals.

[0008] Based on the current work tasks of multiple fault items and alarm lights, determine the project priority of each fault item, and determine the dynamic maintenance events of the alarm lights according to the project priority of each fault item, the corresponding project content and the scene in which the alarm lights are located.

[0009] Based on the identification of the dynamic maintenance event, multiple sub-maintenance contents are determined. According to the multiple sub-maintenance contents, the corresponding maintenance time, and the work task list of the alarm light, the corresponding maintenance path is determined. Based on the maintenance path, the maintenance part of the alarm light relative to each fault item, and the work content of the alarm light, the autonomous maintenance system of the alarm light is determined.

[0010] This invention provides a visual detection-based fault identification system for alarm lights. The visual detection-based fault identification system is applied to the aforementioned visual detection-based fault identification method for alarm lights. The visual detection-based fault identification system for alarm lights includes:

[0011] The current lighting event module is used to determine the lighting image based on the visual detection of the alarm light by the camera, determine multiple lighting areas based on the recognition of the lighting image, and determine the current lighting event based on the area location, area shape and alarm light task list of each lighting area.

[0012] The internal abnormal lighting event module is used to identify multiple external abnormal lighting events based on the identification of the current lighting event, and to identify internal abnormal lighting events based on multiple working data of the alarm light during the lighting display process and the current lighting mode of the alarm light.

[0013] The fault item module is used to determine multiple fault contents of the alarm light based on multiple external abnormal light events, internal abnormal light events and the overall shape of the alarm light, collect the temperature distribution map of the alarm light, and determine multiple fault items of the alarm light based on each set of fault contents, the temperature distribution map of the alarm light and the corresponding fault signal.

[0014] The dynamic maintenance event module is used to determine the project priority of each fault project based on the current work tasks of multiple fault projects and alarm lights, and to determine the dynamic maintenance event of the alarm light according to the project priority of each fault project, the corresponding project content and the scene in which the alarm light is located.

[0015] The autonomous maintenance system module is used to identify multiple sub-maintenance contents based on the identification of the dynamic maintenance event, determine the corresponding maintenance path according to the multiple sub-maintenance contents, the corresponding maintenance time, and the alarm light's work task list, and determine the autonomous maintenance system of the alarm light based on the maintenance path, the maintenance part of the alarm light relative to each fault item, and the work content of the alarm light.

[0016] Compared with the prior art, the beneficial effects of the present invention are:

[0017] In this embodiment of the invention, the method determines a light image based on visual detection of the alarm light by a camera, identifies multiple light areas based on the recognition of the light image, and determines a current light event based on the location, shape, and task list of each light area. Multiple external abnormal light events are identified based on the recognition of this current light event, and internal abnormal light events are identified based on multiple working data of the alarm light during its light display process and its current light mode. Multiple sets of fault content for the alarm light are determined based on the multiple external abnormal light events, internal abnormal light events, and the overall shape of the alarm light. A temperature distribution map of the alarm light is collected, and multiple fault items of the alarm light are determined based on each set of fault content, the temperature distribution map, and the corresponding fault signals. This method incorporates both internal and external abnormal light events, and considers the overall fault content, temperature distribution map, and corresponding fault signals, thus improving the accuracy of multiple fault items of the alarm light.

[0018] Therefore, based on the current work tasks of multiple fault items and alarm lights, the project priority of each fault item is determined. Dynamic maintenance events for the alarm lights are then determined according to the project priority, corresponding project content, and the scene in which the alarm lights are located. Based on the identification of these dynamic maintenance events, multiple sub-maintenance contents are determined. The corresponding maintenance path is determined based on these sub-maintenance contents, their corresponding maintenance times, and the alarm light's work task list. Based on this maintenance path, the maintenance location of the alarm light relative to each fault item, and the alarm light's work content, an autonomous maintenance system for the alarm lights is established. The introduction of dynamic maintenance events further controls the maintenance path, achieving a holistic consideration of the maintenance path, the maintenance location of the alarm light relative to each fault item, and the alarm light's work content, thus improving the accuracy of the autonomous maintenance system for the alarm lights. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the fault identification method for alarm lights based on visual detection in an embodiment of the present invention.

[0020] Figure 2 This is a flowchart illustrating step S11 of the visual detection-based alarm light fault identification method in this embodiment of the invention.

[0021] Figure 3 This is a flowchart illustrating step S12 in the visual detection-based alarm light fault identification method in this embodiment of the invention.

[0022] Figure 4 This is a flowchart illustrating step S13 in the visual detection-based alarm light fault identification method in this embodiment of the invention.

[0023] Figure 5This is a flowchart illustrating step S14 of the visual detection-based alarm light fault identification method in this embodiment of the invention.

[0024] Figure 6 This is a flowchart illustrating step S15 of the visual detection-based alarm light fault identification method in this embodiment of the invention.

[0025] Figure 7 This is a schematic diagram of the structural composition of a fault identification system for alarm lights based on visual detection in an embodiment of the present invention. Detailed Implementation

[0026] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0027] Please see Figures 1 to 7 A fault identification method for alarm lights based on vision detection is proposed and applied to vision detection scenarios. The fault identification method for alarm lights based on vision detection includes:

[0028] Step S11: Determine the light image based on the visual detection of the alarm light by the camera, determine multiple light areas based on the recognition of the light image, and determine the current light event based on the area location, area shape and alarm light task list of each light area;

[0029] Step S12: Based on the identification of the current lighting event, determine multiple external abnormal lighting events, and based on multiple working data of the alarm light during the lighting display process and the current lighting mode of the alarm light, determine internal abnormal lighting events;

[0030] Step S13: Determine multiple sets of fault contents of the alarm light based on multiple external abnormal light events, internal abnormal light events and the overall shape of the alarm light, collect the temperature distribution map of the alarm light, and determine multiple fault items of the alarm light based on each set of fault contents, the temperature distribution map of the alarm light and the corresponding fault signal.

[0031] Step S14: Determine the project priority of each fault project based on the current work tasks of multiple fault projects and alarm lights, and determine the dynamic maintenance event of the alarm light according to the project priority of each fault project, the corresponding project content and the scene where the alarm light is located.

[0032] Step S15: Based on the identification of the dynamic maintenance event, determine multiple sub-maintenance contents, determine the corresponding maintenance path according to the multiple sub-maintenance contents, the corresponding maintenance time, and the alarm light's work task list, and determine the autonomous maintenance system of the alarm light based on the maintenance path, the maintenance part of the alarm light relative to each fault item, and the work content of the alarm light.

[0033] refer to Figure 2In step S11, the specific steps are as follows:

[0034] S111: Collect the current position of the alarm light, determine the corresponding multiple cameras based on the surrounding detection of the current position of the alarm light, the multiple cameras are distributed around the alarm light, and dynamically capture the alarm light to collect the light image of the alarm light.

[0035] S112: Based on the recognition of the alarm light's light image, determine multiple light body features and mark the spatial position of each light body feature. Based on the spatial position of each light body feature and the image content of the light image, determine multiple light areas. At the same time, collect the alarm light's task list and determine the current light event based on the area position, area shape, and alarm light's task list of each light area.

[0036] In the embodiments of this application, the current position of the alarm light is acquired, and multiple cameras are determined based on the surrounding detection of the current position of the alarm light. The multiple cameras are distributed around the alarm light and dynamically capture images of the alarm light to acquire light images of the alarm light. This approach takes into account the overall consideration of the surrounding detection of the current position of the alarm light and ensures the accuracy of the multiple cameras.

[0037] At this point, the system will obtain the precise coordinates of the alarm light in three-dimensional space through a digital twin or device management database. Centered on these coordinates, the system will query the entire camera network, analyze which cameras' fields of view can cover the location, and exclude devices blocked by obstacles. Finally, it will select 2-4 optimal cameras to form a multi-angle observation layout.

[0038] When the alarm light is activated or enters the inspection cycle, the system sends a synchronous shooting command to these selected cameras. To ensure the accuracy of subsequent analysis, all cameras must achieve high-precision time synchronization through protocols such as PTP to ensure that the captured image frames have a unified timestamp. At the same time, the cameras can also dynamically adjust parameters such as exposure and gain according to real-time lighting conditions, such as automatically switching to HDR mode at night to prevent overexposure of lights or loss of details.

[0039] After receiving the instruction, the camera continuously captures a video stream at a set frame rate and transmits it in real time to the edge computing node via industrial Ethernet or 5G network. The system extracts key frames from the video stream for analysis as needed. Once the image reaches the processing node, a series of preprocessing operations are immediately performed. The first is distortion correction, which eliminates barrel or pincushion distortion caused by wide-angle lenses and restores the true geometry of the object. The next step is image enhancement, which uses adaptive filtering algorithms to suppress noise and applies algorithms such as CLAHE to enhance the contrast between the lamp body outline and the background, thereby making key features more prominent and facilitating subsequent feature extraction.

[0040] Specifically, the alarm light is installed on the key equipment of production line 3, and has three working states: solid green, flashing yellow, and strobe red. When the system needs to identify a fault, it executes S111-1. The system retrieves the precise three-dimensional coordinates of the alarm light from the equipment management database and immediately matches it with the camera network. After calculation, the system selects three optimal cameras: Cam-01 (located directly in front of the equipment), Cam-07 (located on the overhead crane in the upper left), and Cam-12 (located on the control cabinet in the lower right), all of which can cover the target without obstruction. Due to the current dim lighting in the workshop, the system, while sending the synchronous shooting command, also commands all cameras to turn on night mode and increase exposure gain to ensure image quality.

[0041] The three cameras begin capturing video streams at a frame rate of 15fps and transmitting them in real time to the edge computing node. Upon receiving the three video streams, the edge node immediately preprocesses each frame. For example, the system performs distortion correction on the image from Cam-07, eliminating the slight barrel distortion caused by its wide-angle lens. The system also performs CLAHE enhancement on the image, making the outline of the red alarm light cover and the internal LED array clearly visible even in low-light environments. At the precise time point T = 10:30:01.500, the system obtains three high-definition images from different perspectives, which have been preprocessed and have perfectly aligned timestamps. These three images together constitute the final output of step S111, laying a solid data foundation for the next steps of light feature recognition and light status analysis.

[0042] Furthermore, multiple light features are identified based on the recognition of the alarm light's light image, and the spatial position of each light feature is marked. Multiple light areas are determined based on the spatial position of each light feature and the image content of the light image. At the same time, the alarm light's task list is collected. The current light event is determined based on the area position, area shape, and alarm light task list of each light area. This comprehensive consideration of the area position, area shape, and alarm light task list of each light area ensures the accuracy of the current light event.

[0043] At this point, the system defines "lamp body features," which are physical elements with stable geometry that are not easily affected by lighting and status, such as the outline of the lampshade, heat dissipation grilles, or mounting bolts. Using a deep learning-based instance segmentation model or a traditional feature matching algorithm, the system accurately segments the pixel regions of these features from the image. After identifying the features, the system performs spatial location marking, not only generating bounding boxes and pixel coordinates in the image, but also using the synchronized images from multiple cameras in step S111 to perform triangulation through a stereo vision algorithm to calculate the three-dimensional spatial position of these features in the physical world coordinate system.

[0044] Based on these characteristics and the actual light emission situation, the system dynamically divides the "light areas" that are actually emitting light or should be emitting light. Region generation can be based on morphology, such as using the identified lampshade outline as a geometric constraint and dividing it into multiple potential functional areas according to preset model knowledge. At the same time, it can also be based on image content, extracting actual emitting pixels through color space conversion and threshold segmentation, and aggregating spatially adjacent and color-similar pixels into specific light areas. This method can effectively capture unexpected light emission, such as light leakage or local abnormal bright spots. The system will determine the attributes of each region, including its location and shape (such as area, perimeter, roundness, etc.), which are key indicators for judging whether the light is normal.

[0045] The system obtains the current task list of the device to which the alarm light belongs in real time through an industrial bus or database interface. This list clearly indicates the expected light state for the current task. The system compares the attributes of multiple light areas (such as position, shape, color, and flashing frequency) with this "expected state". Through a rule-based inference engine or classification model, the system can make an accurate judgment. If the detected light completely matches the expected state, it is judged as an "normal operation" event. If there is a mismatch, a specific abnormal event will be generated, such as "unexpected emergency shutdown" or "warning light malfunction", thus transforming a simple visual anomaly into a fault event with clear business implications.

[0046] Specifically, the current task of the key equipment on production line 3 is "normal operation," and the alarm light is expected to be constantly green. When the system executes S112-1, it inputs pre-processed images from Cam-01, Cam-07, and Cam-12. The instance segmentation model successfully identifies three key features of the alarm light: the outline of the red lampshade at the top, the outline of the yellow lampshade in the middle, and the outline of the green lampshade at the bottom. Through a stereo vision algorithm, the system accurately calculates the three-dimensional coordinates of the center of the bottom green lampshade outline and marks it as Feature_Green_Base.

[0047] The system used the outline of Feature_Green_Base as a geometric constraint to delineate a candidate green light region in the image. At the same time, the system performed HSV color space analysis on the image and found that there were only a few green pixels below the brightness threshold in this region, and most of the area was dark. The system finally determined a light region named Light_Zone_1, whose attributes were recorded as follows: the position corresponds to the bottom lampshade, but the shape area is only 15% of the expected area, the roundness is low, and the brightness is weak.

[0048] The system obtains the current task list from the device's PLC and clearly defines the expected state as "constantly green". The system then compares the expected state with the actual detected Light_Zone_1 attribute. The expected state requires the green area to be full and the brightness to be stable, but the actual state is incomplete and the brightness is weak. The two are seriously inconsistent. Therefore, the system finally determines the current light event as "green light group failure (manifested as severely insufficient brightness)". This specific event description provides accurate input for subsequent fault diagnosis and maintenance.

[0049] refer to Figure 3 In step S12, the specific steps are as follows:

[0050] S121: Based on the identification of the current light event, determine multiple external abnormal features, and determine the corresponding external abnormal light event based on the characteristic location, corresponding feature shape and working history of the alarm light, so as to collect multiple external abnormal light events.

[0051] S122: Real-time monitoring of the operation of the alarm lights, determining multiple operational data of the alarm lights during the lighting display process based on the alarm light's operational database, marking the current lighting mode of the alarm lights, and determining internal abnormal lighting events based on the multiple operational data of the alarm lights during the lighting display process, the corresponding fault signals, and the current lighting mode of the alarm lights.

[0052] In the embodiments of this application, multiple external abnormal features are determined based on the identification of the current light event. The corresponding external abnormal light event is determined based on the characteristic location, corresponding feature shape, and working history of the alarm light of the multiple external abnormal features. This method collects multiple external abnormal light events, taking into account the overall consideration of the characteristic location, corresponding feature shape, and working history of the alarm light of the multiple external abnormal features, thus ensuring the accuracy of the corresponding external abnormal light events.

[0053] At this point, the system employs specialized recognition algorithms for different types of anomalies. For surface attachments such as stains or coatings, a deep learning-based anomaly detection model can be used, such as an autoencoder trained on an image of a "clean" alarm light. When an image with stains is input, the model will produce a high reconstruction error when reconstructing the area, thus locating the anomaly. For physical damage such as cracks or breaks, traditional computer vision algorithms are mainly relied upon. Linear cracks are identified through Canny edge detection and Hough transform, or irregular breaks are detected by analyzing contour curvature. For dynamic occlusions, optical flow or background subtraction algorithms are used in conjunction with target detection models to identify and determine their state. After identifying these features, the system quantifies their attributes, including their precise location on the image and 3D model, as well as geometric and color attributes such as area, perimeter, and color distribution.

[0054] The system accesses the alarm light's "working history" database, a time-series knowledge base containing historical maintenance records, environmental condition data, and past fault types. The system then performs correlation analysis between currently identified external anomalies and this historical data to identify potential causal relationships. The system uses a rule-based inference engine to generate events. For example, if a crack is identified in a light-transmitting area, the system generates an event titled "Light scattering causes blurred display." If a large area of ​​opaque obstruction is found, it determines "Light is largely blocked, causing signal failure." The system generates a structured "External Anomaly Light Event" data object, containing detailed information such as event ID, type, root cause description, impact assessment, and confidence level, transforming a visual phenomenon into an actionable maintenance guideline.

[0055] Specifically, in the previous steps, the system has determined that the current lighting event of the alarm light is "abnormal red flashing light pattern (manifested as unfocused light with scattering)". Now, it proceeds to step S121 to diagnose the external cause. When executing S121-1, the system focuses on a high-definition image of the red lampshade area. The anomaly detection model detects a patch with high reconstruction error in the upper right corner of the red lampshade. At the same time, the edge detection operator extracts irregular fine lines inside the patch, but these do not meet the typical characteristics of a crack. The system finally identifies an external anomaly: it is an irregular sheet-like object located in the upper right corner of the red lampshade, with an area of ​​about 0.8 cm2, a yellowish color, and a semi-transparent appearance. The internal texture is "spider web".

[0056] The system queried the alarm light's operational history database and found a key record: "Two days ago, welding repair work was carried out on the pipe 5 meters above the alarm light." The system's inference engine immediately started and matched a rule: if the characteristic shape is "semi-transparent sheet-like" and the internal texture is "spider web-like," and the operational history includes "welding work," then the root cause is "welding spatter adhesion." The system further evaluated the impact: when the red LED flashes at high power, the light passes through this uneven semi-transparent spatter, which will be refracted and scattered, which is completely consistent with the initially observed "light non-concentration" phenomenon.

[0057] The system generated and collected an external abnormal lighting event, the root cause of which was precisely described as "welding spatter adhering to the surface of the red lampshade". The impact assessment was "the spatter caused the red flash line to scatter, affecting the clarity and effective distance of the signal", with a confidence level of 92%. This specific event description provided precise guidance for subsequent maintenance decisions: maintenance personnel need to prepare scrapers and cleaning agents to go to the site to remove the welding spatter, instead of blindly inspecting the electrical wiring.

[0058] Furthermore, the system monitors the operation of alarm lights in real time, determines multiple operational data points of the alarm lights during the lighting display process based on the alarm light's operational database, and marks the current lighting mode of the alarm lights. Based on these multiple operational data points, corresponding fault signals, and the current lighting mode of the alarm lights, internal abnormal lighting events are determined. This comprehensive approach, which considers multiple operational data points, corresponding fault signals, and the current lighting mode of the alarm lights during the lighting display process, ensures the accuracy of identifying internal abnormal lighting events.

[0059] At this time, the system will poll the alarm light controller at high frequency through the industrial fieldbus or debugging interface, and store key operating data such as input voltage, drive current of each color channel, PWM control signal duty cycle, chip temperature, and communication status in a time-series database in real time. At the same time, the system will not only record the instructions issued by the host computer, but also independently verify and mark the "current lighting mode" by analyzing these control signals. For example, by detecting the duty cycle and frequency of the red channel PWM signal, the system can automatically mark the current mode as "red strobe (2Hz)". The system maintains a "lighting mode library" containing standard parameter ranges to provide a benchmark for subsequent comparison and analysis.

[0060] The system will perform data consistency verification, comparing the collected real-time working data with the standard parameters defined in the "mode library" for the current lighting mode item by item. During this process, the system will use a variety of anomaly detection algorithms, including threshold detection for key parameters such as temperature and current, trend analysis for slowly changing parameters, and checking the logical relationship between control signals and execution results.

[0061] In addition, the system will integrate fault codes (DTCs) actively reported by the alarm light controller, combining this high-confidence evidence with the inference results of data anomalies. The system uses an expert system or decision tree model to perform causal reasoning based on all information to generate specific internal abnormal events. For example, if the PWM signal is normal but the drive current is zero and a fault code "LEDOpenLoad" is received, the system will generate an event of "green LED string open circuit fault". The final output event will contain structured information such as fault module, fault type, severity level and confidence level.

[0062] Specifically, the system has already determined in previous steps that the current lighting event of the alarm light is "yellow warning light not responding," while the current task requires it to display yellow flashing. Now, it proceeds to step S122 to diagnose the internal cause. When executing S122-1, the system monitors the alarm light's working database in real time via Modbus TCP / IP and collects a set of key data: the input voltage is normal, and the PWM duty cycle of the yellow channel is switching between 50% and 0% at a frequency of 1Hz, which perfectly matches the "yellow flashing" instruction. However, contradictorily, the drive current of the yellow channel is constant at 0.05A, far below the normal operating range of 0.8A-1.2A. Based on this, the system marks the current lighting mode as "yellow flashing (1Hz)."

[0063] The system performed a data consistency check, comparing the actual data with the standard parameters of the "yellow flashing" mode in the mode library. It immediately discovered a serious logical inconsistency between "normal PWM signal" and "abnormally low drive current". At the same time, the system received a diagnostic message actively reported by the alarm light controller: DTC is "YellowLEDChannelOpenLoad" (yellow LED channel open circuit).

[0064] The system's decision tree model matched a high-confidence rule: when logic inconsistency (PWM normal, current low) and DTC "OpenLoad" occur simultaneously, it can be confirmed as a hardware open-circuit fault. The system generates an internal abnormal lighting event, and the fault module is precisely located as "LED driver board - yellow channel", the fault type is "load open circuit", the root cause is described as "the yellow LED string or its connecting line has an open circuit, causing the driver to be unable to load current normally", the severity level is "high", and the confidence level is as high as 99%. This internal event is in stark contrast to the external event generated by S121 (such as "the lamp cover is completely blocked"), providing maintenance personnel with completely different repair paths: the former requires replacing the LED light board or checking the wiring, while the latter only requires removing the obstruction.

[0065] refer to Figure 4 In step S13, the specific steps are as follows:

[0066] S131: Determine the overall shape of the alarm light based on the detection of the alarm light, and determine the first sub-fault content based on multiple external abnormal light events and the overall shape of the alarm light.

[0067] S132: Determine the second sub-fault content based on the internal abnormal light events and the overall shape of the alarm light; determine multiple sets of fault content for the alarm light based on the matching of the first sub-fault content, the second sub-fault content and the working history of the alarm light.

[0068] S133: Based on the temperature detection of the alarm light, determine multiple temperature data and corresponding temperature detection locations; determine the temperature distribution map of the alarm light based on the multiple temperature data, corresponding temperature detection locations, and the overall shape of the alarm light; determine the first fault information based on each group of fault content and the temperature distribution map of the alarm light; determine the second fault information based on each group of fault content and corresponding fault signals; and determine multiple fault items of the alarm light based on the multiple interactions of the first fault information and the second fault information.

[0069] In the embodiments of this application, the overall shape of the alarm light is determined based on the detection of the alarm light, and the first sub-fault content is determined according to multiple external abnormal light events and the overall shape of the alarm light. This takes into account the overall consideration of multiple external abnormal light events and the overall shape of the alarm light, ensuring the accuracy of the first sub-fault content.

[0070] At this point, the system will utilize the image sequences synchronously acquired from multiple cameras in step S111, through multi-view... Figure 3 The 3D reconstruction technology creates a detailed digital model for the alarm light. This process begins by using algorithms such as SIFT or ORB to extract and match thousands of key points from images from different perspectives, and then generating a sparse 3D point cloud using the principle of triangulation. A multi-view stereo matching algorithm is then used to generate a dense point cloud, and finally, algorithms such as Poisson surface reconstruction are used to transform it into a continuous triangular mesh model. This model is the "overall shape" of the alarm light. The system performs morphological parameterization analysis, registers and compares the reconstructed mesh model with the ideal CAD model of the alarm light model, and uses the distance between the two to quantitatively assess whether there is macroscopic deformation. At the same time, it detects potential cracks or damaged edges by analyzing the continuity of the mesh normal vectors.

[0071] The system performs spatial-semantic association, accurately mapping each external anomaly feature identified in S121 onto the 3D mesh model through the camera projection matrix, thereby obtaining its precise location and range in 3D space; the system analyzes the context of the feature in the overall shape.

[0072] The system has a built-in expert knowledge base containing a large number of "feature-morphology-fault" inference rules. For example, if the external feature is "attachment" and the overall morphology shows that the surface of the area is continuous and undamaged, it is inferred to be "functional damage". If the external feature is "crack" and the morphological analysis confirms that it penetrates the mesh surface, it is inferred to be "structural damage". The inferred "first-level fault content" is encapsulated into a structured data object, which includes the fault nature, description, impact assessment and associated 3D model area, thereby enabling accurate diagnosis of physical damage.

[0073] Specifically, in step S121, the system identified two external anomalous features of the alarm light: ① welding spatter on the upper right corner of the red lampshade; ② a thin crack on the surface of the yellow lampshade. Now, the system proceeds to step S131 for physical attribution. When executing S131-1, the system uses image sequences from multiple cameras to successfully reconstruct a high-precision 3D mesh model of the alarm light using a multi-view stereo matching algorithm. After comparing the model with the ideal CAD model, the system found no obvious macroscopic depressions or bends, and the overall geometric integrity was good. However, during surface continuity analysis, the system detected a 15mm long abrupt change in the normal vector boundary in the yellow lampshade area, and its spatial location perfectly matched the "crack" feature identified in S121.

[0074] The system begins processing these two external anomalies. For feature ①, "welding spatter," the system maps it onto the 3D model and finds that it is located in the upper right corner of the red lampshade. The mesh surface in this area is continuous, with smooth curvature and no damage. Based on this, the inference engine matches a rule, infers it as "functional damage," and generates the first sub-fault content 1: "Welding spatter adheres to the surface of the red lampshade, causing a local light transmittance decrease of about 70%, affecting the clarity and effective range of the red signal." For feature ②, "crack," the system maps it onto the 3D model and finds that it completely coincides with the normal vector mutation boundary detected by morphological analysis, and confirms that the crack penetrates the mesh surface of the lampshade. The inference engine matches another rule, infers it as "structural damage," and generates the first sub-fault content 2: "The yellow lampshade has a penetrating crack, which damages its IP protection level and is prone to short circuits or corrosion in humid or dusty environments." The system ultimately obtained two clear "first-order sub-fault contents," which accurately diagnosed the external physical state of the alarm light from both functional and structural dimensions, laying the foundation for the next step of fusion analysis.

[0075] Furthermore, the second sub-fault content is determined based on the internal abnormal lighting events and the overall shape of the alarm lights. Multiple sets of fault content for the alarm lights are determined based on the matching of the first sub-fault content, the second sub-fault content, and the working history of the alarm lights. This approach takes into account the overall consideration of matching the first sub-fault content, the second sub-fault content, and the working history of the alarm lights, ensuring the accuracy of the multiple sets of fault content for the alarm lights.

[0076] At this point, the system maps the internal fault module (such as the "yellow LED driving channel") onto the generated 3D mesh model to determine its physical location. The system checks whether there are any abnormalities in the "overall morphology" of the area corresponding to the internal fault module. If the morphology is intact, the internal fault is attributed to the natural aging of the components or random electronic failure. If the morphology shows impact marks or cracks, then the internal fault is directly caused by these external physical damages. The system uses the reasoning rules in the expert knowledge base to generate "secondary sub-fault content" based on the results of the correlation analysis, clearly inferring whether the root cause of the fault is physical damage or electronic degradation, and outputs it in a structured manner.

[0077] The system will combine all the "first sub-fault contents" output by S131 with the "second sub-fault contents" that have been output to form all fault combinations; the system will access the "working history" time-series database of the alarm light to extract key information such as equipment running time, historical maintenance records and environmental event logs.

[0078] Using a weighted scoring model, the system evaluates the plausibility of each fault combination. For example, a combination of "the lampshade is intact but the LED is open" will receive a high score if historical data shows that the equipment has exceeded its service life. Similarly, a combination of "the lampshade has an impact crack and the internal circuit is short-circuited" will also receive a high score if there are forklift operation records nearby, because it constructs a complete causal chain. The system will output one or more "multiple sets of fault content" sorted by plausibility score. Each set is a complete fault hypothesis containing internal and external information and supported by historical data, providing multiple alternative solutions for the final diagnosis.

[0079] Specifically, the system has obtained two first-level sub-fault contents through S131: ① welding spatter adhering to the red lampshade (functional damage); ② a through-crack in the yellow lampshade (structural damage); simultaneously, an internal abnormal lighting event is obtained through S122: the yellow LED string is open-circuited; now, the system proceeds to step S132 for fusion diagnosis; the system will map the internal abnormal event "yellow LED string open-circuit" onto the 3D model, and its position corresponds exactly to the yellow lampshade area; the system checks the "overall morphology" of this area and finds that a "through-crack" has been clearly identified; based on this, the inference engine judges that it is unlikely that a perfectly good lampshade would have an internal LED open-circuit and then happen to develop an external crack, and that the external crack would lead to internal water ingress, corrosion, or stress damage, thus causing an open circuit, which is a more reasonable causal chain; therefore, the system generates a second-level sub-fault content: the fault module is "yellow LED module", the fault type is "open circuit", and the inferred root cause is "internal electrical function failure caused by external structural damage (crack)".

[0080] The system combines two first-level elements with one second-level element to form several preliminary fault combinations. The system then queries the alarm light's operational history and finds a key record: "Two days ago, welding repair work was carried out on the pipe 5 meters above, during which sparks flew." The system begins to evaluate the plausibility of each combination. The combination containing "cracked yellow lampshade" and "open circuit in yellow LED" highly matches the external event of "welding spatter." High-temperature metal spatter impacting the lampshade could easily cause both cracks and internal damage simultaneously; therefore, this combination receives the highest plausibility score. While "spatter on red lampshade" also matches the welding operation record, it has no direct causal relationship with the yellow light malfunction and is judged as a concurrent, independent problem.

[0081] The system outputs a set of fault information sorted by priority: the primary fault group (high confidence) is "the yellow lampshade suffered a structural crack due to the impact of welding spatter, which in turn caused the internal yellow LED module to open circuit and fail"; the secondary fault group (concurrent problem) is "the surface of the red lampshade was simultaneously contaminated by welding spatter, resulting in a decrease in its light transmittance". This output is no longer a collection of scattered information points, but a fault story with primary and secondary points and a cause-and-effect relationship, which greatly improves the depth and accuracy of diagnosis.

[0082] Therefore, multiple temperature data points and corresponding temperature detection locations are determined based on the temperature detection of the alarm light. A temperature distribution map of the alarm light is then determined based on these multiple temperature data points, the corresponding temperature detection locations, and the overall shape of the alarm light. First fault information is determined based on each set of fault content and the temperature distribution map of the alarm light. Second fault information is determined based on each set of fault content and the corresponding fault signals. Multiple fault items of the alarm light are determined based on the multiple interactions between the first and second fault information. This comprehensive approach considers the multiple interactions between the first and second fault information, ensuring the accuracy of the multiple fault items of the alarm light. Furthermore, internal and external abnormal light events are introduced, further improving the accuracy of the multiple fault items of the alarm light by incorporating comprehensive considerations of each set of fault content, the temperature distribution map of the alarm light, and the corresponding fault signals.

[0083] At this time, the system will simultaneously collect multi-source temperature data, including the surface two-dimensional temperature matrix obtained by a high-resolution infrared thermal imager, and the internal sensor data (such as MCU and driver chip temperature) read through the industrial bus; the system uses spatial registration technology to align the two-dimensional view of the infrared thermal image with the generated three-dimensional mesh model, and establish a mapping relationship between pixel coordinates and three-dimensional model coordinates.

[0084] The system maps the temperature data from the infrared thermal image as a "texture" onto the surface of the 3D mesh model and attaches the internal temperature data to the corresponding virtual components in the model. The system generates a complete and visualized "temperature distribution map," which is a 3D model. Users can observe its surface temperature from any angle and query the real-time temperature of key internal components.

[0085] The system analyzes the temperature distribution map and compares it with the built-in standard thermodynamic model to identify temperature anomalies. For example, if a region has an "overheated hot spot" that far exceeds the threshold, it points to a short-circuit fault, while a region that should be emitting light has a "low temperature spot" that is no different from the ambient temperature, it strongly supports an "open circuit" fault. These observations are generated as "first fault information". At the same time, the system directly extracts the diagnostic fault codes (DTCs) and communication status information actively reported by the controller and structures them as "second fault information". These two types of information, one from indirect physical phenomenon inference and the other from direct electronic self-test reports, together form the basis of cross-validation.

[0086] The system employs fusion algorithms such as Bayesian inference or DS evidence theory to interactively verify the first and second fault information. If both strongly point to the same conclusion (e.g., the low temperature point is consistent with the open circuit DTC), the confidence level of the fault content will be greatly increased. If the two contradict each other, the system will mark it as a "complex composite fault" and generate a new hypothesis. If the evidence is singular, the confidence level will be reduced and it will be marked as "requires further confirmation".

[0087] After verification, all fault information is sorted according to the final confidence level. The one with the highest confidence level is determined as the final "fault item". This is a maintenance-oriented, highly specific set of instructions, which includes fault location, type, root cause, impact assessment, confidence level, and specific maintenance recommendations to directly guide subsequent maintenance work.

[0088] Specifically, the system has obtained a highly plausible fault description through S132: {The yellow lampshade suffered a structural crack due to the impact of welding spatter, which in turn caused an open circuit failure in the internal yellow LED module}; the system will control the infrared thermal imager to collect a thermal image of the alarm light, and at the same time read the internal sensor data; after data fusion, the system generates a temperature distribution map of the alarm light; the map clearly shows that the temperature of the red and green LED areas is normal, while the temperature of the yellow LED area is almost the same as the ambient temperature, and its driver chip temperature is also abnormally low.

[0089] The system analyzes the temperature distribution map and finds a "low temperature point" phenomenon in the yellow LED area, which is completely consistent with the physical principle that "an open circuit in an LED results in no power consumption and no heat generation". Therefore, the system generates the first fault information: "abnormal temperature distribution (low temperature), strongly supporting yellow LED open circuit fault". The system extracts the DTCB1004-YellowLEDChannelOpenLoad recorded in S122 from the communication bus and generates the second fault information: "controller self-test report open circuit fault, directly confirming electrical open circuit in the yellow channel". The first and second fault information corroborate each other, and the confidence level of the fault content is calculated by the system to be 99.8%, thus being identified as the final fault item. The system ultimately outputs a structured fault item: its fault location is precisely located as "alarm light, yellow LED module and lamp cover", the fault type is "compound fault - structural damage and electrical function failure", the root cause is described as "welding spatter impact caused structural cracks in the lamp cover, and moisture intrusion caused corrosion and open circuits in the internal circuit of the yellow LED module", and specific maintenance recommendations are given: "Immediately replace the yellow LED module and use epoxy resin to repair the lamp cover cracks to restore the IP protection level". This output is no longer a hypothesis, but a maintenance instruction that has been verified multiple times and can be directly executed.

[0090] refer to Figure 5 In step S14, the specific steps are as follows:

[0091] S141: Collect multiple fault items, determine the corresponding item content based on the identification of multiple fault items, and determine the item priority of each fault item according to the item content of multiple fault items, the corresponding work impact level and the current work task of the alarm light.

[0092] S142: Based on the current location of the alarm light, multiple environmental features are determined by environmental detection, and the scene where the alarm light is located is determined according to the feature areas of the multiple environmental features, the corresponding feature shapes, and the overall shape of the alarm light.

[0093] S143: Determine the first dynamic maintenance content based on the scene where the alarm light is located and the project priority of each fault item; determine the second dynamic maintenance content based on the scene where the alarm light is located and the project content of each fault item; and determine the dynamic maintenance event of the alarm light based on the first and second dynamic maintenance content.

[0094] In the embodiments of this application, multiple fault items are collected, and the corresponding item content is determined based on the identification of multiple fault items. The item priority of each fault item is determined according to the item content of multiple fault items, the corresponding work impact level and the current work task of the alarm light. This approach takes into account the overall consideration of the item content of multiple fault items, the corresponding work impact level and the current work task of the alarm light, thus ensuring the accuracy of the item priority of each fault item.

[0095] At this point, the system will obtain one or more "fault items" object lists from the output of step S133, and traverse each object to extract key fields such as fault location, fault type, functional impact, root cause, and confidence level. More importantly, the system will perform semantic understanding of the content and, through the built-in mapping table, convert technical "functional impact" (such as "complete loss of early warning function") into "business impact" closely related to business operations. This semantic conversion is the basis for subsequent risk assessment and priority calculation.

[0096] The system automatically assigns a basic work impact level to each fault item based on its functional impact, such as "catastrophic," "severe," "moderate," or "minor." The system incorporates the context of the current work task for weighting. It also acquires the current task status of the equipment associated with the alarm light in real time (e.g., "normal operation," "shutdown for maintenance," or "high-risk chemical reaction") and assigns a weighting factor to each status. This factor amplifies or reduces the basic impact level; for example, in the "high-risk chemical reaction" task, the urgency of a "severe" fault is significantly amplified. The system uses a comprehensive scoring model that combines the basic impact score, context weight, and fault confidence to calculate the final priority score. All fault items are then sorted in descending order based on this score, resulting in a clear maintenance priority list that guides resource allocation.

[0097] Specifically, the system generated two fault items for the alarm light: Fault item 1 (FP-A-001) is an open circuit in the yellow LED module, resulting in the loss of the warning function; Fault item 2 (FP-A-002) is welding spatter adhering to the red lamp cover, causing a 70% decrease in the brightness of the red signal. Meanwhile, the current task of production line 3, where the alarm light is located, is "mixing and reacting highly hazardous chemicals." When executing S141-1, the system analyzes these two items separately. For FP-A-001, the system extracts its functional impact as "complete loss of warning function," with a confidence level of 99.8%; for FP-A-002, the system extracts its functional impact as "a 70% decrease in the brightness of the red signal," with a confidence level of 95%.

[0098] The system performs a work impact level assessment: "Complete loss of early warning function" is automatically rated as "Severe" (quantitative score of 3), while "Decrease in red signal brightness" is rated as "Moderate" (quantitative score of 2); the system obtains that the current task is "mixed reaction of high-risk chemicals", and queries the weight table to obtain a context weight factor of 1.5; the system calculates the final priority score; for FP-A-001, its score is (31.5)^0.998 = 4.491; for FP-A-002, its score is (21.5)^0.95 = 2.85; by comparison, the system determines that 4.491 is greater than 2.85; therefore, step S141 finally outputs an ordered list of fault items: priority 1 is fault item FP-A-001 (loss of early warning function), priority 2 is fault item FP-A-002 (decrease in red signal brightness), this list will directly guide the maintenance system to dispatch work orders to handle the fault of the early warning function, because it has the highest risk under the current high-risk task.

[0099] Furthermore, multiple environmental features are determined based on environmental detection of the current location of the alarm light. The scene where the alarm light is located is determined according to the feature areas, corresponding feature shapes, and overall shape of the alarm light. This comprehensive consideration of the feature areas, corresponding feature shapes, and overall shape of the alarm light ensures the accuracy of the scene where the alarm light is located.

[0100] At this point, the system integrates data sources from both visual and non-visual perspectives. Visual data comes from nearby surveillance cameras or inspection robots, while non-visual data is obtained by subscribing to real-time data streams from environmental sensors via an IoT platform, such as temperature, humidity, gas concentration, and noise. The system performs feature extraction and classification. For both static and dynamic objects, the system uses a target detection model trained on industrial scenarios (such as YOLOv8) to identify key objects such as "personnel," "forklifts," and "toolboxes" in real time within the video stream. For environmental state parameters, the system directly parses sensor data packets to extract specific values. Simultaneously, through techniques such as optical flow, the system can also determine whether there is dynamic activity or special ground conditions in a specific area, such as "oil stains on the ground."

[0101] The system performs spatial and morphological correlation of features, constructing a spatial topology map of all identified environmental features in a unified world coordinate system to clarify their relative positional relationships. Simultaneously, the system correlates these environmental features with the overall shape of the alarm light to uncover deeper connections. Most importantly, the system performs fuzzy matching between the real-time extracted feature set and a built-in "scene library," which contains templates for various typical industrial scenarios, such as "normal production," "maintenance work," or "high-risk work." The system calculates the matching degree between the current environment and each scene template through probabilistic reasoning, ultimately determining the scene with the highest matching degree as the "scene where the alarm light is located" and outputting its confidence level.

[0102] Specifically, the alarm light is located next to a critical piece of equipment on production line 3. The system needs to determine its location to provide a basis for subsequent maintenance decisions. The system will activate the analysis of video streams from nearby cameras. The target detection model detects in consecutive frames: a forklift is reversing about 3 meters to the left of the alarm light; there is an open toolbox on the ground directly below the alarm light; and a person in maintenance clothing is taking tools from the toolbox. At the same time, the system reads from the IoT platform that the ambient temperature and humidity are normal and there are no harmful gas alarms. The area status analysis also shows that the activities of the forklift and the person are concentrated in a small area around the alarm light.

[0103] The system constructs a spatial relationship diagram in a unified coordinate system, clearly showing the relative positions of the alarm light, toolbox, maintenance personnel, and forklift. The system matches this feature set with a scenario library. It has a low match with the "normal production" template because of the presence of maintenance personnel and toolbox. It also has a low match with the "high-risk operation" template because there is no high temperature or gas alarm. However, it has a very high match with the "maintenance operation" template because the key features required by this template, such as "open toolbox," "personnel," and "area activity," are basically met in the current environment. The presence of the forklift is reasonably interpreted as "auxiliary activity during maintenance operation." After probability calculation, the system determines that the scenario where the alarm light is located is a "planned maintenance operation scenario" with a confidence level of 95%. This scenario identification result is crucial because it tells the maintenance decision system that dealing with the alarm light malfunction now is not an emergency intrusion, but a planned collaborative operation that can be integrated into the existing maintenance work. Safety strategies and resource allocation will revolve around this scenario.

[0104] Therefore, the first dynamic maintenance content is determined based on the scene where the alarm light is located and the project priority of each fault item. The second dynamic maintenance content is determined based on the scene where the alarm light is located and the project content of each fault item. The dynamic maintenance event of the alarm light is determined based on the first and second dynamic maintenance content, which takes into account the overall consideration of the first and second dynamic maintenance content and ensures the accuracy of the dynamic maintenance event of the alarm light.

[0105] At this point, the system receives the priority-sorted list of fault items output by S141 and uses a threshold decision model. If the highest priority is "critical" or "catastrophic," the system triggers the "immediate response" mode and marks the maintenance content as "urgent." If the priority is medium or low, the "planned maintenance" mode is triggered. In addition, the system will also dynamically adjust based on the scenario information output by S142. For example, a "medium" priority fault will have its urgency temporarily increased if the scenario is "equipment is about to start." The system will generate a highly generalized action instruction, such as "Immediately handle the alarm light warning function loss fault (priority: Critical)," as the first dynamic maintenance content.

[0106] The system receives scenario information and fault details, and queries a "scenario-policy" knowledge base. This knowledge base defines the standard operating procedures and safety protocols to be followed in different scenarios. For example, the policy in a "hazardous materials area" is to wear protective clothing and apply for a hot work permit, while in a "planned maintenance operation" scenario, the policy is to collaborate with the on-site team and share safety measures and tools. The system combines these scenario policies with the specific fault details to generate customized guidance, ultimately forming a detailed set of operating and safety constraints as the second dynamic maintenance content.

[0107] The system combines the first and second pieces of information into a complete, scalable digital maintenance work order. It merges the information from the first two steps: the first piece defines the work order's "title" and "priority," while the second piece fills in the work order's "description" and "safety precautions." Simultaneously, the system automatically extracts required spare parts and estimated working hours from the fault item. The system generates a standardized "dynamic maintenance event" object and intelligently pushes it to the most suitable executor based on the maintenance personnel's skills, location, and workload, thus achieving a closed loop from diagnosis to decision-making to execution.

[0108] Specifically, the system has identified two fault items: FP-A-001 (loss of warning function, priority Critical) and FP-A-002 (decrease in red signal brightness, priority Moderate), and has identified the current scenario as a "planned maintenance operation scenario"; the system obtains the highest priority as "Critical" (from FP-A-001); since the current scenario is a "planned maintenance operation", which is a safe environment for maintenance operations, the system does not need to delay; therefore, the system generates the first dynamic maintenance content: "Immediately handle the fault of loss of warning function of alarm light (priority: Critical)".

[0109] The system identifies the scenario as "planned maintenance work" and queries the strategy library, determining that it should collaborate with the on-site team, sharing safety measures and tools. The system combines this strategy with the fault item content (replacing LED modules, repairing lamp covers) to generate a second dynamic maintenance content: "Operation Guidelines: This is a planned maintenance scenario. Maintenance personnel can collaborate with the on-site team. Safety Measures: Confirm that the existing safety warning line on site is effective. Wearing a standard safety helmet and safety shoes is sufficient. Resource Suggestions: You can borrow the on-site team's prepared aerial work platform and insulated toolbox without having to apply again."

[0110] The system integrates the first and second contents, supplements the detailed information of the fault items, and finally determines and generates a dynamic maintenance event, which is pushed to the mobile terminal of the maintenance engineer responsible for the area. The dynamic maintenance event not only includes the task title and priority, but also provides a detailed task description, safety and collaboration guidelines, spare parts list and estimated working hours.

[0111] refer to Figure 6 In step S15, the specific steps are as follows:

[0112] S151: Collect the dynamic maintenance event, determine multiple dynamic maintenance markers based on the detection of the dynamic maintenance event, determine the corresponding sub-maintenance content according to the tracing of each dynamic maintenance marker, collect multiple sub-maintenance content, and mark the corresponding maintenance time;

[0113] S152: Collect the work task list of the alarm light, determine multiple work tasks based on the identification of the work task list of the alarm light, and determine the corresponding maintenance path according to the multiple work tasks, multiple sub-maintenance contents and corresponding maintenance time.

[0114] S153: Based on the matching of the alarm light and each fault item, determine the maintenance part corresponding to each fault item for the alarm light. Based on the location of the maintenance part, the corresponding part shape and the maintenance path, determine the first level of autonomous maintenance content. Based on the location of the maintenance part, the corresponding part shape and the working content of the alarm light, determine the second level of autonomous maintenance content. Based on the first level of autonomous maintenance content and the second level of autonomous maintenance content, determine the autonomous maintenance system of the alarm light.

[0115] In the embodiments of this application, the dynamic maintenance event is collected, multiple dynamic maintenance markers are determined based on the detection of the dynamic maintenance event, and the corresponding sub-maintenance content is determined according to the tracing of each dynamic maintenance marker, so as to collect multiple sub-maintenance content and mark the corresponding maintenance time, which is compatible with the overall consideration of tracing each dynamic maintenance marker and ensures the accuracy of the corresponding sub-maintenance content.

[0116] At this point, the system separates the received maintenance events by field and uses a pre-trained language model fine-tuned in the industrial maintenance field to perform in-depth analysis of the core text descriptions. Through Named Entity Recognition (NER), the model identifies key entities such as "actions" (e.g., replacement, repair), "objects" (e.g., yellow LED modules), and "tools" (e.g., epoxy resin). Through relation extraction, the system constructs a subject-verb-object structure knowledge graph among these entities. The system outputs a structured list of "dynamic maintenance tags," such as {action: "replace", object: "yellow LED module"}, converting natural language instructions into machine-understandable semantic units.

[0117] The system accesses a large "task knowledge base" that stores various standard operating procedures (SOPs). Each SOP consists of a unique task ID and a series of ordered atomic steps. The system uses semantic similarity calculation based on a vector space model to perform fuzzy matching between the dynamically maintained tags and the SOPs in the knowledge base. Once the best matching SOP is found, the system traces back from the knowledge base and collects all the atomic steps contained in that SOP. These steps constitute the specific "sub-maintenance content".

[0118] The system associates the "standard working hours" of each atomic step with the database; the system sorts all sub-maintenance content according to the logical order defined in the SOP and sets a virtual start time (T=0); by sequentially accumulating the standard working hours of each step, the system calculates the "estimated start time" and "estimated end time" of each step; the system outputs a complete list of sub-maintenance content with timestamps, which constitutes a maintenance Gantt chart accurate to the second, laying the foundation for subsequent path planning and resource scheduling.

[0119] Furthermore, the alarm light's task list is collected, and multiple tasks are identified based on the identification of the alarm light's task list. The corresponding maintenance path is determined according to the multiple tasks, multiple sub-maintenance contents, and corresponding maintenance time. This overall consideration of multiple tasks, multiple sub-maintenance contents, and corresponding maintenance time ensures the accuracy of the corresponding maintenance path.

[0120] At this point, the system will communicate in real time with the factory's MES or ERP system through the API gateway to obtain the "work task list" of the equipment for the next forecast period (such as the next 24 hours). The system will parse this list and extract key information such as the planned time and running status of each task. According to the running status, the system will classify the tasks. Only those tasks in the "stop" or "safe standby" state will be identified as potential "maintenance windows" to provide a basis for subsequent time planning.

[0121] The system summarizes the total maintenance time calculated in S151 and uses a sliding window algorithm to scan the work task timeline identified in S152-1 to find a sufficiently long "downtime" period. If found, the system anchors the planned start time of the maintenance task to the starting point of the window, forming a "time path." At the same time, the system performs spatial path planning. It determines the current position of the executor (such as the maintenance robot) and loads the digital twin map of the factory. Using optimal path algorithms such as DLite, it calculates the optimal collision-free path from the starting point to the target point (alarm light) and outputs a series of waypoint coordinates, i.e., the "spatial path." The system integrates the "time path" and the "spatial path" to form a final "maintenance path" instruction containing spatiotemporal information, precisely specifying when to depart, how to arrive, and when to start the operation.

[0122] Therefore, based on the matching of alarm lights and various fault items, the maintenance parts corresponding to each fault item are determined. The first level of autonomous maintenance content is determined based on the location of this maintenance part, its corresponding shape, and the maintenance path. The second level of autonomous maintenance content is determined based on the location of this maintenance part, its corresponding shape, and the alarm light's operating function. The autonomous maintenance system of the alarm light is determined based on both levels, ensuring the accuracy of the system. Furthermore, dynamic maintenance events are introduced to further control the maintenance path, achieving a holistic consideration of the maintenance path, the maintenance parts corresponding to each fault item, and the alarm light's operating function, thus improving the accuracy of the autonomous maintenance system.

[0123] At this point, the system maps the fault location in the fault item (such as "yellow LED module") to the corresponding geometric component in the 3D model, thereby determining the "maintenance area". For non-component-level faults such as "lamp cover crack", the system will generate a bounding box or polygonal region on the model to define it based on the previously identified feature location. The system will perform geometric morphology analysis on the determined maintenance area, extract its surface type, edges, holes and other features, and automatically calculate key dimensions. At the same time, the system will also query the physical attributes associated with the part, such as material, surface roughness and fastening method, from the equipment digital twin database, providing comprehensive data support for subsequent refined operations.

[0124] The first level of autonomous maintenance focuses on generating transitional instructions from macroscopic paths to microscopic operations, ensuring the robot can safely and accurately reach and be ready to perform maintenance. This includes global navigation using the waypoint sequence output by S152, and generating a series of fine-grained approach, positioning, and scanning instructions based on the shape of the maintenance site after reaching the destination, to compensate for the deviation between the theoretical model and the actual position. The second level of autonomous maintenance focuses on generating fine-grained operation instructions for performing specific sub-maintenance tasks on the maintenance site. The system will automatically select appropriate end effectors (such as screwdrivers or grippers) and set operating parameters (such as torque) based on the shape of the site and the task content, transforming the sub-maintenance steps in S151 into a sequence of low-level control instructions for the robot, such as movement trajectories and tool movements.

[0125] The system uses the maintenance time marked in S151 as the main axis to seamlessly orchestrate the first and second layers of content, and designs the entire process as a finite state machine that includes navigation, execution, verification, and error handling to ensure the robustness of the system. The system encapsulates all orchestrated instructions, state machine logic, and related data into a standardized "autonomous maintenance system" program package and deploys it into the control system of the maintenance robot. After receiving the instructions, the robot can autonomously complete the entire process from navigation, perception, operation to verification without any human intervention, truly realizing "autonomous maintenance".

[0126] Please see Figure 7 , Figure 7 This is a schematic diagram of the structural composition of a visual detection-based alarm light fault identification system according to an embodiment of the present invention; the visual detection-based alarm light fault identification system includes:

[0127] The current lighting event module 21 is used to determine the lighting image based on the visual detection of the alarm light by the camera, determine multiple lighting areas based on the recognition of the lighting image, and determine the current lighting event based on the area location, area shape and alarm light task list of each lighting area.

[0128] The internal abnormal lighting event module 22 is used to determine multiple external abnormal lighting events based on the identification of the current lighting event, and to determine internal abnormal lighting events based on multiple working data of the alarm light during the lighting display process and the current lighting mode of the alarm light.

[0129] The fault item module 23 is used to determine multiple fault contents of the alarm light based on multiple external abnormal light events, internal abnormal light events and the overall shape of the alarm light, collect the temperature distribution map of the alarm light, and determine multiple fault items of the alarm light based on each set of fault contents, the temperature distribution map of the alarm light and the corresponding fault signal.

[0130] The dynamic maintenance event module 24 is used to determine the project priority of each fault project based on the current work tasks of multiple fault projects and alarm lights, and to determine the dynamic maintenance event of the alarm light according to the project priority of each fault project, the corresponding project content and the scene where the alarm light is located.

[0131] The autonomous maintenance system module 25 is used to determine multiple sub-maintenance contents based on the identification of the dynamic maintenance event, determine the corresponding maintenance path according to the multiple sub-maintenance contents, the corresponding maintenance time, and the alarm light's work task list, and determine the autonomous maintenance system of the alarm light based on the maintenance path, the maintenance part of the alarm light relative to each fault item, and the work content of the alarm light.

[0132] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A fault identification method for alarm lights based on visual detection, characterized in that, include: The light image is determined based on the visual detection of the alarm light by the camera. Multiple light areas are determined based on the recognition of the light image. The current light event is determined based on the area location, area shape and alarm light task list of each light area. Based on the identification of the current lighting event, multiple external abnormal lighting events are identified, and internal abnormal lighting events are identified based on multiple working data of the alarm light during the lighting display process and the current lighting mode of the alarm light. Based on multiple external abnormal lighting events, internal abnormal lighting events, and the overall shape of the alarm lights, multiple sets of fault contents of the alarm lights are determined. Temperature distribution maps of the alarm lights are collected. Based on each set of fault contents, the temperature distribution maps of the alarm lights, and the corresponding fault signals, multiple fault items of the alarm lights are determined. The fault items include fault location, type, root cause, impact assessment, confidence level, and specific maintenance recommendations. Based on the current work tasks of multiple fault items and alarm lights, the project priority of each fault item is determined. According to the project priority of each fault item, the corresponding project content and the scene where the alarm light is located, the dynamic maintenance event of the alarm light is determined. The dynamic maintenance event not only includes the task title and priority, but also provides a detailed task description, safety and collaboration guidelines, spare parts list and estimated working hours. Based on the identification of the dynamic maintenance event, multiple sub-maintenance contents are determined. According to the multiple sub-maintenance contents, the corresponding maintenance time, and the work task list of the alarm light, the corresponding maintenance path is determined. Based on the maintenance path, the maintenance part of the alarm light relative to each fault item, and the work content of the alarm light, the autonomous maintenance system of the alarm light is determined.

2. The fault identification method for alarm lights based on visual detection according to claim 1, characterized in that, The process involves determining a light image based on visual detection of the alarm light using a camera, identifying multiple light areas based on the recognition of this light image, and determining the current light event based on the location, shape, and task list of each light area, including: The current position of the alarm light is collected, and multiple cameras are determined based on the surrounding detection of the current position of the alarm light. The multiple cameras are distributed around the alarm light and dynamically capture images of the alarm light to collect the light image of the alarm light. Multiple light features are identified based on the light images of the alarm lights, and the spatial position of each light feature is marked. Multiple light areas are determined based on the spatial position of each light feature and the image content of the light images. At the same time, the work task list of the alarm lights is collected, and the current light event is determined according to the area position, area shape and work task list of each light area.

3. The fault identification method for alarm lights based on visual detection according to claim 1, characterized in that, The process of identifying multiple external abnormal lighting events based on the recognition of the current lighting event, and identifying internal abnormal lighting events based on multiple working data of the alarm light during the lighting display process and the current lighting mode of the alarm light, includes: Based on the identification of the current light event, multiple external abnormal features are identified. Based on the characteristic location, corresponding feature shape and working history of the alarm light of the multiple external abnormal features, the corresponding external abnormal light event is determined, so as to collect multiple external abnormal light events. The system monitors the operation of alarm lights in real time, determines multiple operational data points of the alarm lights during the lighting display process based on the alarm light's operational database, and marks the current lighting mode of the alarm lights. Based on the multiple operational data points of the alarm lights during the lighting display process, the corresponding fault signals, and the current lighting mode of the alarm lights, internal abnormal lighting events are determined.

4. The fault identification method for alarm lights based on visual detection according to claim 1, characterized in that, The process involves determining multiple sets of fault content for the alarm light based on various external and internal abnormal lighting events and the overall shape of the alarm light; collecting a temperature distribution map of the alarm light; and determining multiple fault items for the alarm light based on each set of fault content, the temperature distribution map of the alarm light, and the corresponding fault signals. These items include: The overall shape of the alarm light is determined based on the detection of the alarm light, and the first sub-fault content is determined based on multiple external abnormal light events and the overall shape of the alarm light. The second sub-fault content is determined based on the internal abnormal lighting events and the overall shape of the alarm light. Multiple sets of fault content for the alarm light are determined based on the matching of the first sub-fault content, the second sub-fault content and the working history of the alarm light.

5. The fault identification method for alarm lights based on visual detection according to claim 4, characterized in that, The process of determining multiple sets of fault content for the alarm light based on multiple external abnormal light events, internal abnormal light events, and the overall shape of the alarm light; collecting the temperature distribution map of the alarm light; and determining multiple fault items for the alarm light based on each set of fault content, the temperature distribution map of the alarm light, and the corresponding fault signals also includes: Multiple temperature data points and corresponding temperature detection locations are determined based on the temperature detection of the alarm light. A temperature distribution map of the alarm light is determined based on the multiple temperature data points, the corresponding temperature detection locations, and the overall shape of the alarm light. First fault information is determined based on each group of fault content and the temperature distribution map of the alarm light. Second fault information is determined based on each group of fault content and the corresponding fault signal. Multiple fault items of the alarm light are determined based on the multiple interactions between the first fault information and the second fault information.

6. The fault identification method for alarm lights based on visual detection according to claim 1, characterized in that, The process involves determining the project priority of each fault item based on the current work tasks of multiple fault items and alarm lights, and determining the dynamic maintenance events of the alarm lights based on the project priority, corresponding project content, and the scene in which the alarm lights are located. This includes: Collect multiple fault items, determine the corresponding item content based on the identification of multiple fault items, and determine the item priority of each fault item according to the item content of multiple fault items, the corresponding work impact level and the current work task of the alarm light. Based on environmental detection of the current location of the alarm light, multiple environmental features are determined. The scene in which the alarm light is located is determined according to the feature areas of the multiple environmental features, the corresponding feature shapes, and the overall shape of the alarm light.

7. The fault identification method for alarm lights based on visual detection according to claim 6, characterized in that, The process of determining the project priority of each fault item based on multiple fault items and the current work tasks of alarm lights, and determining the dynamic maintenance events of alarm lights based on the project priority of each fault item, the corresponding project content, and the scene in which the alarm light is located, also includes: The first dynamic maintenance content is determined based on the scene where the alarm light is located and the project priority of each fault item. The second dynamic maintenance content is determined based on the scene where the alarm light is located and the project content of each fault item. The dynamic maintenance event of the alarm light is determined based on the first dynamic maintenance content and the second dynamic maintenance content.

8. The fault identification method for alarm lights based on visual detection according to claim 1, characterized in that, The process involves identifying multiple sub-maintenance items based on the dynamic maintenance event, determining corresponding maintenance paths based on these sub-maintenance items, their corresponding maintenance times, and the alarm light's task list, and then establishing an autonomous maintenance system for the alarm light based on this maintenance path, the maintenance location of the alarm light relative to each fault item, and the alarm light's task content. This system includes: The dynamic maintenance event is collected, and multiple dynamic maintenance markers are determined based on the detection of the dynamic maintenance event. The corresponding sub-maintenance content is determined by tracing each dynamic maintenance marker, so as to collect multiple sub-maintenance content and mark the corresponding maintenance time. Collect the task list of alarm lights, identify multiple tasks based on the identification of the task list of alarm lights, and determine the corresponding maintenance path according to the multiple tasks, multiple sub-maintenance contents and corresponding maintenance time.

9. The fault identification method for alarm lights based on visual detection according to claim 8, characterized in that, The process of identifying multiple sub-maintenance contents based on the dynamic maintenance event, determining corresponding maintenance paths based on these sub-maintenance contents, corresponding maintenance times, and alarm light task lists, and establishing an autonomous maintenance system for the alarm lights based on these maintenance paths, the maintenance locations corresponding to each fault item, and the alarm lights' work content, further includes: Based on the matching of alarm lights and various fault items, the maintenance parts corresponding to each fault item are determined. The first level of autonomous maintenance content is determined according to the location of the maintenance part, the corresponding part shape, and the maintenance path. The second level of autonomous maintenance content is determined according to the location of the maintenance part, the corresponding part shape, and the working content of the alarm light. The autonomous maintenance system of the alarm light is determined based on the first and second levels of autonomous maintenance content.

10. A fault identification system for alarm lights based on visual detection, characterized in that, The visual detection-based alarm light fault identification system is applied to the visual detection-based alarm light fault identification method as described in any one of claims 1-9, wherein the visual detection-based alarm light fault identification system comprises: The current lighting event module is used to determine the lighting image based on the visual detection of the alarm light by the camera, determine multiple lighting areas based on the recognition of the lighting image, and determine the current lighting event based on the area location, area shape and alarm light task list of each lighting area. The internal abnormal lighting event module is used to identify multiple external abnormal lighting events based on the identification of the current lighting event, and to identify internal abnormal lighting events based on multiple working data of the alarm light during the lighting display process and the current lighting mode of the alarm light. The fault item module is used to determine multiple fault contents of the alarm light based on multiple external abnormal light events, internal abnormal light events and the overall shape of the alarm light, collect the temperature distribution map of the alarm light, and determine multiple fault items of the alarm light based on each set of fault contents, the temperature distribution map of the alarm light and the corresponding fault signal. The dynamic maintenance event module is used to determine the project priority of each fault project based on the current work tasks of multiple fault projects and alarm lights, and to determine the dynamic maintenance event of the alarm light according to the project priority of each fault project, the corresponding project content and the scene in which the alarm light is located. The autonomous maintenance system module is used to identify multiple sub-maintenance contents based on the identification of the dynamic maintenance event, determine the corresponding maintenance path according to the multiple sub-maintenance contents, the corresponding maintenance time, and the alarm light's work task list, and determine the autonomous maintenance system of the alarm light based on the maintenance path, the maintenance part of the alarm light relative to each fault item, and the work content of the alarm light.

Images