Device condition monitoring method, device detection system, and computer-readable storage medium

Abstract

The application discloses a device state monitoring method, comprising: establishing a preset database. Obtain multiple historical rendering images of multiple detection devices, and train a reference model to obtain a target detection model. Real-time acquisition of continuous multiple target rendering images generated by the target detection device at the target detection point. Based on the target detection model, each target rendering image is detected to determine a template region image set. Calculate the first similarity index of each target template region image and the target standard template rendering image to determine the first similarity index sequence and the corresponding template similarity index. Calculate the second similarity index between the target template region image and the corresponding historical template region image to determine the second similarity index sequence and the corresponding stability index. According to the comparison result of the template similarity index and the first preset threshold value, and / or the comparison result of the stability index and the second preset threshold value, the device state of the target detection device is judged.

Description

Technical Field

[0001] This application relates to the field of equipment monitoring technology, and in particular to an equipment condition monitoring method, an equipment testing system, and a computer-readable storage medium. Background Technology

[0002] In the appearance inspection process, optical inspection equipment is one of the key pieces of equipment ensuring product processing accuracy and appearance quality. The template positioning accuracy of optical inspection equipment directly determines the accuracy of defect detection and product yield. In related technologies, the accuracy verification and anomaly diagnosis of optical inspection equipment typically employs offline methods of manual periodic sampling and experience-based judgment. This involves on-site maintenance personnel manually selecting inspection renderings and comparing them to the template frame positions, relying on their experience to determine if the equipment positioning is abnormal. Often, fuzzy acceptance standards set manually are used as the basis for judgment to ensure the basic inspection accuracy of the optical inspection equipment. However, this approach cannot continuously perceive the changes and fluctuations in the positioning accuracy of each inspection point in real time, nor can it quantify and distinguish between the random fluctuations of the equipment's own mechanical vibration and the systematic offset of the template reference drift—two types of anomalies. Therefore, it cannot achieve dynamic monitoring of the accuracy status and unified benchmarking and control of equipment across multiple plants when the equipment's motion module ages, production line environmental vibration interference intensifies, or the template reference drifts. This easily leads to the equipment failing to detect accuracy degradation in time, resulting in inspection frame offset, missed defects, and misjudgments. Summary of the Invention

[0003] This application provides a method for monitoring equipment condition, an equipment testing system, and a computer-readable storage medium.

[0004] This application provides a device status monitoring method, the method comprising: Establish a preset database, which includes historical rendering images and standard template rendering images of multiple detection devices at multiple detection points; Multiple historical rendering images from various detection devices are acquired, and a benchmark model is trained to obtain a target detection model. Multiple consecutive target rendering images generated by the target detection device at the target detection point are acquired in real time and stored in the preset database; Based on the target detection model, region of interest detection is performed on each of the target rendering images, and the region of interest is cropped to determine a template region image set, wherein the template region image set includes multiple target template region images; Calculate the first similarity index for each target template region image and the target standard template rendering image to determine the first similarity index sequence and the template similarity index corresponding to the first similarity index sequence. The target standard template rendering image is used to indicate the standard template rendering image of the target detection device at the target detection point. Calculate a second similarity index between the target template region image and the corresponding historical template region image to determine the second similarity index sequence and the stability index corresponding to the second similarity index sequence; The device status of the target detection device is determined based on the comparison result of the template similarity index with the first preset threshold and / or the comparison result of the stability index with the second preset threshold.

[0005] Thus, this application provides a device status monitoring method. It establishes a preset database, which includes historical rendering images and standard template rendering images of multiple detection devices at multiple detection points. Next, multiple historical rendering images from the multiple detection devices are acquired, and a benchmark model is trained to obtain a target detection model. Subsequently, multiple consecutive target rendering images generated by the target detection devices at target detection points are acquired in real time and stored in the preset database. Furthermore, based on the target detection model, a region of interest (ROI) detection is performed on each target rendering image, and the ROI is cropped to determine a template region image set, which includes multiple target template region images. Then, a first similarity index is calculated between each target template region image and the target standard template rendering image to determine a first similarity index sequence and a template similarity index corresponding to the first similarity index sequence. The target standard template rendering image is used to indicate the standard template rendering image of the target detection device at the target detection point. Simultaneously, a second similarity index is calculated between the target template region image and the corresponding historical template region image to determine a second similarity index sequence and a stability index corresponding to the second similarity index sequence. Finally, based on the comparison results of the template similarity index with the first preset threshold and / or the stability index with the second preset threshold, the equipment status of the target detection equipment is determined. This allows for the evaluation of equipment status through template similarity and / or stability indices, distinguishing between two types of anomalies: systematic template positioning offset and random self-positioning fluctuations. This enables automated and refined diagnosis of equipment positioning accuracy, providing clear optimization directions for equipment maintenance. Simultaneously, relying on unified quantitative evaluation standards and centralized database management, horizontal benchmarking of detection accuracy across factories and multiple machines can be achieved, providing reliable data for unifying detection quality standards throughout the factory. This ensures consistency in detection across different production lines and equipment, avoiding quality control loopholes caused by differences in equipment accuracy.

[0006] In some implementations, the real-time acquisition of multiple consecutive target rendering images generated by the target detection device at the target detection point and the storage of them in the preset database includes: Based on the target detection device, visual positioning and photography are performed at the target detection point to generate the original image; The original image is subjected to template frame overlay processing to generate the target rendering image, and the target rendering image is stored in the preset database. The template frame is used to define the target area to be detected corresponding to the target detection point.

[0007] In this way, the generated target rendering image can simultaneously carry the shooting features of the actual component and the template positioning results of the detection equipment, intuitively reflecting the actual visual positioning status of the equipment at the target detection point. This ensures that the analysis object of the subsequent similarity calculation directly corresponds to the template positioning accuracy of the equipment, rather than simply the difference in image content, providing a unified and reliable image benchmark for subsequent calculations.

[0008] In some embodiments, the preset database also stores processing information corresponding to each of the rendered images, including the detection equipment number, product model, shooting time, detection point number, and / or image storage path.

[0009] In this way, the structured and indexed management of the rendered graph data can be achieved, providing clear and reliable data support for subsequent indicator calculation, status assessment and operation and maintenance traceability.

[0010] In some implementations, the baseline model includes a Backbone module, a Neck module, and a Head module; The Backbone module consists of a CBS convolutional block, a C3K2 module, an SPPF module, and a C2PSA module, and is used to extract basic features of the image; The Neck module includes an upsampling module, a feature concatenation module, a C3K2 module, and a CBS convolutional block, used to fuse feature maps of different scales; The Head module consists of a stack of CBS convolutional blocks, DSC modules, and Conv2d convolutional layers, and is used to output predicted bounding boxes, confidence scores, and category information.

[0011] In this way, the architecture and core module composition of the benchmark model can be clearly defined, ensuring the feature extraction capability and detection accuracy of the target detection model for industrial rendering images. Among them, the multi-type convolution and attention modules of the backbone can accurately capture the visual features of the template region, the multi-scale feature fusion of the Neck can adapt to the detection needs of rendering images of different sizes and different fields of view, and the head module can output accurate bounding box localization results.

[0012] In some implementations, acquiring multiple historical rendering images from multiple detection devices and training a baseline model to obtain a target detection model includes: In the baseline model, a coordinate attention mechanism is embedded in the backbone network of the Backbone module, and a multi-scale detail preservation path is added to the feature fusion network in the Neck module; Based on the baseline model, region of interest detection is performed on the historical rendering images, and the first template region bounding box is marked in each of the historical rendering images; Based on the bounding box of the first template region and the bounding box of the second template region in the standard template rendering image, determine the bounding box loss, classification loss, and distribution focus loss; Based on preset weights, the bounding box loss, the classification loss, and the distribution focus loss are weighted and summed to determine the training loss function; Based on the training loss function, the baseline model is iteratively optimized to obtain the target detection model.

[0013] In this way, by optimizing the network architecture and subjecting it to the joint constraints of multi-dimensional loss functions, the localization accuracy, scene robustness, and industrial adaptability of the object detection model for the rendered image template region can be improved, thereby providing stable and reliable model support for the accurate cropping of the region of interest in the future.

[0014] In some implementations, calculating a first similarity index for each of the target template region images and the target standard template rendering image to determine a first similarity index sequence and a template similarity index corresponding to the first similarity index sequence includes: Based on the first preset sliding window, the target template region image and the target standard template rendering image are calculated block by block to obtain the first local similarity value of the corresponding sliding window; Calculate the average of the first local similarity values ​​of all sliding windows to determine the first similarity index between the currently processed template region image and the target standard template rendering image; Based on the first similarity index, determine the first similarity index sequence; Calculate the arithmetic mean of all first similarity indices in the first similarity index sequence to determine the template similarity index.

[0015] Thus, by calculating block by block using a sliding window and then averaging the results, the interference of local noise and minor artifacts on the similarity calculation results can be effectively reduced, improving the stability and reliability of the similarity index. Furthermore, obtaining the final template similarity index by taking the arithmetic mean of the similarity indices from multiple consecutive images can further offset the impact of random fluctuations in a single detection, more accurately reflecting the degree of systematic deviation between the device's detection results and the standard template, and avoiding misjudgments caused by anomalies in a single image.

[0016] In some implementations, the step of calculating the first local similarity value of the corresponding sliding window by block-by-block calculation of the currently processed target template region image and the target standard template rendering image includes: Based on the first preset sliding window, calculate the first local mean, the first local variance, and the first local covariance between the currently processed target template region image and the target standard template rendering image; Calculate the first luminance component based on the first local mean; Calculate the first contrast component based on the first local variance; Calculate the first structural component based on the first local covariance; The first luminance component, the first contrast component, and the first structure component are weighted and fused to obtain the first local similarity value.

[0017] In this way, by comprehensively calculating image similarity from three dimensions of brightness, contrast, and structure, the calculation logic of local similarity values ​​can be made more in line with the laws of human visual perception and more suitable for the imaging characteristics of industrial inspection scenarios. This can effectively avoid the defects of traditional pixel-level error algorithms that are easily affected by light fluctuations and random noise interference, thus misjudging anomalies.

[0018] In some implementations, calculating a second similarity index between the target template region image and the corresponding historical template region image to determine a second similarity index sequence and a stability index corresponding to the second similarity index sequence includes: Based on the second preset sliding window, the target template region image and the corresponding historical template region image are calculated to obtain the second local similarity value of the corresponding sliding window; Calculate the average of the second local similarity values ​​of all sliding windows to determine the second similarity index between the current template region image and the currently processed historical template region image; Based on the second similarity index, determine the second similarity index sequence; Calculate the dispersion statistics of all second similarity indices in the second similarity index sequence to determine the stability index.

[0019] Thus, by using a statistical method of pairwise comparison of the entire sample and a quantitative logic of dispersion, it is possible to characterize the fluctuation characteristics of continuous detection results at the same location of the same device, objectively reflect the positioning stability of the device itself, and provide an independent quantitative evaluation dimension for distinguishing between random jitter anomalies of the device and systematic offset anomalies of the template.

[0020] In some implementations, the step of calculating a second local similarity value for the target template region image and the corresponding historical template region image based on a second preset sliding window includes: Based on the second preset sliding window, calculate the second local mean, second local variance and second local covariance of the target template region image and the corresponding historical template region image; The second luminance component is calculated based on the second local mean. The second contrast component is calculated based on the second local variance; The second structural component is calculated based on the second local covariance; The second luminance component, the second contrast component, and the second structure component are weighted and fused to obtain the second local similarity value.

[0021] In this way, by comprehensively calculating image similarity from three dimensions of brightness, contrast, and structure, the calculation logic of local similarity values ​​can be made more in line with the laws of human visual perception and more suitable for the imaging characteristics of industrial inspection scenarios. This can effectively avoid the defects of traditional pixel-level error algorithms that are easily affected by light fluctuations and random noise interference, thus misjudging anomalies.

[0022] In some implementations, determining the device status of the target detection device based on the comparison result of the template similarity index with a first preset threshold and / or the comparison result of the stability index with a second preset threshold includes: If the template similarity index is less than or equal to the first preset threshold, it is determined that the target standard template rendering image does not match the target detection point; and / or If the stability index is greater than or equal to the second preset threshold, the target detection device is determined to be unstable in its positioning.

[0023] Thus, by using an independent threshold judgment mechanism based on dual-dimensional indicators, it is possible to accurately distinguish between two different root causes: template positioning adaptability anomalies and equipment performance anomalies. This provides clear fault indications for equipment operation and maintenance, improving the accuracy of equipment fault diagnosis and the efficiency of operation and maintenance.

[0024] In some embodiments, the method further includes: If the target detection device itself is unstable or the target standard template rendering does not match the target detection point, an alarm message is generated. The alarm message includes the anomaly type, the time of the anomaly, the identifier of the target detection device and / or the identifier of the target detection point.

[0025] In this way, the proactive alarm function can notify maintenance personnel as soon as the equipment accuracy is determined to be abnormal, which can significantly shorten the response time to abnormalities, avoid batch product defects caused by continuous abnormal operation, and effectively reduce product quality risks and production losses.

[0026] In some embodiments, the method further includes: The stability index of the target detection device at the target detection point is continuously acquired over multiple consecutive time windows to form stability trend data; If the rate of change of the stability trend data within a preset time period is greater than or equal to a third preset threshold, an early warning message is generated. The early warning message includes the abnormal warning time, the identifier of the target detection device, and / or the identifier of the target detection point.

[0027] Thus, by using the trend warning function based on the rate of change of stability indicators, it is possible to make advance predictions of equipment accuracy anomalies. That is, it is possible to issue early warnings when the equipment performance shows a trend of decline but before defects occur, so as to guide maintenance personnel to carry out preventive maintenance in a timely manner.

[0028] This application also provides a device testing system, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the above method.

[0029] Thus, the equipment inspection system provided in this application can evaluate the equipment status through template similarity and / or stability indicators, distinguishing between two types of abnormal root causes: systematic template positioning offset and random self-positioning fluctuation. This enables automated and refined diagnosis of equipment positioning accuracy, providing a clear direction for equipment operation and maintenance. Simultaneously, relying on unified quantitative evaluation standards and centralized database management, it allows for horizontal benchmarking of inspection accuracy across factories and multiple machines, providing reliable data for unifying inspection quality standards throughout the factory, ensuring consistency in inspection across different production lines and equipment, and avoiding quality control loopholes caused by differences in equipment accuracy.

[0030] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method. Attached Figure Description

[0031] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, wherein: Figure 1 This is one of the flowcharts illustrating a device status monitoring method according to certain embodiments of this application; Figure 2 This is a second schematic flowchart of a device status monitoring method according to certain embodiments of this application; Figure 3 This is the third flowchart illustrating a device status monitoring method according to certain embodiments of this application; Figure 4 This is a schematic diagram of the target detection model according to certain embodiments of this application; Figure 5 This is the fourth flowchart illustrating a device status monitoring method according to certain embodiments of this application; Figure 6 This is the fifth flowchart illustrating a device status monitoring method according to certain embodiments of this application; Figure 7 This is a sixth schematic flowchart of a device status monitoring method according to certain embodiments of this application; Figure 8 This is the seventh flowchart illustrating a device status monitoring method according to certain embodiments of this application; Figure 9 This is the eighth flowchart illustrating a device status monitoring method according to certain embodiments of this application; Figure 10 This is the ninth flowchart of a device status monitoring method according to certain embodiments of this application; Figure 11 This is the tenth flowchart illustrating a device status monitoring method according to certain embodiments of this application. Detailed Implementation

[0032] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of this application, and should not be construed as limiting the embodiments of this application.

[0033] In the appearance inspection processes of precision manufacturing fields such as electronic components, semiconductor packaging, and printed circuit boards, optical inspection equipment is one of the inspection tools that ensures the processing accuracy and appearance quality of products. It is widely used in key workstations such as solder joint defect detection, dimensional accuracy verification, and appearance defect screening. Among these, the template positioning accuracy of optical inspection equipment directly determines the accuracy of defect detection and the product yield.

[0034] In related technologies, the accuracy verification and anomaly diagnosis of optical inspection equipment typically employs offline troubleshooting methods involving periodic manual sampling and experience-based judgment. On-site maintenance personnel usually select a limited number of sample points from the equipment's generated inspection renderings at a fixed frequency per shift, day, or week. They then visually compare the relative positions of the inspection frame lines in the renderings with the standard template, relying on their professional experience to roughly determine if there are any anomalies in the equipment's positioning. Furthermore, the pass / fail criteria for optical inspection equipment are often manually set by maintenance personnel based on experience, lacking unified quantitative indicators, and often using the absence of noticeable deviations to the naked eye as a pass / fail criterion, thus ensuring the basic inspection accuracy of the optical inspection equipment.

[0035] However, the above-mentioned model has many inherent limitations and cannot adapt to the current needs of large-scale, high-precision production line management. First, manual sampling inspection is an intermittent offline inspection, which cannot perceive the changes and fluctuations in the positioning accuracy of each inspection point of the equipment in real time and continuously. The process of equipment accuracy slowly declining over time is completely outside the monitoring blind zone. Often, by the time obvious abnormalities are found in sampling inspection, the accuracy deviation has already persisted for a long production cycle.

[0036] Furthermore, the aforementioned model cannot quantify and distinguish between two fundamentally different root causes of anomalies: random fluctuations caused by the equipment's own mechanical vibration and systematic offsets caused by template reference drift. Specifically, the former is mostly caused by changes in hardware performance such as guide rail wear, increased lead screw backlash, servo motor vibration, and environmental vibration interference, manifesting as irregular fluctuations in the positioning position of each detection. The latter is mostly caused by reference issues such as template calibration deviation, overall coordinate system offset, and incorrect changeover parameter settings, manifesting as an overall shift of the detection position in a fixed direction.

[0037] Thus, the failure to detect equipment accuracy degradation in a timely manner can directly lead to detection frame misalignment, resulting in missed detection of actual defects and misclassification of good products. Furthermore, the inability to pinpoint the root cause of the anomaly can increase the time and manpower costs of equipment maintenance. Maintenance personnel need to check all possible aspects, including mechanical structure, parameter settings, and template calibration, extending equipment downtime and impacting the overall operational efficiency of the production line. In addition, the lack of unified quantitative standards may lead to inconsistent detection accuracy levels across different factories and production lines, making it impossible to standardize product quality.

[0038] Based on the above issues, please refer to Figure 1 This application provides a device status monitoring method, the method comprising: S011: Establish a preset database; S012: Acquire multiple historical rendering images from multiple detection devices and train the benchmark model to obtain the target detection model; S013: Real-time acquisition of multiple consecutive target rendering images generated by the target detection device at the target detection point, and storage in a preset database; S014: Based on the object detection model, perform region of interest detection on each target rendering image, crop the region of interest, and determine the template region image set; S015: Calculate the first similarity index of each target template region image and the target standard template rendering image to determine the first similarity index sequence and the template similarity index corresponding to the first similarity index sequence; S016: Calculate the second similarity index between the target template region image and the corresponding historical template region image to determine the second similarity index sequence and the stability index corresponding to the second similarity index sequence; S017: Determine the device status of the target detection device based on the comparison results of the template similarity index with the first preset threshold and / or the comparison results of the stability index with the second preset threshold.

[0039] This application provides an equipment detection device. The equipment status monitoring method of this application can be implemented by the equipment detection device of this application. Specifically, the equipment detection device includes an acquisition module, a model training module, a detection module, a calculation module, and a determination module. The model training module is used to establish a preset database. The model training module is also used to acquire multiple historical rendering images of multiple detection devices and train a benchmark model to obtain a target detection model. The acquisition module is also used to acquire multiple consecutive target rendering images generated by the target detection device at the target detection point in real time and store them in the preset database. The detection module is used to perform region of interest detection on each target rendering image based on the target detection model, and to crop the region of interest to determine a template region image set. The calculation module is used to calculate a first similarity index between each target template region image and a target standard template rendering image to determine a first similarity index sequence and a template similarity index corresponding to the first similarity index sequence. It also calculates a second similarity index between the target template region image and the corresponding historical template region image to determine a second similarity index sequence and a stability index corresponding to the second similarity index sequence. The determination module is used to determine the device status of the target detection device based on the comparison results of the template similarity index with the first preset threshold and / or the comparison results of the stability index with the second preset threshold.

[0040] This application also provides a computer device, which includes a memory and a processor. The device status monitoring method of this application can be implemented by the computer device of this application. Specifically, the memory stores a computer program, and the processor is used to establish a preset database, acquire multiple historical rendering images from multiple detection devices, train a benchmark model, and obtain a target detection model. It also acquires multiple consecutive target rendering images generated by the target detection devices at target detection points in real time and stores them in the preset database. The processor is further used to perform region of interest detection on each target rendering image based on the target detection model, crop the region of interest, and determine a template region image set. It also calculates a first similarity index between each target template region image and a target standard template rendering image to determine a first similarity index sequence and a template similarity index corresponding to the first similarity index sequence. Finally, it calculates a second similarity index between the target template region image and the corresponding historical template region image to determine a second similarity index sequence and a stability index corresponding to the second similarity index sequence. The processor is also used to determine the device status of the target detection device based on the comparison result of the template similarity index with a first preset threshold and / or the comparison result of the stability index with a second preset threshold.

[0041] Specifically, the preset database refers to a cloud-based or local database used for centralized storage and management of all plant-wide testing data. The preset database stores historical renderings generated by each testing device at each testing point, verified standard template renderings, and processing information associated with each image. This processing information includes, but is not limited to, equipment number, product model, shooting time, point number, and image path.

[0042] Inspection equipment refers to automated optical inspection (AOI) equipment used in industrial production to detect surface defects in products. It takes pictures of products through a vision system and completes defect identification and location based on preset templates.

[0043] Inspection points refer to the specific physical locations where AOI equipment needs to perform visual positioning and defect inspection when inspecting a single product. Different product models correspond to different sets of inspection points.

[0044] Historical rendering images refer to images generated by AOI equipment during historical inspections. The original image is captured after the AOI equipment completes the visual positioning of the product, and then a green template frame line is superimposed on the original image to define the area to be inspected, thus forming the final rendering image.

[0045] The standard template rendering image refers to the baseline image corresponding to a specific detection point that has been verified through multiple rounds. It represents the ideal state of template positioning at that point and serves as the reference standard for all subsequent similarity calculations.

[0046] The baseline model refers to the initial neural network architecture used to train the target detection model. In this application, the YOLO series target detection model is used as the baseline model, which has the characteristics of fast detection speed and high accuracy, and is suitable for industrial real-time detection scenarios.

[0047] The object detection model refers to the baseline model that has been trained in a supervised manner on historical rendering images and can automatically identify and accurately locate template regions in any rendering image.

[0048] Target detection equipment and target detection points refer to the specific AOI equipment currently undergoing status monitoring and its corresponding specific detection location.

[0049] The target rendering image refers to the rendering image generated in real time by the target detection device at the target detection point. The generation method is the same as that of the historical rendering. Figure 1 To.

[0050] The Region of Interest (ROI) refers specifically to the area enclosed by the template outline in the rendered image, as described in this application. This area reflects the device's positioning accuracy, and since the background area does not affect the accuracy assessment, it needs to be extracted separately to eliminate interference.

[0051] The template region image set refers to the collection of multiple target template region images obtained after performing ROI detection and cropping on multiple consecutive target rendering images. It serves as the input data for subsequent two-dimensional index calculations.

[0052] The first similarity index refers to the structural similarity index (SSIM) between a single target template region image and the corresponding standard template rendering image, which is used to measure the degree of deviation between a single detection result and the ideal standard.

[0053] The first similarity index sequence refers to a sequence formed by arranging multiple first similarity indices corresponding to multiple consecutive target rendering images in chronological order of shooting time.

[0054] The template similarity index refers to the arithmetic mean of all values ​​in the first similarity index sequence, reflecting the overall deviation level between the equipment detection results and the standard template over a period of time.

[0055] The second similarity index refers to the structural similarity index between any two different target template region images in the template region image set, which is used to measure the degree of difference between two consecutive detection results.

[0056] The second similarity index sequence refers to the sequence consisting of the second similarity indices corresponding to all pairwise image combinations in the template region image set.

[0057] The stability index refers to the statistic of the dispersion of the second similarity index sequence (usually using the standard deviation). The larger the value, the more drastic the fluctuation of continuous detection results and the worse the stability of equipment positioning.

[0058] The first preset threshold and the second preset threshold refer to the critical values ​​obtained based on a large amount of historical normal data statistics, which are used to determine whether the template positioning is appropriate and whether the device itself is stable.

[0059] Equipment status refers to the comprehensive judgment result of the template positioning adaptability and its own operational stability of AOI inspection equipment. It usually includes three categories: normal status, template positioning mismatch status, and equipment unstable status.

[0060] In some implementations, the process of the equipment status monitoring method provided in this application may be as follows: First, collect the historical rendering images generated by all AOI devices at each detection point, as well as the verified standard template rendering images of each point, and store each image in association with the corresponding processing information.

[0061] Subsequently, multiple historical rendering images covering different devices, locations, and lighting conditions were extracted from a pre-set database as a training set. Manually labeled template regions were used as ground truth to supervise the training of the baseline model. By iteratively optimizing the model parameters, the model was able to accurately identify and locate template regions in the rendering images under complex industrial scenarios, ultimately resulting in a target detection model with generalization capabilities.

[0062] Next, during normal equipment production, multiple consecutive target rendering images generated by the target detection equipment at the target detection points are acquired in real time and uploaded to a preset database along with processing information. In this way, by continuously acquiring multiple images, random errors from a single detection can be eliminated through statistical methods, ensuring the reliability of the indicator calculations.

[0063] Furthermore, each real-time captured target rendering image is input into the trained target detection model, which automatically outputs the bounding box coordinates of the template region. The image is then cropped based on these coordinates to obtain a target template region image that contains only the template region. All cropped images form a template region image set.

[0064] Then, for each image in the template region image set, the first similarity index is calculated with the target standard template rendering image. All indices are arranged in chronological order to form the first similarity index sequence. The arithmetic mean of the sequence is then calculated to obtain the template similarity index.

[0065] Simultaneously, all pairs of distinct and unique image combinations in the template region image set are traversed, and the second similarity index between each pair of images is calculated to form a second similarity index sequence. Then, the standard deviation of the sequence is calculated to obtain the stability index.

[0066] Finally, the calculated template similarity index is compared with the first preset threshold, and the stability index is compared with the second preset threshold: if the template similarity index is less than or equal to the first preset threshold, it indicates that the detection result deviates too much from the standard deviation, and the template and the detection point are mismatched. If the stability index is greater than or equal to the second preset threshold, it indicates that the continuous detection fluctuations are too large, and the equipment itself is unstable. If both indices are within the threshold range, the equipment is considered to be in normal condition.

[0067] In summary, this application provides a device status monitoring method. It establishes a preset database, which includes historical rendering images and standard template rendering images of multiple detection devices at multiple detection points. Next, multiple historical rendering images from the multiple detection devices are acquired, and a baseline model is trained to obtain a target detection model. Subsequently, multiple consecutive target rendering images generated by the target detection devices at target detection points are acquired in real time and stored in the preset database. Furthermore, based on the target detection model, a region of interest (ROI) detection is performed on each target rendering image, and the ROI is cropped to determine a template region image set, which includes multiple target template region images. Then, a first similarity index is calculated between each target template region image and the target standard template rendering image to determine a first similarity index sequence and a template similarity index corresponding to the first similarity index sequence. The target standard template rendering image is used to indicate the standard template rendering image of the target detection device at the target detection point. Simultaneously, a second similarity index is calculated between the target template region image and the corresponding historical template region image to determine a second similarity index sequence and a stability index corresponding to the second similarity index sequence. Finally, based on the comparison results of the template similarity index with the first preset threshold and / or the stability index with the second preset threshold, the equipment status of the target detection equipment is determined. This allows for the evaluation of equipment status through template similarity and / or stability indices, distinguishing between two types of anomalies: systematic template positioning offset and random self-positioning fluctuations. This enables automated and refined diagnosis of equipment positioning accuracy, providing clear optimization directions for equipment maintenance. Simultaneously, relying on unified quantitative evaluation standards and centralized database management, horizontal benchmarking of detection accuracy across factories and multiple machines can be achieved, providing reliable data for unifying detection quality standards throughout the factory. This ensures consistency in detection across different production lines and equipment, avoiding quality control loopholes caused by differences in equipment accuracy.

[0068] Please see Figure 2In some embodiments, step S013 includes: S0131: Based on the target detection equipment, visual positioning and photographing are performed at the target detection point to generate the original image; S0132: Perform template frame overlay processing on the original image to generate a target rendering image, and store the target rendering image in a preset database.

[0069] In some implementations, the processor is further configured to visually locate and photograph the target detection point based on the target detection device, generating an original image; and to perform template frame overlay processing on the original image to generate a target rendering image, and store the target rendering image in a preset database.

[0070] In some implementations, the acquisition module is further configured to visually locate and photograph the target detection point based on the target detection device, generating an original image; and to perform template frame overlay processing on the original image to generate a target rendering image, and store the target rendering image in a preset database.

[0071] Specifically, target inspection equipment refers to specific inspection equipment that is currently performing condition monitoring. It is equipped with a high-precision industrial camera and a motion control system, and can automatically complete the visual positioning, image acquisition and defect detection of products.

[0072] Target inspection points refer to the specific inspection locations currently monitored by the target inspection equipment, corresponding to a specific product model. Each inspection point is pre-configured with a standard template, inspection parameters, and template outline parameters.

[0073] Visual positioning refers to the process by which AOI equipment uses built-in image processing algorithms to identify pre-made positioning marks on the product surface, such as circular markers or crosshairs. This allows the equipment to calculate the precise coordinates and rotation angle of the product in the camera coordinate system, adjust the position of the camera or platform, and thus ensure that the product is centered in the camera's field of view and has the correct posture.

[0074] The raw image refers to the digital image of the product surface directly captured by an industrial camera after the AOI equipment has completed visual positioning, without any post-processing. It can usually include the actual appearance information of the product, defects generated during the production process, and environmental background information.

[0075] Template outlines refer to pre-defined rectangular or polygonal visual lines used to define the area to be inspected. They are usually made of high-contrast green to distinguish them from the product background, and their position, size, and shape are consistent with the standard template for the corresponding inspection point.

[0076] Template frame overlay processing refers to an image processing operation that uses the product coordinates obtained by visual positioning as a reference, and accurately overlays the pre-stored standard template frame lines corresponding to the detection points onto the corresponding coordinate positions of the original image according to preset size, color, and line width parameters. The overlaid frame lines can intuitively indicate the relative positional relationship between the theoretical detection area and the actual product image.

[0077] The target area to be inspected refers to the functional area on the product that needs to be inspected for defects and have its positioning accuracy assessed, such as pads and chip pins on a PCB board. The positioning accuracy of this area directly affects the inspection quality of the inspection equipment.

[0078] The target rendering image refers to a composite image generated by superimposing template frames on the original image according to preset parameters. It can retain the actual appearance information of the product and clearly mark the theoretical position of the area to be detected. It is the core data carrier used to evaluate the positioning accuracy of the device in the embodiments of this application.

[0079] First, the target inspection equipment performs visual positioning processing on the target inspection points. This involves identifying positioning marks on the product to determine its precise position and orientation, ensuring that the product's position and angle in each captured image are essentially consistent. After positioning is complete, the target inspection equipment takes a picture using an industrial camera, generating the original image.

[0080] Subsequently, the device calls the preset template outline parameters corresponding to the target detection point, precisely superimposing the template outline onto the corresponding position in the original image to generate the target rendering image. During the superimposition process, the position, size, color, and line thickness of the template outline strictly adhere to preset standards and cannot be arbitrarily modified, ensuring that all rendering images of the same detection point have a unified benchmark. After generation, the device automatically uploads the target rendering image to a preset database and establishes a correlation with the corresponding processing information for storage.

[0081] In this way, the generated target rendering image can simultaneously carry the shooting features of the actual component and the template positioning results of the detection equipment, intuitively reflecting the actual visual positioning status of the equipment at the target detection point. This ensures that the analysis object of the subsequent similarity calculation directly corresponds to the template positioning accuracy of the equipment, rather than simply the difference in image content, providing a unified and reliable image benchmark for subsequent calculations.

[0082] In some implementations, the preset database also stores processing information corresponding to each rendered image, including the detection equipment number, product model, shooting time, detection point number, and / or image storage path.

[0083] Specifically, processing information refers to structured metadata that describes the context in which the rendered image is generated. It serves as a bridge connecting unstructured image data with the actual production process and is used to identify the source, time, object, and storage location of each rendered image.

[0084] The detection device number refers to the identifier assigned to each AOI device, which is used to distinguish different physical devices in the database and is the core identifier for locating abnormal devices.

[0085] The product model refers to the model of the product being tested corresponding to the rendering image. Different product models have different testing points and standard templates, which serve as the basis for selecting the corresponding standard templates and testing parameters.

[0086] The shooting time refers to the precise timestamp generated by the rendered image, which is used to organize data in chronological order, analyze the trend of equipment performance changes, and trace the specific time when an anomaly occurred.

[0087] The inspection point number refers to the identifier assigned to each inspection location of each product. It is used to locate the specific inspection point corresponding to the rendering image and is the key to matching the corresponding standard template rendering image.

[0088] Image storage path refers to the physical storage address of the rendered image file in the database or file system. It is used to quickly locate and read the corresponding image file and supports manual review and algorithm recalculation.

[0089] In some implementations, during the data acquisition phase, when the AOI device generates a target rendering image, it automatically generates corresponding processing information simultaneously. This information includes the device ID that generated the rendering image, the model of the product being inspected, the current system time, the current inspection point ID, and the file path where the rendering image will be stored. Subsequently, the device packages the rendering image file and processing information together and uploads them to a preset database. The database stores the processing information in a structured data table and establishes a mapping relationship between the processing information and the storage address of the rendering image file, ensuring that each rendering image can be uniquely retrieved through its processing information.

[0090] In this way, the structured and indexed management of the rendered graph data can be achieved, providing clear and reliable data support for subsequent indicator calculation, status assessment and operation and maintenance traceability.

[0091] In some implementations, the baseline model includes a Backbone module, a Neck module, and a Head module; The Backbone module consists of a CBS convolutional block, a C3K2 module, an SPPF module, and a C2PSA module, and is used to extract basic features of the image. The Neck module includes an upsampling module, a feature concatenation module, a C3K2 module, and a CBS convolutional block, which are used to fuse feature maps of different scales; The Head module consists of a stack of CBS convolutional blocks, DSC modules, and Conv2d convolutional layers, and is used to output predicted bounding boxes, confidence scores, and category information.

[0092] Specifically, the baseline model refers to an initial neural network architecture with basic object detection capabilities. Through optimized training based on industrial rendering datasets, an object detection model specifically designed for template region detection can be obtained.

[0093] The Backbone module refers to the feature extraction backbone network of the baseline model, responsible for transforming the input rendered image into multi-scale abstract feature maps, and is the foundation of the entire model's performance. Specifically, the Backbone module performs feature downsampling based on CBS convolutional blocks, combines the cross-stage feature enhancement strategy of the C3K2 module to improve the richness of feature representation, uses the SPPF module to achieve multi-scale global feature aggregation, and further strengthens the weight ratio of key features with the channel and spatial joint attention mechanism of the C2PSA module. It can efficiently extract multi-dimensional visual features such as template outlines and component textures in the rendered image, and has strong anti-interference ability against common industrial scene lighting fluctuations, background noise, and slight imaging blur, ensuring the accuracy and robustness of feature extraction from template regions in template region images.

[0094] The Neck module is an intermediate layer connecting the Backbone and Head for feature fusion. It integrates feature maps of different resolutions output by the Backbone, ensuring that each scale's features simultaneously contain detailed information and global contextual information. Specifically, the Neck module achieves cross-scale fusion of shallow and deep features through upsampling and feature concatenation operations. Combined with the C3K2 module and CBS convolutional blocks, it further optimizes the quality of the fused features. This preserves edge contours and positional details in shallow features while incorporating the semantic discriminative capabilities of deep features. This effectively avoids feature loss in template regions due to size differences or positional shifts, adapting to the detection needs of rendering images with different detection points and field of view sizes, and improving the localization accuracy of template regions in various production scenarios.

[0095] The Head module refers to the final detection output layer of the model, responsible for transforming the fused feature maps into interpretable detection results, including the target's location, confidence level, and category. Specifically, the Head module employs a stacked structure of CBS convolutional blocks, DSC modules, and Conv2d convolutional layers. This reduces the number of computational parameters while ensuring prediction accuracy, efficiently outputting the predicted bounding boxes, confidence levels, and category information of the template region. The lightweight DSC convolutional design improves model inference speed, adapting to the deployment requirements of cloud servers processing large batches of rendered images in real time. It ensures the accurate location and complete edges of the cropped template region, avoiding the introduction of redundant background or omission of effective template regions due to cropping errors, thus laying a solid data foundation for the accurate calculation of subsequent similarity metrics.

[0096] CBS convolutional blocks refer to basic convolutional blocks composed of convolutional layers, batch normalization layers, and SiLU activation functions connected in series. They are the smallest functional modules for building the entire model and are used to extract local features of an image.

[0097] The C3K2 module refers to an improved module based on the CSP cross-stage enhancement structure. It adopts a multi-kernel size convolution and channel separation strategy to improve the extraction capability of complex features while reducing the amount of computation.

[0098] The SPPF module refers to an optimized version of the Spatial Pyramid Pooling module, which efficiently captures multi-scale global contextual information of an image by parallelizing multiple max pooling layers of different sizes.

[0099] The C2PSA module refers to an improved module that combines channel attention and spatial attention, introducing a multi-head attention mechanism that can adaptively focus on template region features and suppress background noise interference.

[0100] The upsampling module refers to the module used to enlarge low-resolution feature maps to the same size as high-resolution feature maps, thereby aligning and stitching features at different scales.

[0101] The feature stitching module refers to the module used to merge feature maps from different sources along the channel dimension, integrating multi-scale feature information.

[0102] The DSC module refers to the depthwise separable convolution module, which decomposes standard convolution into depthwise convolution and pointwise convolution, which can significantly reduce the amount of computation and parameters while maintaining accuracy.

[0103] Conv2d convolutional layers refer to standard two-dimensional convolutional layers, used to complete the final feature transformation and output detection results.

[0104] In this way, the architecture and core module composition of the benchmark model can be clearly defined, ensuring the feature extraction capability and detection accuracy of the target detection model for industrial rendering images. Among them, the multi-type convolution and attention modules of the backbone can accurately capture the visual features of the template region, the multi-scale feature fusion of the Neck can adapt to the detection needs of rendering images of different sizes and different fields of view, and the head module can output accurate bounding box localization results.

[0105] Please see Figure 3 In some embodiments, step S012 includes: S0121: A coordinate attention mechanism is embedded in the backbone network of the Backbone module in the baseline model, and a multi-scale detail preservation path is added to the feature fusion network in the Neck module. S0122: Based on the baseline model, perform region of interest detection on the historical rendering images and mark the first template region bounding box in each historical rendering image; S0123: Determine the bounding box loss, classification loss, and distribution focus loss based on the bounding box of the first template region and the bounding box of the second template region in the standard template rendering image; S0124: Based on preset weights, the bounding box loss, classification loss, and distribution focus loss are weighted and summed to determine the training loss function; S0125: Based on the training loss function, the baseline model is iteratively optimized to obtain the object detection model.

[0106] In some implementations, the processor is further configured to embed a coordinate attention mechanism into the backbone network of the Backbone module in the benchmark model, and to add a multi-scale detail-preserving path to the feature fusion network in the Neck module. Based on the benchmark model, it performs region of interest detection on historical rendering images, marking the first template region bounding box in each historical rendering image. Based on the first template region bounding box and the second template region bounding box in the standard template rendering image, it determines the bounding box loss, classification loss, and distribution focus loss. The processor is also configured to perform a weighted summation of the bounding box loss, classification loss, and distribution focus loss based on preset weights to determine the training loss function. Based on the training loss function, iteratively optimizes the benchmark model to obtain the object detection model.

[0107] In some implementations, the model training module is further used to embed a coordinate attention mechanism into the backbone network of the Backbone module in the benchmark model, and to add a multi-scale detail-preserving path to the feature fusion network in the Neck module. Based on the benchmark model, it performs region of interest detection on historical rendering images, marking the first template region bounding box in each historical rendering image. Based on the first template region bounding box and the second template region bounding box in the standard template rendering image, it determines the bounding box loss, classification loss, and distribution focus loss. The model training module is also used to perform a weighted summation of the bounding box loss, classification loss, and distribution focus loss based on preset weights to determine the training loss function. Based on the training loss function, iteratively optimizes the benchmark model to obtain the object detection model.

[0108] Specifically, coordinate attention mechanism refers to a lightweight attention mechanism that embeds spatial location information in the horizontal and vertical directions into channel attention. It can capture long-distance dependencies of feature maps in the width and height dimensions, enabling the model to accurately locate the spatial position of the target while suppressing irrelevant feature responses in the background region.

[0109] The multi-scale detail preservation path refers to the parallel feature transfer path added in the Neck module, which directly concatenates the low-level high-resolution feature map output by the Backbone with the high-level semantic feature map across layers. This avoids the loss of edge and texture details in the traditional upsampling and downsampling process, and thus effectively preserves the fine features of small-sized template outlines.

[0110] The first template region bounding box refers to the predicted bounding box output by the improved baseline model after preliminary detection of the historical rendering image, representing the model's initial estimate of the template region's location.

[0111] The second template region bounding box refers to the actual template region bounding box that has been labeled and verified in the standard template rendering image. It is the ground truth label in the model training process and is used to measure the accuracy of the predicted bounding box.

[0112] Bounding box loss refers to a loss function used to quantify the positional and dimensional differences between the predicted bounding box and the true bounding box. It can be used to optimize the prediction accuracy of the bounding box center point coordinates, width, and height.

[0113] Classification loss refers to the loss function used to quantify the accuracy of the model's prediction of the target category. In the embodiments of this application, it mainly distinguishes between two categories: template region and background region.

[0114] Distribution focus loss refers to an improved cross-entropy loss function that automatically reduces the weight of easily classified samples and increases the contribution of difficult samples during training by introducing a balance factor and a focus parameter, effectively solving the problems of imbalance between positive and negative samples and insufficient learning of difficult samples.

[0115] Preset weights refer to the weight coefficients of the three loss functions that are pre-defined according to the characteristics of industrial inspection tasks, used to balance the impact of different loss terms on model training. Since localization accuracy is usually the core requirement of inspection tasks, the bounding box loss has the highest weight.

[0116] The training loss function is an objective function used to quantify the degree of error between the predicted results output by the object detection model and the real labels annotated by humans. It is the core basis for driving the iterative optimization of model parameters.

[0117] Iterative optimization refers to the process of gradually updating the model's weights and biases by calculating the gradient of the training loss function with respect to the model parameters based on the backpropagation algorithm, so that the training loss is continuously reduced until the model converges.

[0118] Please see Figure 4 In some implementations, the structure of the target detection model is as follows: Figure 4 As shown. The object detection model is generally divided into three main components: the Backbone feature extraction network, the Neck multi-scale feature fusion network, and the Head detection output network. The optimization and training process of the object detection model can be as follows: First, targeted structural optimization is performed on the architecture of the basic detection model.

[0119] Specifically, for the Backbone backbone network, a C2PSA attention module is introduced into the deep feature branches based on multi-level downsampling convolutional stacking. This C2PSA attention module combines channel attention and spatial multi-head attention mechanisms, which can adaptively strengthen the key feature weights of the template bounding box during the feature extraction stage and suppress background noise interference. At the same time, it is paired with an SPPF pooling module to complete multi-scale global feature aggregation, further improving the expressive power of deep features. The Backbone as a whole is composed of CBS convolutional blocks, C3K2 feature enhancement modules, SPPF pooling modules, and C2PSA attention modules connected in series. After multi-level downsampling of the input image, the final output feature maps of three scales, 64×80×80, 128×40×40, and 256×20×20, are sent to the Neck module.

[0120] For the Neck feature fusion network, a bidirectional multi-scale fusion path is constructed as a multi-scale detail-preserving path, employing both top-down upsampling and bottom-up sampling. The Upsample unit progressively enlarges the 256×20×20 deep semantic features, concatenating them with the corresponding scale shallow features output from the Backbone via a Contact unit in the channel dimension, followed by feature optimization using the C3K2 module. Simultaneously, CBS convolution is used for downsampling, progressively reducing the shallow detail features, which are then concatenated and fused again with the corresponding scale features output from the upsampled branch. The Neck module ultimately outputs fused feature maps at three scales: 64×80×80, 128×40×40, and 256×20×20, which are then input into the Head module.

[0121] For the Head detection output network, a set of parallel dual-branch detection heads is matched to the feature maps at three scales: one path uses two-stage CBS convolutional blocks connected in series and then connected to a Conv2d convolutional layer to ensure detection and localization accuracy. The other path uses DSC modules connected in series and then connected to a Conv2d convolutional layer to reduce the number of computational parameters in the model.

[0122] Next, historical rendering images covering different devices, locations, lighting conditions, and product models are obtained from a pre-set database as a training dataset. These images are then input into the improved baseline model. The model performs region of interest detection on each historical rendering image and outputs the corresponding first template region bounding box, confidence score, and category information.

[0123] Subsequently, the bounding box of the first template region output by the model is compared pixel by pixel with the bounding box of the second template region in the corresponding standard template rendering image, and the bounding box loss, classification loss and distribution focus loss are calculated respectively.

[0124] Then, based on preset weights, the bounding box loss, classification loss, and distribution focus loss are weighted and summed to determine the training loss function.

[0125] Finally, based on the training loss function, backpropagation is performed using stochastic gradient descent or Adam optimization algorithms to progressively update all trainable parameters of the model. In some implementations, after each training round, the mean accuracy and bounding box localization error of the model are evaluated using a validation set. When the validation set performance no longer improves for several consecutive rounds, the model is considered to have converged, training is stopped, and the final object detection model is saved.

[0126] In this way, by optimizing the network architecture and subjecting it to the joint constraints of multi-dimensional loss functions, the localization accuracy, scene robustness, and industrial adaptability of the object detection model for the rendered image template region can be improved, thereby providing stable and reliable model support for the accurate cropping of the region of interest in the future.

[0127] Please see Figure 5 In some implementations, step S015 includes: S0151: Based on the first preset sliding window, the target template region image and the target standard template rendering image are calculated block by block to obtain the first local similarity value of the corresponding sliding window; S0152: Calculate the average of the first local similarity values ​​of all sliding windows to determine the first similarity index between the currently processed template region image and the target standard template rendering image; S0153: Determine the first similarity index sequence based on the first similarity index; S0154: Calculate the arithmetic mean of all first similarity indices in the first similarity index sequence to determine the template similarity index.

[0128] In some embodiments, the processor is further configured to perform block-by-block calculations on the currently processed target template region image and target standard template rendering image based on a first preset sliding window to obtain a first local similarity value for the corresponding sliding window; and calculate the average of the first local similarity values ​​of all sliding windows to determine a first similarity index between the currently processed template region image and the target standard template rendering image; and determine a first similarity index sequence based on the first similarity index, calculate the arithmetic mean of all first similarity indices in the first similarity index sequence, and determine a template similarity index.

[0129] In some implementations, the calculation module is further configured to perform block-by-block calculations on the currently processed target template region image and the target standard template rendering image based on a first preset sliding window to obtain a first local similarity value for the corresponding sliding window; and to calculate the average of the first local similarity values ​​of all sliding windows to determine a first similarity index between the currently processed template region image and the target standard template rendering image; and to determine a first similarity index sequence based on the first similarity index, and to calculate the arithmetic mean of all first similarity indices in the first similarity index sequence to determine a template similarity index.

[0130] Specifically, the first similarity index refers to the overall structural similarity quantification value between a single target template region image and the corresponding target standard template rendering image. The value range is usually from 0 to 1, and the closer the value is to 1, the higher the similarity between the two images.

[0131] The first preset sliding window refers to a pre-set rectangular window used for scanning images block by block. Its size can be adjusted according to the size of the template area and the detection accuracy requirements to achieve full coverage scanning with no overlap or partial overlap.

[0132] The first local similarity value refers to the quantified similarity between the target template region image and the standard template rendering image within the local sub-block covered by the first preset sliding window, reflecting the degree of matching of local regions of the image.

[0133] The first similarity index sequence refers to an ordered sequence formed by arranging the first similarity indices corresponding to all target template region images in the template region image set according to the chronological order of shooting time, which can reflect the changing trend of template similarity over time.

[0134] The single target template region image obtained by cropping through the target detection model is aligned with the corresponding target standard template rendering image retrieved from the preset database. Then, starting from the top left corner, the two aligned images are scanned row by row and column by column with a fixed step size using a first preset sliding window. The similarity of the local sub-blocks covered by each sliding window is calculated to obtain the first local similarity value at the corresponding position.

[0135] Next, the first local similarity values ​​corresponding to all sliding windows are traversed, and their arithmetic mean is calculated. This average value is used as the first similarity index between the current target template region image and the standard template rendering image. In this way, by averaging all local similarity values, the overall matching degree of the image can be comprehensively evaluated, while smoothing out the influence of minor local noise to a certain extent, balancing detection sensitivity and anti-interference ability.

[0136] Subsequently, according to the shooting time sequence of the target rendering image, the first similarity indices corresponding to all images in the template region image set are arranged sequentially to form the first similarity index sequence.

[0137] Finally, the arithmetic mean of all the values ​​in the first similarity index sequence is calculated to obtain the final template similarity index.

[0138] Thus, by calculating block by block using a sliding window and then averaging the results, the interference of local noise and minor artifacts on the similarity calculation results can be effectively reduced, improving the stability and reliability of the similarity index. Furthermore, obtaining the final template similarity index by taking the arithmetic mean of the similarity indices from multiple consecutive images can further offset the impact of random fluctuations in a single detection, more accurately reflecting the degree of systematic deviation between the device's detection results and the standard template, and avoiding misjudgments caused by anomalies in a single image.

[0139] Please see Figure 6 In some embodiments, step S0151 includes: S01511: Based on the first preset sliding window, calculate the first local mean, the first local variance, and the first local covariance between the currently processed target template region image and the target standard template rendering image; S01512: Calculate the first luminance component based on the first local mean; S01513: Calculate the first contrast component based on the first local variance; S01514: Calculate the first structural component based on the first local covariance; S01515: The first luminance component, the first contrast component, and the first structure component are weighted and fused to obtain the first local similarity value.

[0140] In some implementations, the processor is further configured to calculate, based on a first preset sliding window, a first local mean, a first local variance, and a first local covariance between the currently processed target template region image and the target standard template rendering image; calculate a first luminance component based on the first local mean; and calculate a first contrast component based on the first local variance. The processor is further configured to calculate a first structure component based on the first local covariance; and perform weighted fusion of the first luminance component, the first contrast component, and the first structure component to obtain a first local similarity value.

[0141] In some implementations, the calculation module is further configured to calculate, based on a first preset sliding window, a first local mean, a first local variance, and a first local covariance between the currently processed target template region image and the target standard template rendering image; calculate a first luminance component based on the first local mean; and calculate a first contrast component based on the first local variance. The calculation module is also configured to calculate a first structural component based on the first local covariance; and perform weighted fusion of the first luminance component, the first contrast component, and the first structural component to obtain a first local similarity value.

[0142] Specifically, the first local mean refers to the arithmetic mean of the gray values ​​of all pixels in a single image within the local area covered by the first preset sliding window, which can reflect the overall brightness level of the local area.

[0143] The first local variance refers to the degree of dispersion of the gray values ​​of all pixels in a single image relative to its local mean within the local area covered by the first preset sliding window, which can reflect the contrast intensity of the local area.

[0144] The first local covariance refers to the degree of correlation between the corresponding pixel grayscale values ​​of the target template region image and the standard template rendering image within the local area covered by the same sliding window. It can reflect the similarity of the pixel spatial structure of the two images in this local area.

[0145] The first luminance component refers to the quantized value calculated based on the first local mean of two images. It can measure the overall brightness difference between the two images in this local area. The value range is usually from 0 to 1, and the closer it is to 1, the more consistent the brightness is.

[0146] The first contrast component refers to the quantized value calculated based on the first local variance of two images. It can measure the difference in brightness and darkness between the two images in this local area. The value range is usually from 0 to 1. The closer it is to 1, the more consistent the contrast is.

[0147] The first structural component refers to the quantized value calculated based on the first local covariance and the first local variance of the two images. It can measure the difference in pixel spatial structure between the two images in this local region. The value range is usually from 0 to 1. The closer it is to 1, the more consistent the structure is.

[0148] Weighted fusion refers to the process of assigning a preset weight coefficient to each of the three visual components based on their contribution to image similarity, and then multiplying or summing the weighted component values ​​to obtain a comprehensive local similarity value.

[0149] In some implementations, the calculation process for the first local similarity value can be as follows: For the pixel area covered by the current first preset sliding window, extract all pixel grayscale values ​​of the target template region image and the target standard template rendering image respectively, and calculate the first local mean, the first local variance, and the first local covariance between the two images in turn.

[0150] Subsequently, the two second local means are substituted into the luminance component calculation formula: ; in, The local mean of the target template region image. The local mean of the target standard template rendering image. is the luminance constant.

[0151] Furthermore, the two second local variances are substituted into the formula for calculating the contrast component: ; in, The local variance of the target template region image. The local variance of the target standard template rendering image. is the contrast constant.

[0152] Meanwhile, the second local covariance and the two local variances are substituted into the structural component calculation formula: ; in, The local covariance of the two images. is a structure constant.

[0153] Finally, the first luminance component, the first contrast component, and the first structure component are weighted and fused according to preset weighting coefficients to obtain the final first local similarity value.

[0154] In this way, by comprehensively calculating image similarity from three dimensions of brightness, contrast, and structure, the calculation logic of local similarity values ​​can be made more in line with the laws of human visual perception and more suitable for the imaging characteristics of industrial inspection scenarios. This can effectively avoid the defects of traditional pixel-level error algorithms that are easily affected by light fluctuations and random noise interference, thus misjudging anomalies.

[0155] Please see Figure 7 In some implementations, step S016 includes: S0161: Based on the second preset sliding window, calculate the second local similarity value of the target template region image and the corresponding historical template region image to obtain the second local similarity value of the corresponding sliding window; S0162: Calculate the average of the second local similarity values ​​of all sliding windows to determine the second similarity index between the current template region image and the currently processed historical template region image; S0163: Determine the second similarity index sequence based on the second similarity index; S0164: Calculate the dispersion statistics of all second similarity indices in the second similarity index sequence to determine the stability index.

[0156] In some embodiments, the processor is further configured to calculate, based on a second preset sliding window, a second local similarity value for the target template region image and the corresponding historical template region image for the corresponding sliding window; and to calculate the average of the second local similarity values ​​of all sliding windows to determine a second similarity index between the current template region image and the currently processed historical template region image. The processor is further configured to determine a second similarity index sequence based on the second similarity index; and to calculate the dispersion statistics of all second similarity indices in the second similarity index sequence to determine a stability index.

[0157] In some implementations, the calculation module is further configured to calculate, based on a second preset sliding window, the target template region image and the corresponding historical template region image to obtain a second local similarity value for the corresponding sliding window. It also calculates the average of the second local similarity values ​​for all sliding windows to determine a second similarity index between the current template region image and the currently processed historical template region image. The calculation module is further configured to determine a second similarity index sequence based on the second similarity index. Finally, it calculates the dispersion statistics of all second similarity indices in the second similarity index sequence to determine a stability index.

[0158] Specifically, the corresponding historical template region image refers to the template region image obtained by cropping the ROI of a historical rendering image that comes from the same detection device and the same detection point as the current target template region image and is located before the current image in time sequence. In some implementations, the corresponding historical template region image is usually selected from the previous several frames in a continuous time sequence and compared with the current image one by one to ensure the temporal correlation and consistency of the comparison.

[0159] The second preset sliding window refers to a rectangular calculation window with a pre-defined size and sliding step size, used to traverse the two images to be compared block by block, realizing fine-grained similarity calculation of local regions. The window size of the second preset sliding window can be flexibly adjusted according to the size of the template area and the detection accuracy requirements.

[0160] The second local similarity value refers to the quantified value of the similarity between the target template region image and the corresponding historical template region image within the coverage area of ​​a single sliding window. It can reflect the local region's positioning consistency and is the basic unit for calculating the global similarity index.

[0161] The second similarity index refers to the overall similarity quantification result of a single set of compared images. It is obtained by taking the arithmetic mean of the second local similarity values ​​of all sliding windows. The value ranges from 0 to 1. The closer the value is to 1, the higher the overall consistency between the two images.

[0162] The second similarity index sequence refers to an ordered dataset formed by arranging multiple sets of consecutive time-series second similarity indices in chronological order of collection time, which fully records the similarity fluctuations of device detection results over a period of time.

[0163] The dispersion statistic refers to a statistical indicator used to measure the degree of dispersion of a set of data relative to the mean. In the embodiments of this application, the standard deviation can be used as the core statistic. The larger the value, the more drastic the data fluctuation, and the worse the positioning stability of the corresponding device.

[0164] First, select the target template region image at the current moment from the template region image set, and match it with the corresponding historical template region images of the same device and the same location to form a single comparison group.

[0165] Next, a second preset sliding window is used to traverse the two images to be compared row by row and column by column with a fixed step size. The similarity is calculated for the local area covered by each window to obtain the second local similarity value at the corresponding position.

[0166] Subsequently, the second local similarity values ​​corresponding to all sliding windows are traversed, and their arithmetic mean is calculated as the second similarity index of the current group of paired images.

[0167] Then, the second similarity indices obtained from multiple consecutive comparisons are arranged in chronological order to form a second similarity index sequence. The dispersion statistic of this sequence is then calculated to obtain the stability index.

[0168] Thus, by using a statistical method of pairwise comparison of the entire sample and a quantitative logic of dispersion, it is possible to characterize the fluctuation characteristics of continuous detection results at the same location of the same device, objectively reflect the positioning stability of the device itself, and provide an independent quantitative evaluation dimension for distinguishing between random jitter anomalies of the device and systematic offset anomalies of the template.

[0169] Please see Figure 8 In some embodiments, step S0161 includes: S01611: Based on the second preset sliding window, calculate the second local mean, second local variance and second local covariance of the target template region image and the corresponding historical template region image; S01612: Calculate the second luminance component based on the second local mean; S01613: Calculate the second contrast component based on the second local variance; S01614: Calculate the second structural component based on the second local covariance; S01615: The second luminance component, the second contrast component, and the second structure component are weighted and fused to obtain the second local similarity value.

[0170] In some embodiments, the processor is further configured to calculate, based on a second preset sliding window, a second local mean, a second local variance, and a second local covariance of the target template region image and the corresponding historical template region image; and to calculate a second luminance component based on the second local mean; and to calculate a second contrast component based on the second local variance. The processor is also configured to calculate a second structural component based on the second local covariance; and to perform weighted fusion of the second luminance component, the second contrast component, and the second structural component to obtain a second local similarity value.

[0171] In some implementations, the calculation module is further configured to calculate, based on a second preset sliding window, a second local mean, a second local variance, and a second local covariance of the target template region image and the corresponding historical template region image; calculate a second luminance component based on the second local mean; and calculate a second contrast component based on the second local variance. The calculation module is also configured to calculate a second structural component based on the second local covariance; and weightedly fuse the second luminance component, the second contrast component, and the second structural component to obtain a second local similarity value.

[0172] Specifically, the second local mean refers to the arithmetic mean of the gray values ​​of all pixels in a single image within the pixel range covered by the second preset sliding window. It can reflect the overall brightness level of the local area and is mainly affected by environmental factors such as light intensity and lens transmittance.

[0173] The second local variance refers to the average of the squared deviations of the gray values ​​of all pixels in a single image relative to their local mean within the pixel range covered by the second preset sliding window. It can measure the dispersion of pixel gray values ​​in that area and reflect the contrast intensity of local areas. It is mainly affected by factors such as product surface texture and lens sharpness.

[0174] The second local covariance refers to the joint dispersion of the corresponding pixel gray values ​​of two images to be compared within the same sliding window coverage area. It can quantify the degree of linear correlation between two variables, directly reflect the consistency of the pixel spatial arrangement structure of the two images, and is a statistical feature that is strongly correlated with the positioning accuracy of the device.

[0175] The second brightness component refers to the normalized similarity component calculated based on the second local mean of the target template region image and the corresponding historical template region image. The value range can be from 0 to 1. The closer the value is to 1, the smaller the local overall brightness difference between the two images. It is used to measure the consistency at the brightness level.

[0176] The second contrast component refers to the normalized similarity component calculated based on the second local variance of the target template region image and the corresponding historical template region image. Its value ranges from 0 to 1. The closer the value is to 1, the smaller the difference in local brightness and darkness intensity between the two images. It is used to measure the consistency at the contrast level.

[0177] The second structural component refers to the normalized similarity component calculated based on the second local covariance and the second local variance of the target template region image and the corresponding historical template region image. Its value range can be 0 to 1, and it is completely unaffected by changes in brightness and contrast. It only reflects the similarity of the pixel spatial arrangement structure and is a component that measures the device positioning offset.

[0178] Weighted fusion refers to the process of pre-assigning corresponding weight coefficients based on the importance of the second luminance component, the second contrast component, and the second structure component to the evaluation of device stability, and then multiplying the three components according to their weights to obtain a comprehensive local similarity value.

[0179] In some implementations, the calculation process for the second local similarity value can refer to the aforementioned calculation process for the first local similarity value, and will not be repeated here.

[0180] In this way, by comprehensively calculating image similarity from three dimensions of brightness, contrast, and structure, the calculation logic of local similarity values ​​can be made more in line with the laws of human visual perception and more suitable for the imaging characteristics of industrial inspection scenarios. This can effectively avoid the defects of traditional pixel-level error algorithms that are easily affected by light fluctuations and random noise interference, thus misjudging anomalies.

[0181] Please see Figure 9 In some implementations, step S017 includes: S0171: If the template similarity index is less than or equal to the first preset threshold, it is determined that the target standard template rendering image does not match the target detection point; and / or S0172: If the stability index is greater than or equal to the second preset threshold, the target detection device is determined to be unstable in its positioning.

[0182] In some implementations, the processor is further configured to determine that the target standard template rendering image and the target detection point do not match if the template similarity index is less than or equal to a first preset threshold. And / or, if the stability index is greater than or equal to a second preset threshold, determine that the target detection device itself is unstable in positioning.

[0183] In some implementations, the determining module is further configured to determine that the target standard template rendering image and the target detection point do not match if the template similarity index is less than or equal to a first preset threshold. And / or, if the stability index is greater than or equal to a second preset threshold, determine that the target detection device itself is unstable in positioning.

[0184] Specifically, the first preset threshold refers to the template similarity threshold obtained based on a large amount of historical data under normal operating conditions. It is the standard for determining whether the standard template rendering image matches the detection point. In some implementations, this first preset threshold is usually set according to product accuracy requirements, process specifications, and historical misjudgment rate, and its value range can be from 0 to 1.

[0185] The second preset threshold refers to a stability critical value obtained based on statistical analysis of historical data under a large amount of normal operating conditions. It serves as a standard for determining whether the device's positioning is stable. In some implementations, this second preset threshold is typically set according to product accuracy requirements and device characteristics, and its value can range from 0 to 1.

[0186] A mismatch between the target standard template rendering and the target detection points indicates a systematic deviation in the template positioning parameters, resulting in the overall similarity between the equipment's multiple consecutive detection results and the standard template being below an acceptable level. Common causes may include incorrect template parameter settings, template drift, and failure to update the template after a product batch change.

[0187] Unstable positioning of the target detection equipment refers to a malfunction in the equipment's mechanical or optical system, causing fluctuations in continuous detection results that exceed acceptable levels. Common causes may include guide rail wear, excessive lead screw backlash, increased motor vibration, and loose lenses.

[0188] The template similarity index calculated in real time is compared with a pre-set first threshold. If the template similarity index is less than or equal to the first threshold, it means that the overall deviation between the results of multiple consecutive detections and the standard template rendering has exceeded the allowable range of the process. This deviation is due to the mismatch between the standard template rendering and the detection points.

[0189] Simultaneously, the stability index calculated in real time is compared with a pre-set second threshold. If the stability index is greater than or equal to the second threshold, it indicates that the fluctuation range of the continuous detection results has exceeded the acceptable level. Such fluctuations are usually caused by mechanical or optical system failures of the equipment itself.

[0190] Thus, by using an independent threshold judgment mechanism based on dual-dimensional indicators, it is possible to accurately distinguish between two different root causes: template positioning adaptability anomalies and equipment performance anomalies. This provides clear fault indications for equipment operation and maintenance, improving the accuracy of equipment fault diagnosis and the efficiency of operation and maintenance.

[0191] Please see Figure 10 In some implementations, the method further includes: S018: An alarm message is generated when the target detection equipment itself is unstable or the target standard template rendering does not match the target detection point.

[0192] In some implementations, the processor is also used to generate alarm information when the target detection device itself is unstable or when the target standard template rendering does not match the target detection point.

[0193] In some implementations, the equipment detection device further includes an alarm module, which generates alarm information when the target detection device itself is unstable or when the target standard template rendering does not match the target detection point.

[0194] Specifically, alarm information refers to structured prompt data automatically generated by the system after detecting equipment anomalies. It can be displayed on the terminal interface, pushed to the operation and maintenance backend, or stored in local logs, serving as an intermediary between anomaly determination and manual repair. In some implementations, alarm information typically includes the anomaly type, the time of anomaly occurrence, the identifier of the target detection device, and the identifier of the target detection point.

[0195] Among them, the anomaly type refers to the field for classifying and labeling equipment failures. It is mainly divided into two categories: mismatch between the standard template and the detection point, and unstable positioning of the detection equipment itself. If both types of anomalies occur at the same time, they are labeled as compound anomalies.

[0196] The time of anomaly occurrence refers to the precise timestamp at which the system determines that an anomaly has occurred. It is used to record the point at which the fault occurred and to support post-event traceability and trend analysis.

[0197] The identification of target inspection equipment refers to the code, number or equipment name assigned to each AOI device, which is used to accurately distinguish faulty equipment in a multi-equipment factory area and is the core field for equipment identification.

[0198] The identification of target inspection points refers to the number assigned to different inspection stations within a single piece of equipment. This is used to pinpoint the specific inspection location where an anomaly has occurred on the equipment, avoiding a comprehensive inspection of the entire machine. In this way, alarm information includes the time of the anomaly, the target inspection equipment identifier, and the inspection point identifier, enabling precise location of the anomaly. In large-scale production line scenarios with multiple factories, multiple machines, and multiple inspection points, this helps maintenance personnel quickly locate the specific workstation where the anomaly occurred, allowing for accurate on-site handling without needing to check each machine individually. This is suitable for large-scale manufacturing scenarios with multiple production lines operating in parallel, improving anomaly response speed.

[0199] In this way, the proactive alarm function can notify maintenance personnel as soon as the equipment accuracy is determined to be abnormal, which can significantly shorten the response time to abnormalities, avoid batch product defects caused by continuous abnormal operation, and effectively reduce product quality risks and production losses.

[0200] Please see Figure 11 In some implementations, the method further includes: S019: Continuously acquire stability indicators of the target detection equipment at the target detection point over multiple consecutive time windows to form stability trend data; S020: If the rate of change of stability trend data is greater than or equal to the third preset threshold within a preset time period, an early warning message will be generated.

[0201] In some implementations, the processor is also used to continuously acquire stability indicators of the target detection device at multiple consecutive time windows at the target detection point, forming stability trend data. And if the rate of change of the stability trend data within a preset time period is greater than or equal to a third preset threshold, an early warning message is generated.

[0202] In some implementations, the alarm module is also used to continuously acquire stability indicators of the target detection device at the target detection point over multiple consecutive time windows, forming stability trend data. And if the rate of change of the stability trend data within a preset time period is greater than or equal to a third preset threshold, an early warning message is generated.

[0203] Specifically, a time window refers to a fixed-duration statistical unit predefined by the system. It can be set and adjusted according to the production cycle and is the basic unit for time-series data collection and statistics. It is used for periodic summarization and recording of equipment stability indicators.

[0204] Stability trend data refers to a time-series dataset formed by arranging stability indicators collected from multiple consecutive time windows in chronological order, which intuitively presents the change process of equipment positioning stability over time.

[0205] The rate of change refers to the speed at which a stability indicator increases or decreases within a preset time period. The magnitude of the value represents the speed at which the equipment performance deteriorates, and the higher the value of the stability indicator, the greater the fluctuation of the equipment.

[0206] The third preset threshold refers to the critical value of the rate of change obtained by statistical analysis based on the normal aging pattern of equipment, historical fault data, and process precision requirements. It is the standard for judging whether the equipment is at risk of rapid deterioration.

[0207] Early warning information refers to proactive alerts, distinct from fault alarms, used to inform maintenance personnel of potential equipment failure risks that have not yet met the hard failure criteria, but require early intervention for inspection and maintenance. In some implementations, early warning information includes the abnormal warning time, the identifier of the target detection device, and / or the identifier of the target detection point.

[0208] Among them, the abnormal warning time refers to the precise timestamp when the system determines that the rate of change exceeds the standard and triggers the warning. It is used to mark the node where the risk occurs and to provide a time basis for subsequent fault tracing and trend review.

[0209] The identification of target inspection equipment refers to the unique code corresponding to each AOI device, which is used to accurately locate equipment with potential risks in a multi-equipment factory area.

[0210] The identification of target detection points refers to the unique number of each detection station within a single device, used to pinpoint the specific detection location where performance degradation has occurred on the device.

[0211] Thus, by using the trend warning function based on the rate of change of stability indicators, it is possible to make advance predictions of equipment accuracy anomalies. That is, it is possible to issue early warnings when the equipment performance shows a trend of decline but before defects occur, so as to guide maintenance personnel to carry out preventive maintenance in a timely manner.

[0212] This application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it implements the steps of the method described above.

[0213] It is understood that a computer program includes computer program code. Computer program code can be in the form of source code, object code, executable files, or some intermediate form. Computer-readable storage media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media, etc.

[0214] This application also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the above-described method.

[0215] In this specification, the terms "specifically," "furthermore," "particularly," "understandably," etc., refer to specific features, structures, materials, or characteristics described in connection with embodiments or examples that are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0216] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of executable request code comprising one or more steps for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order according to the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0217] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method for monitoring equipment condition, characterized in that, The method includes: Establish a preset database, which includes historical rendering images and standard template rendering images of multiple detection devices at multiple detection points; Multiple historical rendering images from various detection devices are acquired, and a benchmark model is trained to obtain a target detection model. Multiple consecutive target rendering images generated by the target detection device at the target detection point are acquired in real time and stored in the preset database; Based on the target detection model, region of interest detection is performed on each of the target rendering images, and the region of interest is cropped to determine a template region image set, wherein the template region image set includes multiple target template region images; Calculate the first similarity index for each target template region image and the target standard template rendering image to determine the first similarity index sequence and the template similarity index corresponding to the first similarity index sequence. The target standard template rendering image is used to indicate the standard template rendering image of the target detection device at the target detection point. Calculate a second similarity index between the target template region image and the corresponding historical template region image to determine the second similarity index sequence and the stability index corresponding to the second similarity index sequence; The device status of the target detection device is determined based on the comparison result of the template similarity index with the first preset threshold and / or the comparison result of the stability index with the second preset threshold.

2. The method according to claim 1, characterized in that, The real-time acquisition of multiple consecutive target rendering images generated by the target detection device at the target detection point and storage in the preset database includes: Based on the target detection device, visual positioning and photography are performed at the target detection point to generate the original image; The original image is subjected to template frame overlay processing to generate the target rendering image, and the target rendering image is stored in the preset database. The template frame is used to define the target area to be detected corresponding to the target detection point.

3. The method according to claim 1, characterized in that, The preset database also stores processing information corresponding to each of the rendered images, including the detection equipment number, product model, shooting time, detection point number, and / or image storage path.

4. The method according to claim 1, characterized in that, The baseline model includes a Backbone module, a Neck module, and a Head module; The Backbone module consists of a CBS convolutional block, a C3K2 module, an SPPF module, and a C2PSA module, and is used to extract basic features of the image; The Neck module includes an upsampling module, a feature concatenation module, a C3K2 module, and a CBS convolutional block, used to fuse feature maps of different scales; The Head module consists of a stack of CBS convolutional blocks, DSC modules, and Conv2d convolutional layers, and is used to output predicted bounding boxes, confidence scores, and category information.

5. The method according to claim 4, characterized in that, The step of acquiring multiple historical rendering images from multiple detection devices and training a benchmark model to obtain a target detection model includes: In the baseline model, a coordinate attention mechanism is embedded in the backbone network of the Backbone module, and a multi-scale detail preservation path is added to the feature fusion network in the Neck module; Based on the baseline model, region of interest detection is performed on the historical rendering images, and the first template region bounding box is marked in each of the historical rendering images; Based on the bounding box of the first template region and the bounding box of the second template region in the standard template rendering image, determine the bounding box loss, classification loss, and distribution focus loss; Based on preset weights, the bounding box loss, the classification loss, and the distribution focus loss are weighted and summed to determine the training loss function; Based on the training loss function, the baseline model is iteratively optimized to obtain the target detection model.

6. The method according to claim 1, characterized in that, The step of calculating a first similarity index for each of the target template region images and the target standard template rendering image to determine a first similarity index sequence and a template similarity index corresponding to the first similarity index sequence includes: Based on the first preset sliding window, the target template region image and the target standard template rendering image are calculated block by block to obtain the first local similarity value of the corresponding sliding window; Calculate the average of the first local similarity values ​​of all sliding windows to determine the first similarity index between the currently processed template region image and the target standard template rendering image; Based on the first similarity index, determine the first similarity index sequence; Calculate the arithmetic mean of all first similarity indices in the first similarity index sequence to determine the template similarity index.

7. The method according to claim 6, characterized in that, The step of calculating the first local similarity value of the corresponding sliding window by block-by-block calculation of the currently processed target template region image and the target standard template rendering image includes: Based on the first preset sliding window, calculate the first local mean, the first local variance, and the first local covariance between the currently processed target template region image and the target standard template rendering image; Calculate the first luminance component based on the first local mean; Calculate the first contrast component based on the first local variance; Calculate the first structural component based on the first local covariance; The first luminance component, the first contrast component, and the first structure component are weighted and fused to obtain the first local similarity value.

8. The method according to claim 1, characterized in that, The step of calculating a second similarity index between the target template region image and the corresponding historical template region image to determine a second similarity index sequence and a stability index corresponding to the second similarity index sequence includes: Based on the second preset sliding window, the target template region image and the corresponding historical template region image are calculated to obtain the second local similarity value of the corresponding sliding window; Calculate the average of the second local similarity values ​​of all sliding windows to determine the second similarity index between the current template region image and the currently processed historical template region image; Based on the second similarity index, determine the second similarity index sequence; Calculate the dispersion statistics of all second similarity indices in the second similarity index sequence to determine the stability index.

9. The method according to claim 8, characterized in that, The step of calculating the second local similarity value of the corresponding sliding window based on the target template region image and the corresponding historical template region image includes: Based on the second preset sliding window, calculate the second local mean, second local variance and second local covariance of the target template region image and the corresponding historical template region image; The second luminance component is calculated based on the second local mean. The second contrast component is calculated based on the second local variance; The second structural component is calculated based on the second local covariance; The second luminance component, the second contrast component, and the second structure component are weighted and fused to obtain the second local similarity value.

10. The method according to claim 1, characterized in that, The step of determining the device status of the target detection device based on the comparison result of the template similarity index with a first preset threshold and / or the comparison result of the stability index with a second preset threshold includes: If the template similarity index is less than or equal to the first preset threshold, it is determined that the target standard template rendering image does not match the target detection point; and / or If the stability index is greater than or equal to the second preset threshold, the target detection device is determined to be unstable in its positioning.

11. The method according to claim 10, characterized in that, The method further includes: If the target detection device itself is unstable or the target standard template rendering does not match the target detection point, an alarm message is generated. The alarm message includes the anomaly type, the time of the anomaly, the identifier of the target detection device and / or the identifier of the target detection point.

12. The method according to claim 1, characterized in that, The method further includes: The stability index of the target detection device at the target detection point is continuously acquired over multiple consecutive time windows to form stability trend data; If the rate of change of the stability trend data within a preset time period is greater than or equal to a third preset threshold, an early warning message is generated. The early warning message includes the abnormal warning time, the identifier of the target detection device, and / or the identifier of the target detection point.

13. A device testing system, characterized in that, The device detection system includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method as described in any one of claims 1-12.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1-12.

Images