Computer startup and shutdown testing and abnormity monitoring method and system based on image recognition

Abstract

The invention relates to the technical field of computer startup and shutdown testing, in particular to a computer startup and shutdown testing and abnormity monitoring method and system based on image recognition, and the method comprises the steps: transmitting an instruction to a jig interface of a startup and shutdown testing jig through a server, the MCU microcontroller controls the relay group to simulate the action of manually pressing the on-off button, so that the on and off of a computer power supply are realized; after the power supply is turned on or turned off, the server monitors the power supply state of the tested computer through ping operation and a signal line; images of a tested computer screen are collected in real time through a camera, an AI image recognition algorithm is used for analyzing the images, and blank screen, blue screen, blurred screen, splash screen, bright line and screen stretching abnormity are recognized; and recording data of each startup and shutdown test, automatically capturing abnormal pictures or videos according to an identification result, and generating a log file containing parameter changes before and after abnormity. The display fault in the test process can be found in time, and the problem can be quickly positioned.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer startup and shutdown testing, in particular to a computer startup and shutdown testing and abnormality monitoring method and system based on image recognition. BACKGROUND

[0002] In the daily use and product testing scenarios of computers, the frequent startup and shutdown operations of computers have very high requirements for stability. According to the three-year warranty period usually set by the product, if the startup and shutdown operation is performed once a day, then the startup and shutdown verification needs to be performed thousands of times in the entire warranty period. Such a high-repetitive and labor-intensive work obviously cannot be completed by manual testing.

[0003] Currently, the existing computer startup and shutdown testing scheme generally uses a mechanical arm to aim at the startup and shutdown button of the computer, and realizes the functions of starting up and shutting down by pressing the button at regular time intervals. However, the mechanical arm lacks intelligent judgment ability and cannot determine whether the computer is normally started up and shut down, nor can it monitor the abnormality in the startup and shutdown process in real time. SUMMARY

[0004] To solve the above problems, the present application provides a computer startup and shutdown testing and abnormality monitoring method and system based on image recognition.

[0005] In a first aspect, the present application provides a computer startup and shutdown testing and abnormality monitoring method based on image recognition, comprising the following steps: A startup and shutdown testing fixture is built, including an MCU microcontroller, a relay group, an interface control chip and a fixture interface, the fixture interface is connected with the MCU microcontroller through the interface control chip, and the MCU microcontroller is connected with the relay group; a high-resolution camera is configured and fixed at the center position of the front of the screen of the computer to be tested at a specified distance; The server sends an instruction to the fixture interface of the startup and shutdown testing fixture, and the MCU microcontroller controls the relay group to simulate the action of manually pressing the startup and shutdown button, so as to realize the opening and closing of the computer power supply; After the power supply is opened or closed, the server monitors the power supply state of the computer to be tested through ping operation and signal line; The camera captures the image of the screen of the computer to be tested in real time, and an AI image recognition algorithm is used to analyze the image to identify the abnormality of black screen, blue screen, flower screen, flashing screen, bright line and screen stretching; The data of each startup and shutdown test is recorded, the abnormal pictures or videos are automatically captured according to the identification result, and the log files containing the parameter changes before and after the abnormality are generated.

[0006] The server sends instructions to control the on-off test fixture, realizing the automation of on-off operation. At the same time, the AI image recognition algorithm is used to analyze the screen image in real time, which can intelligently identify various abnormal display problems, solving the problem that the existing mechanical arm scheme cannot monitor abnormalities in real time.

[0007] By real-time acquisition of screen images through the camera and analysis using the AI image recognition algorithm, abnormal display problems during the on-off process can be monitored in real time. At the same time, the data of each on-off test is recorded, and the potential fault points are analyzed through machine learning algorithm, solving the problem that the existing test method cannot record and analyze abnormal conditions in real time. Through the automatic and intelligent test method, manual intervention is reduced, and the test efficiency is improved. At the same time, the AI image recognition algorithm can accurately identify various abnormal display problems, improving the accuracy of the test.

[0008] As a further limitation of the technical solution of the present application, after the server sends the boot instruction, the control and monitoring are realized through the following steps: ICMP ping request is sent to the IP address of the computer under test to detect network link connectivity; The Power Supply On signal line level of the power supply is monitored, if the Power Supply On signal is low and the GND signal is grounded, it is determined that the power supply has started; if the Power Supply On signal is high or floating, it is determined that the power supply is not responding; Control the waiting time to ensure that the power supply is stable after starting, and then perform the ping operation and Power Supply On signal and GND signal monitoring again to confirm whether the power supply state is stable.

[0009] This multi-dimensional detection method combining network and physical signals can more comprehensively and accurately determine whether the computer has successfully booted, avoiding the misjudgment that may occur in single detection method, and improving the reliability of the boot state detection.

[0010] The rules for determining whether the power supply has started according to the level state of the Power Supply On signal and the GND signal are clearly specified, such as determining that the power supply has started when the Power Supply On signal is low and the GND signal is grounded, and determining that the power supply is not responding if the Power Supply On signal is high or floating. This precise determination rule provides a clear standard for monitoring the power supply startup state, which helps to discover power startup abnormal problems in time.

[0011] Through the multiple monitoring and confirmation, the false judgment caused by the temporary fluctuation in the power starting process can be effectively avoided, the computer can be ensured to run in the stable power state, and the accuracy and reliability of the whole test process are improved.

[0012] As a further limitation of the technical scheme of the application, after the server sends the shutdown instruction, the control and monitoring are realized through the following steps: The IP address of the computer under test is pinged, and if it cannot be connected and the Power Supply On signal is high, it is determined that the power has been turned off. After a waiting set time, the ping operation and power signal monitoring are executed again to confirm that the power state is off.

[0013] When it cannot be connected and the Power Supply On signal is high, it is determined that the power has been turned off. This judgment method comprehensively considers the network and power physical signals, can accurately determine the shutdown state of the computer, and avoids the occurrence of false judgment. Through multiple monitoring and confirmation, it can be ensured that the computer has been completely turned off, avoiding the inaccurate state judgment caused by the delay or abnormality in the shutdown process, and improving the reliability of the shutdown state detection.

[0014] As a further limitation of the technical scheme of the application, the monitoring of the Power Supply On signal and the GND signal line is realized by the MCU microcontroller of the power-on and power-off test fixture: The MCU microcontroller collects the Power Supply On signal and the GND signal line level signal in real time, converts the signal state into a digital signal, and transmits it to the server through the interface control chip and the fixture interface. The server analyzes the signal state to determine whether the power is on or off.

[0015] This real-time collection and transmission method can timely feedback the power signal state to the server, so that the server can timely understand the state change of the power, and provides a guarantee for the accurate monitoring of the power state. The intelligent level and accuracy of the power state monitoring are improved.

[0016] As a further limitation of the technical scheme of the application, the relay group includes a first relay and a second relay. The computer power is connected to the power interface of the computer through the normally open contact of the first relay, and the control end of the first relay is connected with the MCU microcontroller. The mainboard PWR_BTN pin is connected to the GND pin of the mainboard through the normally open contact of the second relay, and the control end of the second relay is connected with the MCU microcontroller. The MCU microcontroller controls the first relay to close the normally open contact to access the computer power supply, and after setting a delay time, the MCU microcontroller controls the normally open contact to short the mainboard PWR_BTN pin and the GND pin, simulating the physical button action.

[0017] The MCU microcontroller first controls the first relay to close the normally open contact to access the computer power supply, and after setting a delay time, controls the second relay to close the normally open contact to short the mainboard PWR_BTN pin and the GND pin. This step-by-step control method can accurately simulate the action of manually pressing the power button, ensuring normal computer startup and shutdown. This modular design also provides convenience for system expansion and upgrading, such as easily adding more relays to control other related hardware devices to meet the testing needs of different models of computers.

[0018] As a further limitation of the technical solution of the present application, the steps of analyzing the image using AI image recognition algorithm to identify black screen, blue screen, screen, flashing screen, bright line and screen stretching abnormality include: The collected image is preprocessed, including denoising, normalization and image enhancement; Based on the preprocessed image, edge detection algorithm is used to identify the edge features in the image; color analysis technology is used to extract color features in the image; texture analysis algorithm is used to extract texture features in the image; The extracted features are input into the trained fault detection model for anomaly detection to identify black screen, blue screen, screen, flashing screen, bright line and screen stretching abnormality; According to the detection result, the abnormal situation is classified and recorded.

[0019] Using a filter or wavelet transform to denoise can effectively remove noise interference in the image and improve the clarity of the image; normalizing the image pixel value to a reasonable range helps to unify the data format of the image and facilitate subsequent processing; image enhancement through histogram equalization and other methods can highlight important information in the image and improve the visual effect of the image, providing high-quality image data for subsequent anomaly detection. This multi-feature extraction method can analyze the image from different angles and fully obtain the information of the image, which helps to more accurately identify various abnormal display conditions and improve the accuracy and reliability of anomaly detection. The fault detection model learns various abnormal feature patterns through a large amount of training data, which can match and judge the input image features in a short time, greatly improving the efficiency of anomaly detection.

[0020] By analyzing the classification and frequency of abnormalities, we can better understand the problems that are prone to occur during computer startup and shutdown, provide targeted suggestions for product improvement and optimization, and also help technicians quickly locate and solve specific types of abnormal problems.

[0021] As a further limitation of the technical solution of the application, the abnormality detection step includes: A plurality of pixel blocks are obtained in the edge region and the middle region of the image respectively, the RGB value of each pixel block is calculated, and if the proportion of similar pixel blocks exceeds a preset threshold, it is judged as a black screen or a blue screen; The image is grayed and reduced, the variance of each row of pixel values in the vertical direction is calculated, and if there is a row with a variance change greater than a set threshold, it is judged as a screen with flowers; Through frame difference algorithm or histogram similarity analysis, the difference change between the front and rear frames in the image sequence is detected, and if the change exceeds the threshold, it is judged as a flashing screen; The edge detection algorithm is used to identify the bright line feature, and the texture analysis algorithm is used to extract the texture feature of the bright line; A reference object on the screen is selected, the proportion change is detected through feature extraction, and if the proportion changes, it is judged as stretching.

[0022] By analyzing the color similarity of the image pixel blocks, the two common abnormal display conditions of black screen and blue screen can be accurately identified, avoiding misjudgment caused by local changes of the image or other interference factors. The screen with flowers usually shows chaotic color stripes or patches, and by analyzing the variance change of the row pixel values of the image, this abnormal feature can be effectively captured to realize accurate identification of the screen with flowers. The characteristics of the flashing screen are that the screen display content flashes quickly, and the frame difference algorithm and the histogram similarity analysis can quickly compare the differences between adjacent frame images, timely discover the flashing screen phenomenon, and improve the real-time and accuracy of the flashing screen detection. Using the edge detection algorithm to identify the bright line feature, combined with the texture analysis algorithm to extract the texture feature of the bright line, the bright line abnormality on the screen can be accurately identified; selecting a reference object on the screen, detecting the proportion change through feature extraction, and if the proportion changes, it is judged as stretching, which can effectively detect the screen stretching abnormality. These two methods are targeted at the two specific abnormal display conditions of bright line and stretching, and use targeted feature extraction and analysis methods to improve the accuracy and reliability of the detection.

[0023] As a further limitation of the technical solution of the application, the method further includes the steps of model training and optimization: A large number of images with abnormal display are collected as a training set, and the abnormal regions are labeled; The segmentation mask of the abnormal region is used to guide the model learning, and the embedding vector is optimized to capture the abnormal features; Through multi-scale prediction and increasing the small target weight coefficient, the abnormal icon detection rate is improved; The preprocessed image is input into the trained fault detection model, and the detection result is output in real time; According to the detection result, the abnormal condition is classified and recorded for model optimization.

[0024] A large number of images showing abnormalities are collected as a training set, and the abnormal regions are labeled, and the segmentation mask of the abnormal region is used to guide the model learning. This targeted training method can make the model pay more attention to the abnormal region.

[0025] In a second aspect, the technical scheme of the present application also provides a computer power-on and power-off test and abnormality monitoring system based on image recognition, comprising: A power-on and power-off test fixture is used to simulate manual pressing of a power-on and power-off button to control the opening and closing of the computer power supply; the power-on and power-off test fixture comprises an MCU microcontroller, a relay group, an interface control chip and a fixture interface, the fixture interface is connected with the MCU microcontroller through the interface control chip, and the MCU microcontroller is electrically connected with the relay group; An image acquisition module comprises a high-resolution camera, which is fixed at the center position of the front of the screen of the computer to be tested at a certain distance, and is used to acquire images of the screen of the computer to be tested in real time; The control and monitoring server comprises: A power-on and power-off test module is used to send power-on and power-off instructions to the power-on and power-off test fixture and monitor the power supply state of the computer to be tested; the server is in communication connection with the fixture interface of the power-on and power-off test fixture, can detect network connectivity through ping operation, and can monitor the power supply state through the signal line; An image recognition processing module is in communication connection with the image acquisition module, is used to receive the images acquired by the camera, and analyzes the images through an AI image recognition algorithm to identify black screen, blue screen, flower screen, flashing screen, bright line and screen stretching abnormality; A data recording and analysis module is in communication connection with the power-on and power-off test module and the image recognition processing module, is used to record each power-on and power-off test data, abnormality recognition result and parameter change before and after the abnormality, automatically captures and stores abnormal pictures or videos, and generates a log file containing abnormal parameters.

[0026] As a further limitation of the technical scheme of the present application, the relay group comprises a first relay and a second relay; The computer power supply is connected to the power supply interface of the computer through the normally open contact of the first relay, and the control end of the first relay is connected with the MCU microcontroller; The PWR_BTN pin of the computer mainboard is connected to the GND pin of the computer mainboard through the normally open contact of the second relay, and the control end of the second relay is connected with the MCU microcontroller; The MCU microcontroller controls the normally open contact of the first relay to be closed to connect the computer power supply, and after a delay time is set, the MCU microcontroller controls the normally open contact to be closed to short the PWR_BTN pin and the GND pin of the computer mainboard, simulating the physical button action.

[0027] As a further limitation of the technical solution of the application, after sending the power-on instruction, the following steps are implemented to control and monitor: An ICMP ping request is sent to the IP address of the computer under test to detect network link connectivity; The Power Supply On signal line level of the power supply is monitored. If the Power Supply On signal is low and the GND signal is grounded, it is determined that the power supply has started. If the Power Supply On signal is high or floating, it is determined that the power supply is not responding; The waiting time is controlled to ensure that the power supply is stable after turning on, and the ping operation and Power Supply On signal and GND signal monitoring are executed again to confirm whether the power supply state is stable.

[0028] After sending the power-off instruction, the following steps are implemented to control and monitor: The IP address of the computer under test is pinged. If it cannot be connected and the Power Supply On signal is high, it is determined that the power supply has been turned off; After waiting for a set time, the ping operation and power supply signal monitoring are executed again to confirm that the power supply state is off.

[0029] The monitoring of the Power Supply On signal and GND signal lines is implemented by the MCU microcontroller of the power-on and power-off test fixture: The MCU microcontroller collects the Power Supply On signal and GND signal line level signals in real time, converts the signal state to a digital signal, and transmits it to the server through the interface control chip and fixture interface. The server analyzes the signal state to determine whether the power supply is on or off.

[0030] As can be seen from the above technical solution, the application has the following advantages: the MCU microcontroller is used to control the relay group to simulate manual pressing of the power-on and power-off button, realizing automatic operation of the computer power-on and power-off. After the power supply is turned on or off, the server monitors the power supply state of the computer under test through ping operation and signal line monitoring. Ping operation can detect network link connectivity, and signal line monitoring can directly obtain physical state information of the power supply. The combination of the two can accurately determine whether the computer is normally powered on and off in real time.

[0031] Using AI image recognition algorithms to analyze images can identify a variety of abnormal display conditions such as black screen, blue screen, screen, flashing screen, bright lines, and screen stretching. It can timely detect display failures that occur during computer power-on and power-off, which helps to quickly locate problems and improve product quality. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the present application, the drawings required to be used in the description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0033] Figure 1 The flowchart of the method provided by the embodiment of the present application is shown.

[0034] Figure 2 The block diagram of the system provided by the embodiment of the present application is shown. DETAILED DESCRIPTION

[0035] In order to make the application purposes, features and advantages of the present application more obvious and easy to understand, the technical solutions protected by the present application will be described clearly and completely by using specific embodiments and drawings. Obviously, the following described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0036] Unless otherwise defined, all technical and scientific terms used in the present application have the same meanings as those commonly understood by those skilled in the art to which the present application belongs. The terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application.

[0037] As shown in Figure 1 The technical solutions of the present application provide a computer startup and shutdown test and abnormality monitoring method based on image recognition, which comprises the following steps: S1, a startup and shutdown test fixture is built, which comprises an MCU microcontroller, a relay group, an interface control chip and a fixture interface, the fixture interface is connected with the MCU microcontroller through the interface control chip, and the MCU microcontroller is connected with the relay group; a high-resolution camera is configured, and the high-resolution camera is fixed at the center position of the front of the screen of the computer to be tested at a specified distance; In the embodiment of the present application, the fixture comprises an MCU microcontroller (such as an STM32 series), a relay group (containing 2-way relays), an interface control chip (such as a MAX232 serial chip) and a fixture interface (containing a COM port and a power interface). Among them, the fixture interface is connected with the MCU microcontroller through the interface control chip, the I / O pin of the MCU microcontroller is electrically connected with the control end of the relay group, and the instruction transmission and action control are realized.

[0038] Select a USB high-definition camera with a resolution of ≥1080P, fix it on the center of the screen of the computer to be tested, 50cm±5cm in front of the screen, the lens axis is perpendicular to the screen plane, to ensure complete capture of the screen display content.

[0039] S2, send instructions to the jig interface of the on-off test fixture through the server, control the relay group by the MCU microcontroller to simulate the action of manually pressing the on-off button, realize the opening and closing of the computer power supply; The server (such as an industrial computer equipped with Windows Server system) connects the COM port of the jig through a serial port line and sends on-off instructions (such as “PowerOn” and “PowerOff” string instructions).

[0040] After the jig interface receives the instructions, it converts them into digital signals recognizable by the MCU microcontroller through the interface control chip; the MCU microcontroller controls the action of the relay group according to the instructions: controls the closing / opening of the relay normally open contact, simulates the physical action of manually pressing the on-off button, and realizes the opening and closing of the computer power supply.

[0041] S3, after the power is turned on or off, the server monitors the power state of the computer to be tested through ping operation and signal line; After the power is turned on / off, the server starts a double monitoring mechanism: Send an ICMP ping request (timeout time 500ms, data packet size 32 bytes) to the preset IP address of the computer to be tested, and record the response result; Collect the Power Supply On (PSON#) signal line and GND signal line level state in real time through the jig to determine the physical on / off state of the power supply.

[0042] S4, collect the image of the screen of the computer to be tested in real time through the camera, analyze the image using AI image recognition algorithm, and identify black screen, blue screen, screen, flashing screen, bright line and screen stretching abnormality; The camera collects screen images in real time at a frame rate of 30fps and transmits them to the AI processing terminal (such as NV JetsonNano development board) through USB. The AI processing terminal runs the image recognition algorithm based on YOLOv8 to analyze the image in real time: through edge detection, color feature extraction, texture analysis and other technologies, identify black screen, blue screen, screen, flashing screen, bright line and screen stretching and other abnormalities.

[0043] S5, record the data of each on-off test, automatically capture abnormal pictures or videos according to the recognition result, and generate log files containing parameter changes before and after the abnormality.

[0044] The data recording module (deployed on the server) records the time, number of times, power signal state, network response data and screen abnormality identification result of each power-on and power-off in real time. When a valid abnormality is identified, the image (resolution 1920x1080) and short video (frame rate 30fps) lasting for 3 seconds at the abnormal time are saved. A structured log file containing the abnormality occurrence time, the model of the device under test, and the parameters before and after the abnormality (CLK frequency, core voltage, temperature, register value, log log segment) is generated and stored in the local database (such as MySQL) of the server.

[0045] In some embodiments, after the server sends the power-on instruction, the control and monitoring are realized by the following steps: ICMP ping requests are sent to the IP address of the computer under test to detect network link connectivity; The Power Supply On signal line level of the power supply is monitored. If the Power Supply On signal is low and the GND signal is grounded, it is determined that the power supply has started; if the Power Supply On signal is high or floating, it is determined that the power supply is not responding; The control waits for a set time to ensure that the power supply is stable after starting, and then the ping operation and Power Supply On signal and GND signal monitoring are performed again to confirm whether the power supply is stable.

[0046] After the server sends the "PowerOn" instruction, three ICMP ping requests are sent to the IP address of the computer under test in succession with a time interval of 1 second. If the number of successful responses in the three requests is ≥2 times (success rate ≥80%), it is determined that the network link is connected; otherwise, the initial connectivity abnormality of the network is recorded.

[0047] The MCU microcontroller collects the PSON# signal line level in real time through the ADC pin (sampling frequency 10Hz), and detects the GND signal grounding state through the GPIO pin (low level for conduction). If the PSON# signal is low (≤0.8V) and the GND signal is grounded, the server determines that the power supply has started; if the PSON# signal is high (≥2.0V) or floating (no stable level), it is determined that the power supply is not responding, triggering a primary alarm. The server controls the waiting for a set time (default 10 seconds, which can be configured to 5-30 seconds through software) to ensure that the power supply output is stable and the device under test completes the POST self-test.

[0048] The ping operation and signal monitoring are repeated 3 times every 10 seconds. When the ping response success rate is ≥80% and the PSON# low level is continuously conducted for 2 consecutive cycles, it is determined that the power supply is stable and turned on, and the normal test phase is entered.

[0049] In some embodiments, after the server sends the power-off instruction, the control and monitoring are realized by the following steps: The ping operation is performed on the IP address of the computer to be tested. If the ping operation fails and the Power Supply On signal is high, it is determined that the power supply has been turned off. The control waits for a set time, and then the ping operation and the power supply signal monitoring are performed again to confirm that the power supply is turned off.

[0050] After the server sends the "PowerOff" instruction, the ping operation is performed immediately. If the ping request fails for three consecutive times (timeout) and the PSON# signal becomes high (≥2.0V), it is preliminarily determined that the power supply has been turned off. The server waits for a set time (default 10 seconds) to ensure that the power supply is completely turned off.

[0051] After the waiting is completed, the ping operation and signal monitoring are performed every 10 seconds. If the ping operation fails for two consecutive times and the PSON# signal is high, it is determined that the power supply is stably turned off. Otherwise, the power-off abnormality is recorded and an alarm is triggered. In the embodiment of the application, the Power Supply On signal is the PSON# signal.

[0052] In some embodiments, the monitoring of the Power Supply On signal and the GND signal line is realized by the MCU microcontroller of the power-on / off test fixture: The MCU microcontroller collects the Power Supply On signal and the GND signal line in real time, converts the signal state into a digital signal, and then transmits the signal to the server through the interface control chip and the fixture interface. The server analyzes the signal state to determine whether the power supply is turned on or not.

[0053] The MCU microcontroller collects the analog level signals of the PSON# and GND signal lines in real time through a dedicated pin, converts the analog signals into digital signals in the range of 0-3.3V (corresponding to AD values of 0-4095).

[0054] The MCU microcontroller sends the digital signals (such as PSON: 0.5V; GND: 0V) to the interface control chip through the UART serial port. The interface control chip converts the TTL level into RS232 level, and then transmits the signals to the server through the fixture COM port.

[0055] The server-side software (based on the Python pyserial library) analyzes the received signal data and converts the AD values into actual level values: when the PSON# level is ≤0.8V and the GND level is ≤0.3V, it is determined that the power supply is turned on; when the PSON# level is ≥2.0V and the GND level is ≥2.0V, it is determined that the power supply is turned off.

[0056] In some embodiments, the relay group includes a first relay and a second relay; The computer power supply is connected to the power interface of the computer through the normally open contact of the first relay, and the control end of the first relay is connected with the MCU microcontroller; The mainboard PWR_BTN pin is connected to the GND pin of the mainboard through the normally open contact of the second relay, and the control end of the second relay is connected with the MCU microcontroller; The MCU microcontroller controls the normally open contact of the first relay to be closed to connect the computer power supply, and after setting a delay time, the MCU microcontroller controls the normally open contact to be closed to short the mainboard PWR_BTN pin and the GND pin, simulating the action of a physical button.

[0057] The first relay (large relay, model such as G2R-2-SNI): the normally open contact thereof is connected in series between the 220V AC power supply and the power interface of the computer to be measured, and the control end is connected with the I / O pin of the MCU microcontroller through a transistor drive circuit.

[0058] The second relay (small relay, model such as G6K-2P-Y): the normally open contact thereof is connected between the PWR_BTN pin and the GND pin of the mainboard of the computer to be measured, and the control end is connected with another I / O pin of the MCU microcontroller through a transistor drive circuit.

[0059] In the embodiment of the application, the coils of the first relay and the second relay are also connected with a DC power supply, after the server sends a start-up instruction, the MCU microcontroller outputs a high level to control the coil of the first relay to be electrified, the normally open contact is closed, and the computer to be measured is connected with the 220V AC power supply (delaying for 0.5 seconds to ensure stable power supply input). After setting a time (such as 2 seconds), the MCU microcontroller outputs a high level to control the coil of the second relay to be electrified, the normally open contact is closed, the PWR_BTN pin of the mainboard is short-circuited with the GND pin (for 1 second, simulating the action of manually pressing the button), and the computer is triggered to start up.

[0060] After the server sends a shutdown instruction, the MCU microcontroller controls the second relay to short-circuit the PWR_BTN pin and the GND pin again (for 1 second), and triggers the system to shut down; after detecting the power-off signal, the first relay is controlled to be disconnected, and the AC power supply input is cut off.

[0061] In some embodiments, the step of analyzing the image by using an AI image recognition algorithm to identify black screen, blue screen, flower screen, flashing screen, bright line and screen stretching abnormalities includes: The collected image is preprocessed, including denoising, normalization and image enhancement; De-noising: Apply a Gaussian filter (kernel size=3x3) to smooth the collected image, or decompose the image through wavelet transform and remove high-frequency noise components.

[0062] Normalization: Map the image pixel value from [0, 255] to the range [0, 1], the formula is: Normalized value = (original pixel value - 0) / (255 - 0).

[0063] Enhancement: Expand the image gray scale range through histogram equalization, enhance the dark or bright details (suitable for low contrast scenes such as black screen, white screen, etc.).

[0064] Based on the pre-processed image, use edge detection algorithm to identify the edge features in the image; through color analysis technology to extract the color features in the image; use texture analysis algorithm to extract the texture features in the image; the training data size of AI image recognition algorithm, wherein the training set contains 10000+ labeled images, covering 6 categories of abnormalities, and each category of sample ≥ 1500.

[0065] It should be noted that the Canny edge detection algorithm (threshold minVal=50, maxVal=150) is used to extract the edge information of the screen frame, icon contour, etc.

[0066] Calculate the mean, variance and histogram distribution of the RGB three channels of the image, extract the color features of black screen (R=G=B≈0), blue screen (B channel value is significantly higher than R / G).

[0067] Calculate the energy, entropy, contrast and other parameters of the image through gray level co-occurrence matrix, identify the blocky texture features of the flower screen.

[0068] Input the extracted features into the trained fault detection model for anomaly detection, identify black screen, blue screen, flower screen, flash screen, bright line and screen stretching anomaly; Input the extracted features into the trained YOLOv8 model (input resolution 1920x1080), the model outputs the abnormal category (black screen, blue screen, etc.) and the confidence; according to the confidence ≥ set threshold (such as 0.9) to determine the abnormal type, and record the abnormal area coordinates. According to the detection result, classify and record the abnormal situation.

[0069] In some embodiments, the anomaly detection step includes: In the edge region and the middle region of the image, a plurality of pixel blocks are obtained, the RGB value of each pixel block is calculated, and if the proportion of similar pixel blocks exceeds a preset threshold (the preset threshold is determined based on historical abnormal data statistics, for example, the proportion threshold of similar pixel blocks in black screen / blue screen judgment is set to 90%), it is judged as a black screen or a blue screen; 8 pixel blocks are taken in the edge region (100x100 pixel blocks are taken in each of the 4 corners) and the middle region (a center 200x200 pixel block), and the RGB average of each block is calculated. If the RGB value of ≥90% of the pixel blocks meets R≈G≈B≤30 (black screen) or B≥200 and R≤50, G≤50 (blue screen), it is determined as corresponding abnormality.

[0070] The image is grayed and reduced, the variance of each row of pixel values in the vertical direction is calculated, and if there is a row with a variance change greater than a set threshold, it is judged as a flower screen; The image is grayed and reduced to 320x240 size, and the variance of each row of pixels in the vertical direction is calculated. If there are ≥5 rows of pixels with variance >0.05 (after normalization), and the variance change rate of adjacent rows is >30%, it is determined as a flower screen.

[0071] Through frame difference algorithm or histogram similarity analysis, the difference change between the front and back frames in the image sequence is detected, and if the change exceeds the threshold, it is judged as a flashing screen; The sum of the absolute values of the pixel differences of the continuous 3 frames of images is calculated by frame difference algorithm, and if the sum of the differences is >30% of the total number of pixels and lasts for 2 frames or more, or the intersection of the histograms of the continuous frames is found to be <0.7 by histogram similarity analysis, it is determined as a flashing screen.

[0072] The edge detection algorithm is used to identify the bright line feature, and the texture analysis algorithm is used to extract the texture feature of the bright line; The Hough transform is used to detect the straight line feature (length ≥80% of the screen height) in the image, the brightness average of the straight line region is calculated, and if the brightness average is >3 times the average brightness of the screen, it is determined as a bright line.

[0073] A reference object on the screen is selected, the proportion change is detected by feature extraction, and if the proportion changes, it is judged as stretching.

[0074] A standard reference object (such as a system icon, a taskbar frame) in the screen is selected, the pixel ratio of the reference object width and height is extracted, and compared with the standard ratio (such as 16:9) of the normal device, and if the deviation rate is ≥5%, it is determined as screen stretching.

[0075] In some embodiments, the method further includes the steps of model training and optimization: A large number of images showing abnormalities are collected as a training set, and the abnormal areas are labeled; the segmentation mask of the abnormal area is used to guide the model learning, and the embedding vector is optimized to capture abnormal features; the abnormal icon detection rate is improved by multi-scale prediction and increasing the small target weight coefficient; the preprocessed image is input into the trained fault detection model, and the detection result is output in real time; according to the detection result, the abnormal condition is classified and recorded for model optimization.

[0076] Specifically, 10000+ abnormal images (including 6 types of abnormalities such as black screen and blue screen, each type ≥ 1500) are collected, the abnormal areas are labeled (rectangular frame or segmentation mask) using LabelImg tool, and the training set and the validation set are divided in the ratio of 8:2.

[0077] The model is initialized based on the YOLOv8 framework, the input resolution is configured as 1920x1080, and the loss function increases the small target weight coefficient (weight value 1.5). The abnormal area segmentation mask is used to guide the model to focus on abnormal features, and the Adam optimizer (learning rate 0.001) is used to iterate training for 50 rounds, and the model accuracy is verified every round, and the training is stopped when the validation set accuracy is ≥ 95%.

[0078] Multi-scale prediction refers to detecting small / medium / large size abnormal targets at different resolutions such as 80x80, 160x160, 320x320, etc., to improve the detection rate of small size bright lines and mosaics. Every 100 misjudged samples (such as misjudging a normal desktop as a screen of flowers), freeze the bottom convolution layer of the model, and only fine-tune the top classification layer (iterate for 10 rounds), so that the misjudgment rate is reduced by ≥ 10%.

[0079] As shown in Figure 2 The embodiment of the present application provides a computer power-on and power-off test and abnormal monitoring system based on image recognition, which comprises: A power-on and power-off test fixture is used to simulate manual pressing of a power-on and power-off button to control the opening and closing of the power supply of a computer; the power-on and power-off test fixture comprises an MCU microcontroller, a relay group, an interface control chip and a fixture interface, the fixture interface is connected with the MCU microcontroller through the interface control chip, and the MCU microcontroller is electrically connected with the relay group; An image acquisition module comprises a high-resolution camera, the camera is fixed at the center position of the front of the screen of the computer to be tested at a certain distance, and is used to acquire images of the screen of the computer to be tested in real time; The control and monitoring server comprises: A power-on and power-off test module is used to send power-on and power-off instructions to the power-on and power-off test fixture and monitor the power supply state of the computer to be tested; the server is in communication connection with the fixture interface of the power-on and power-off test fixture, can detect network connectivity through ping operation, and can monitor the power supply state through signal line; The image recognition processing module is in communication connection with the image acquisition module, and is used for receiving the image collected by the camera, and analyzing the image through an AI image recognition algorithm to identify a black screen, a blue screen, a flower screen, a flashing screen, a bright line and a screen stretching abnormality. The data recording and analysis module is in communication connection with the power-on and power-off test module and the image recognition processing module respectively, and is used for recording each power-on and power-off test data, abnormality recognition result and parameter change before and after the abnormality, automatically capturing and storing abnormal pictures or videos, and generating a log file containing abnormal parameters.

[0080] The power-on and power-off test fixture is mainly an MCU microcontroller (STM32F103), which is connected with a COM port of the fixture through an interface control chip (MAX232), and a relay group (containing 2-way G2R-2 relays) is connected with I / O pins of the MCU, so as to realize power-on and power-off action control.

[0081] The 1080P USB camera (frame rate 30fps) and the support are included, the camera is connected with an AI processing terminal (NVJetson Nano) through a USB line, and is fixed at a center position 50 cm in front of the screen to be tested.

[0082] The power-on and power-off test module (a serial communication program developed based on C#), the image recognition processing module (a Python service deploying a YOLOv8 model), the data recording and analysis module (a MySQL database + a Python data analysis script), and the abnormality storage module (a local disk array) are included.

[0083] The server sends a power-on and power-off instruction to the fixture through a serial port, the fixture MCU controls the relay to realize power-on and power-off, the server confirms the power state through ping and signal monitoring, the camera collects screen images and transmits them to the AI processing terminal, the AI module identifies the abnormality and feeds back to the server, the data recording module stores test data and abnormality information, and the abnormality storage module saves abnormal pictures / videos and generates a log.

[0084] In the embodiment of the application, the relay group includes a first relay and a second relay; The computer power supply is connected to the power interface of the computer through the normally open contact of the first relay, and the control end of the first relay is connected with the MCU microcontroller; The PWR_BTN pin of the computer mainboard is connected to the GND pin of the computer mainboard through the normally open contact of the second relay, and the control end of the second relay is connected with the MCU microcontroller; The MCU microcontroller controls the normally open contact of the first relay to be closed to connect the computer power supply, and after a delay time (such as 2 seconds), the MCU microcontroller controls the normally open contact to be closed to short the PWR_BTN pin and the GND pin of the computer mainboard, thereby simulating the physical button action.

[0085] The switch test module sends an ICMP ping request to the IP address of the computer under test after sending a start-up instruction to detect network link connectivity; The Power Supply On signal line level of the power supply is monitored. If the Power Supply On signal is low and the GND signal is grounded, it is determined that the power supply has started. If the Power Supply On signal is high or floating, it is determined that the power supply is not responding; The waiting time is controlled to ensure that the power supply is stable after starting, and then the ping operation and the Power Supply On signal and GND signal monitoring are executed again to confirm whether the power supply state is stable.

[0086] The switch test module pings the IP address of the computer under test after sending a shutdown instruction. If it cannot be connected and the Power Supply On signal is high, it is determined that the power supply has been turned off; The waiting time is controlled, and then the ping operation and power supply signal monitoring are executed again to confirm that the power supply state is off.

[0087] The above description of disclosed embodiments enables those skilled in the art to carry out or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined in this application can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown in this application, but will conform to the widest scope consistent with the principles and novel features disclosed in this application.

Claims

1. An image recognition based computer power on / off test and anomaly monitoring method, characterized in that, The method comprises the following steps: Build a switch test fixture, including MCU microcontroller, relay group, interface control chip and fixture interface, the fixture interface is connected with the MCU microcontroller through the interface control chip, and the MCU microcontroller is connected with the relay group; configure a high-resolution camera, and fix the high-resolution camera at the center position of the front of the screen of the computer to be tested at a set distance; The server sends an instruction to the fixture interface of the switch test fixture, and the MCU microcontroller controls the relay group to simulate the action of manually pressing the switch button, so as to realize the opening and closing of the computer power supply; After the power supply is turned on or off, the server monitors the power supply state of the computer to be tested through the ping operation and the signal line; The camera captures the image of the screen of the computer to be tested in real time, analyzes the image by using an AI image recognition algorithm, and identifies the black screen, blue screen, flower screen, flash screen, bright line and screen stretching abnormality; The data of each switch test is recorded, the abnormal picture or video is automatically captured according to the identification result, and a log file containing the parameter change before and after the abnormality is generated.

2. The image recognition based computer power on and off test and anomaly monitoring method according to claim 1, characterized in that, After the server sends the start-up instruction, the control and monitoring are realized through the following steps: An ICMP ping request is sent to the IP address of the computer to be tested to detect the network link connectivity; The power supply state is determined to be started when the power supply on signal is low and the GND signal is grounded; If the power supply on signal is high or floating, it is determined that the power supply is not responsive. The power supply state is determined to be stable after the power supply is turned on stably.

3. The image recognition based computer power on and off test and anomaly monitoring method of claim 2, wherein, After the server sends the shutdown instruction, the control and monitoring are realized through the following steps: The IP address of the computer to be tested is pinged, and it is determined that the power supply is turned off when the IP address cannot be connected and the power supply on signal is high. The power supply state is determined to be turned off after the ping operation and the power supply signal monitoring are executed again.

4. The image recognition based computer power on and off test and anomaly monitoring method of claim 3, wherein, The monitoring of the power supply on signal and the GND signal line is realized through the MCU microcontroller of the switch test fixture: The MCU microcontroller collects the power supply on signal and the GND signal line in real time, converts the signal state into a digital signal, and then transmits the signal to the server through the interface control chip and the fixture interface, so that the server analyzes the signal state to determine whether the power supply is turned on.

5. The image recognition based computer power on and off test and anomaly monitoring method of claim 4, wherein, The relay group comprises a first relay and a second relay; The computer power supply is connected to the power supply interface of the computer through the normally open contact of the first relay, and the control end of the first relay is connected with the MCU microcontroller; The PWR_BTN pin of the mainboard is connected to the GND pin of the mainboard through the normally open contact of the second relay, and the control end of the second relay is connected with the MCU microcontroller. The MCU microcontroller controls the normally open contact of the first relay to close to connect the computer power supply, and after setting a delay time, the MCU microcontroller controls the normally open contact of the second relay to close to short the mainboard PWR_BTN pin and the GND pin, simulating the action of a physical button.

6. The image recognition based computer power on and off test and anomaly monitoring method of claim 5, wherein, The steps of analyzing the image using an AI image recognition algorithm to identify black screen, blue screen, screen, screen, bright line and screen stretching abnormalities include: Preprocessing the collected image, including denoising, normalization and image enhancement; Based on the preprocessed image, use edge detection algorithm to identify the edge features in the image; extract the color features in the image through color analysis technology; use texture analysis algorithm to extract the texture features in the image; Input the extracted features into the trained fault detection model for anomaly detection to identify black screen, blue screen, screen, screen, bright line and screen stretching abnormalities; Classify and record the abnormal conditions according to the detection results.

7. The image recognition based computer power on and off test and anomaly monitoring method of claim 6, wherein, The steps of anomaly detection include: Get multiple pixel blocks in the edge area and the middle area of the image, calculate the RGB value of each pixel block, and if the proportion of similar pixel blocks exceeds the preset threshold, it is judged as black screen or blue screen; Gray the image and reduce the processing, calculate the variance of each row of pixel values in the vertical direction, and if there is a row with variance greater than the set threshold, it is judged as a screen; Through frame difference algorithm or histogram similarity analysis, detect the difference between the front and back frames in the image sequence, and if the change exceeds the threshold, it is judged as a screen; Use edge detection algorithm to identify bright line features, and use texture analysis algorithm to extract texture features of bright lines; Select a reference on the screen, detect the proportion change through feature extraction, and if the proportion changes, it is judged as stretching.

8. The image recognition based computer power on and off test and anomaly monitoring method of claim 7, wherein, The method also includes the steps of model training and optimization: Collect a large number of abnormal display images as training set, and label the abnormal areas; Use the segmentation mask of the abnormal area to guide the model learning, and optimize the embedding vector to capture the abnormal features; Through multi-scale prediction and increasing small target weight coefficient, the abnormal icon detection rate is improved; Input the preprocessed image into the trained fault detection model to output the detection results in real time; According to the detection results, classify and record the abnormal conditions for model optimization.

9. An image recognition based computer power-on and off test and abnormality monitoring system, characterized by, It includes: Power-on and power-off test fixture for simulating manual pressing of power-on and power-off button to control the opening and closing of computer power supply; the power-on and power-off test fixture includes MCU microcontroller, relay group, interface control chip and fixture interface, the fixture interface is connected with the MCU microcontroller through the interface control chip, and the MCU microcontroller is electrically connected with the relay group; Image acquisition module including high-resolution camera, the high-resolution camera is fixed at the center position of the measured computer screen at a certain distance in front of the screen, for real-time acquisition of the image of the measured computer screen; The control and monitoring server includes: Power-on and power-off test module for sending power-on and power-off instructions to the power-on and power-off test fixture and monitoring the power state of the measured computer; the server is in communication connection with the fixture interface of the power-on and power-off test fixture, can detect network connectivity through ping operation, and monitor the power state through signal line; The image recognition processing module is in communication connection with the image acquisition module, is used for receiving the image collected by the camera, and analyzes the image through an AI image recognition algorithm to identify a black screen, a blue screen, a flower screen, a flashing screen, a bright line, and a screen stretching anomaly; The data recording and analysis module is in communication connection with the power-on and power-off test module and the image recognition processing module, is used for recording each power-on and power-off test data, an abnormality recognition result, and a parameter change before and after the abnormality, automatically capturing and storing an abnormality picture or video, and generating a log file containing an abnormality parameter.

10. The image recognition based computer power on and off testing and anomaly monitoring system of claim 9, wherein, The relay group includes a first relay and a second relay; The computer power supply is connected to the power interface of the computer through the normally open contact of the first relay, and the control end of the first relay is connected with the MCU microcontroller; The PWR_BTN pin of the computer mainboard is connected to the GND pin of the computer mainboard through the normally open contact of the second relay, and the control end of the second relay is connected with the MCU microcontroller; The MCU microcontroller controls the normally open contact of the first relay to be closed to connect the computer power supply, and after a delay time is set, the MCU microcontroller controls the PWR_BTN pin and the GND pin of the computer mainboard to be short-circuited when the normally open contact is closed, to simulate a physical button action.

Images