BMC page automatic screenshot method and device and storage medium

Abstract

The invention relates to a BMC page automatic screenshot method and device and a storage medium, and the method comprises the steps: detecting the product information of a to-be-white product, and recognizing the product information needing to be modified; based on the product information needing to be modified, customized product information of the white card database is checked and supplemented through the upper computer; on the basis of the customized demand information, checking and supplementing dependent files of the white card database through the upper computer; and matching the information of the white card database, and executing white card customization operation. The related information of the to-be-white-signed product is obtained through the out-of-band management tool of the hardware product such as the out-of-band server, the customization information of the to-be-white-signed product is matched from the white-signed database, the customization requirement specified by a client is input into the white-signed database, the white-signed database automatically matches the corresponding customization script and the installation package, and the customization requirement of the to-be-white-signed product is met. Downloading, installing and deploying are carried out through the out-of-band server, and the purpose of automatic white board customization is achieved.

Description

Technical Field

[0001] The present invention relates to the field of server automated testing, and particularly to a method, device, and storage medium for automatically capturing screenshots of BMC pages. Background Art

[0002] When performing batch server testing tasks, it is usually necessary to obtain test data and information for each server, including capturing screenshots of the BMC page information of the server. Currently, manual page screenshots are taken for each server, which involves a large amount of repetitive work and is prone to errors. Moreover, when manually operating, there may be other applications and windows open that affect the screenshot content, resulting in screenshots of other screens, making the submitted data incorrect, and it is difficult for testers to notice. Summary of the Invention

[0003] The present invention provides a method, device, and storage medium for automatically capturing screenshots of BMC pages, aiming to solve at least one of the technical problems existing in the prior art.

[0004] The technical solution of the present invention is a method for automatically capturing screenshots of BMC pages, and the method for automatically capturing screenshots of BMC pages includes the following steps:

[0005] S100. Jump to the BMC page of the host to be captured through routing;

[0006] S200. Perform simulated click and automated screenshot operations on the page to obtain a screenshot file;

[0007] S300. Based on a pre-built and pre-trained YOLOv5 recognition model, detect whether the screenshot file meets the requirements. If the screenshot file does not meet the requirements, discard the current screenshot file and repeat steps S200 and S300 until a qualified screenshot file is obtained, and save the qualified screenshot file;

[0008] S400. If it is necessary to capture screenshots of other BMC pages of the current host, repeat steps S100 to S300 until all screenshots of the BMC pages of the current host are completed;

[0009] S500. If it is necessary to capture screenshots of BMC pages of other hosts, repeat steps S100 to S400 until all screenshots of BMC pages of all hosts are completed.

[0010] Furthermore, the present invention also proposes a device for automatically capturing screenshots of BMC pages for executing the method for automatically capturing screenshots of BMC pages. The device for automatically capturing screenshots of BMC pages is installed on a host computer, and a Python installation and running environment is set on the host computer. The host computer is electrically connected to at least one server through a network device. The device for automatically capturing screenshots of BMC pages includes:

[0011] Configuration module, used to configure and store a host information file for the host information that requires BMC page screenshots;

[0012] Simulation event click module, used to execute simulation click events in the browser;

[0013] Screenshot module, used to execute page screenshot operations to obtain screenshot files;

[0014] Detection and control module, used to perform matting and detection on the screenshot files until qualified screenshot files are obtained;

[0015] Storage module, used to store qualified screenshot files;

[0016] The configuration module, the simulation event click module, the screenshot module, and the storage module are respectively electrically connected to the detection and control module.

[0017] Furthermore, the present invention also proposes a computer-readable storage medium, on which program instructions are stored, and when the program instructions are executed by a processor, the BMC page automatic screenshot method described above is implemented.

[0018] The beneficial effects of the present invention are:

[0019] For the BMC page automatic screenshot method, device and storage medium, only one program runs during the screenshot process, and no other program windows will pop up to interfere with the screenshot effect; it ensures that the screenshots are of the same size, and the submitted data also has consistency and standardization; through the detection and control module, the screenshot files are automatically detected and controlled, and qualified screenshot files are saved, achieving the technical effect of automatically screenshotting the server BMC page information. Description of the Drawings

[0020] Figure 1 It is the overall flowchart of the BMC page automatic screenshot method.

[0021] Figure 2 It is the overall flow schematic diagram of the BMC page automatic screenshot method.

[0022] Figure 3 It is the schematic diagram of the automatic screenshot operation process in the BMC page automatic screenshot method.

[0023] Figure 4 It is the schematic diagram of the browser warning page in the BMC page automatic screenshot method.

[0024] Figure 5 It is the schematic diagram of logging in to the BMC interface in the BMC page automatic screenshot method.

[0025] Figure 6It is a schematic diagram of automatic screenshot method for BMC page, such as screenshots of conventional information like fan information without clicking page buttons.

[0026] Figure 7 It is a schematic diagram of the automatic screenshot method for BMC page, such as the schematic diagram of clicking on the detailed information of CPU.

[0027] Figure 8 It is a schematic diagram after clicking on the detailed information of CPU in the automatic screenshot method for BMC page.

[0028] Figure 9 It is a schematic diagram of the construction and training process of yolov5 recognition model in the automatic screenshot method for BMC page.

[0029] Figure 10 It is a schematic diagram of the BackBone part of feature extraction of yolov5 recognition model in the automatic screenshot method for BMC page.

[0030] Figure 11 It is a schematic diagram of the Neck part of multi-scale feature fusion of yolov5 recognition model in the automatic screenshot method for BMC page.

[0031] Figure 12 It is a schematic diagram of the Head part of the detection head of yolov5 recognition model in the automatic screenshot method for BMC page.

[0032] Figure 13 It is a schematic diagram of yolov5 recognition model in the automatic screenshot method for BMC page.

[0033] Figure 14 It is a schematic diagram of loading graphics in the automatic screenshot method for BMC page.

[0034] Figure 15 It is a schematic diagram of the recognition model performing matting and detection in the automatic screenshot method for BMC page. Detailed implementation manners

[0035] The following will clearly and completely describe the concept, specific structure and technical effects generated by the present invention in combination with embodiments and drawings, so as to fully understand the purpose, solution and effects of the present invention. It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0036] It should be noted that, unless otherwise specified, when a certain feature is referred to as "fixed" or "connected" to another feature, it can be directly fixed or connected to another feature, or indirectly fixed or connected to another feature. In addition, the up, down, left, right, top, bottom, etc. used in the present invention are only relative to the mutual positional relationship of each component of the present invention in the drawings.

[0037] In addition, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which this technology belongs. The terms used in the description of this specification are only for describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any combination of one or more of the related listed items.

[0038] It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various elements, these elements should not be limited to these terms. These terms are only used to distinguish elements of the same type from each other. For example, without departing from the scope of this disclosure, the first element may also be referred to as the second element, and similarly, the second element may also be referred to as the first element.

[0039] Refer to Figures 1 to 15 , in some embodiments, the technical solution of the present invention is a method for automatically capturing screenshots of BMC pages. Refer to Figure 1 , the method for automatically capturing screenshots of BMC pages includes the following steps:

[0040] S100. Jump to the host BMC page that needs to be captured through the route;

[0041] S200. Perform simulated clicks and automated screenshot operations on the page to obtain a screenshot file;

[0042] S300. Based on the pre-built and pre-trained yolov5 recognition model, detect whether the screenshot file meets the requirements. If the screenshot file does not meet the requirements, discard the current screenshot file and repeat steps S200 and S300 until a qualified screenshot file is obtained, and save the qualified screenshot file;

[0043] S400. If it is necessary to capture screenshots of other BMC pages of the current host, repeat steps S100 to S300 until all screenshots of the BMC pages of the current host are completed;

[0044] S500. If it is necessary to capture screenshots of BMC pages of other hosts, repeat steps S100 to S400 until all screenshots of BMC pages of all hosts are completed.

[0045] The beneficial effects of the present invention are:

[0046] For the method, device and storage medium for automatically capturing screenshots of BMC pages, only one program runs during the screenshot process, and no other program windows will pop up to interfere with the screenshot effect; it ensures that the screenshots are of the same size, and the submitted data also has consistency and standardization; through the detection and control module, the screenshot file is automatically detected and controlled, and the qualified screenshot file is saved, achieving the technical effect of automatically capturing screenshots of the server BMC page information.

[0047] Further, in step S100,

[0048] Adjust the page zoom ratio before the first configuration, keep the browser open until the screenshot process is completed, and load different routes for screenshotting.

[0049] Adjust the page zoom ratio before the first configuration. It can be zoomed manually or set to zoom automatically. The reason for zooming is that content such as CPU and memory is generally folded. If the folded information is expanded, the entire page information will be stretched. Therefore, it is necessary to perform reasonable zooming on the page, and the general ratio will not be lower than 0.7.

[0050] Keep the browser open until the screenshot process is completed. This function and program setting will only keep the browser open once (until all screenshots are completed and closed). This function is mainly to maintain the zoom ratio inside the browser. If the browser is frequently opened and closed, the zoom ratio will be reset.

[0051] Specifically, regarding page zooming, neither through automated configuration nor by inputting the page zoom ratio can meet the requirements of actual projects. Because the resolutions of each computer are different and the browser settings are also different. If the interface zoom ratio is directly configured, it may backfire. Instead, the zoom ratio should be adjusted manually during the first configuration. As long as the browser settings are not closed, different routes are loaded for screenshotting, so that the zoom ratio will not change and can self-adjust to an optimal range.

[0052] Therefore, in the code, only open the Chrome browser once and keep its persistence. Then, the browser can be zoomed manually. After reaching the appropriate ratio, only the first screenshot needs to be completed and manually verified to ensure that the information of subsequent screenshots is complete.

[0053] In a specific embodiment, the specific implementation code is intercepted as follows:

[0054] def main(config):

[0055] chromedriver_path = config["chromedriver_path"]

[0056] login_url = config['login_url']

[0057] images_url = config['images_url']

[0058] excel_file = config["excel_file"]

[0059] screenshot_dir = config.get('screenshot_dir', 'pillow_screenshots')

[0060] # Create a Chrome driver instance

[0061] chrome_options = setup_chrome_options()

[0062] service = Service(chromedriver_path)

[0063] driver = webdriver.Chrome(service=service, options=chrome_options)

[0064] try:

[0065] # Read the Excel file

[0066] df = pd.read_excel(excel_file)

[0067] for index, row in df.iterrows():

[0068] ip_address = row['ip']

[0069] username = row['username']

[0070] password = row['password']

[0071] try:

[0072] login(driver, ip_address, login_url, username,password)

[0073] # set_zoom(driver, '0.7')

[0074] for url_info in images_url:

[0075] take_screenshot(driver, ip_address, url_info, screenshot_dir)

[0076] except Exception as e:

[0077] logging.error(f"Error processing IP {ip_address}:{e}")

[0078] continue

[0079] finally:

[0080] driver.quit()

[0081] Furthermore, step S200 includes:

[0082] S210. If the Web application test environment selenium does not exist on the local machine, configure the Web application test environment selenium on the local machine by installing Chrome browser, the Google Chrome driver Chromedriver corresponding to the version, the Python running environment, the selenium dependency library, and the pytorch dependency library on the local machine;

[0083] S220. Configure the path of the Google Chrome driver Chromedriver, the network address URL to be screenshot, the naming of the screenshot image, the IP address of the BMC, and the SN list through a json configuration file;

[0084] S230. Point to the host information file for storing the host information of the BMC page screenshot to be taken through a json configuration file;

[0085] S240. If a simulated event click is required, perform the simulated event click and jump to the automatic login interface, and perform the automated self-signed https certificate security verification operation;

[0086] S250. Based on webdriver, switch to the login page, find the ID of the button in the page structure, and automatically fill in the data to perform the login operation;

[0087] S260. Perform page jumps in sequence according to the configured routing. If there is a configured click event, the click event needs to be executed after switching to the corresponding page;

[0088] S270. Perform the desktop screenshot operation to obtain the screenshot file.

[0089] In step S200, since a lot of hardware information is folded, it is necessary to simulate a click event on the page to open the folded information and perform route switching, and then take an automated screenshot of the desktop. To prevent forged screenshot data, the date and time of the current screenshot need to be displayed in all screenshots.

[0090] Specifically, step S210 is to configure the local selenium environment: use the Chrome browser on the local machine, put the corresponding version of chromedriver into the root directory where the browser is installed, and then install the python environment and the dependent libraries including selenium, pytorch and other dependent libraries to ensure that the local machine can successfully call the browser and open the page.

[0091] Step S220 is to configure the route, picture naming, BMC's IP address and SN list: by configuring a json file, the chromedriver path, the login url are passed in. The configuration files that need to be screenshot are composed of a data list of the screenshot name and the url that needs to be screenshot, and the data such as clicking on the web page nodes are unified and thus configured in the json file. What is more special is that the excel_file configuration passes in the bmc_ip and the corresponding account, password, serial number SN, and the file generally exists in the form of an excel file saved.

[0092] In a specific embodiment, the json file is as follows:

[0093] config.json

[0094] {"chromedriver_path": "C:\\Users\\s\\AppData\\Local\\Google\\Chrome\\Application\\133.0.6943.60\\chromedriver.exe",

[0095] "login_url": " / # / login",

[0096] "excel_file": "userinfo.xlsx",

[0097] "images_url":

[0098] {"description": "Switching", "url": " / # / navigate / home","screen":false},

[0099] {"description": "Home Page", "url": " / # / navigate / home","screen":true},

[0100] {"description": "Processor","screen":true, "url": "# / navigate / system / info / processor","storage":[{"id":"CPU1"},{"id":"CPU2"}]},

[0101] {"description": "Memory","screen":true, "url": "# / navigate / system / info / memory","id":"detailInfo0"},

[0102] {"description": "Array Card","screen":true, "url": "# / navigate / system / info / others","storage":[{"id":"PCIeCardNode"},{"id":"extendAttr2"}]},

[0103] {"description": "Switching", "url": " / # / navigate / home","screen":false},

[0104] {"description": "Network Card","screen":true, "url": "# / navigate / system / info / others","storage":[{"id":"PCIeCardNode"},{"id":"extendAttr1"},{"id":"extendAttr4"}]},

[0105] {"description": "Switching", "url": " / # / navigate / home","screen":false},

[0106] {"description": "Graphics card", "screen": true, "url": "# / navigate / system / info / others", "storage": [{"id": "PCIeCardNode"}, {"id": "extendAttr0"}, {"id": "extendAttr3"}]},

[0107] {"description": "Hard disk 446.625G", "screen": true, "select": true, "url": " / # / navigate / system / storage", "storage": [{"class": ".ti3-icon-plus-square"}, {"class": "#storageTree_0_0_icon.ti3-icon-plus-square"}, {"id": "HDDPlaneDisk0"}]},

[0108] {"description": "Switching", "url": " / # / navigate / home", "screen": false},

[0109] {"description": "Hard disk 3.493T", "screen": true, "select": true, "url": " / # / navigate / system / storage", "storage": [{"class": ".ti3-icon-plus-square"}, {"class": "#storageTree_0_1_icon.ti3-icon-plus-square"}, {"class": "#storageTree_0_1_0_icon.ti3-icon.ti3-icon-plus-square"}, {"id": "HDDPlaneDisk2"}]} ]}

[0110] Supplementary instructions for the json configuration file:

[0111] 1) Screenshots can be taken just by switching to the current interface;

[0112] 2) To switch to the current page, you need to click on the dom node to expand the information;

[0113] 3) There is a circular DOM node structure where hierarchical multiple clicks are required to expand the information content (such as the hard drive).

[0114] For the above situations:

[0115] 1) A new routing during switching is added. By navigating to the home page (without taking a screenshot), the page that was clicked and expanded can be restored, and then it can be switched back to take a screenshot.

[0116] 2) Regarding the problem of multi-level expansion, through various tag selections and hierarchical selection configurations, the relevant hard drive information can finally be expanded.

[0117] Specifically, referring to Figure 4 , step S240 is the self-signed HTTPS certificate security verification. Since the HTTPS certificates of the BMC services are all self-signed, the browser will give a warning and the user needs to choose whether to trust this address. The user needs to click the "Advanced" button and continue to go to 192.168.xx.xx (unsafe) in this way. If there is such a page, the following code will be executed. Referring to Figure 5 , and finally it will all jump to the login interface.

[0118] Specifically, for step S240, in a specific embodiment, the simulated click event code is as follows:

[0119] def handle_certificate_error(driver):

[0120] try:

[0121] # Wait and click the "Advanced" button

[0122] advanced_button = WebDriverWait(driver, 5).until(

[0123] EC.element_to_be_clickable((By.ID, 'details-button')) )

[0124] advanced_button.click()

[0125] # Wait and click the "Proceed" link

[0126] proceed_link = WebDriverWait(driver, 5).until(

[0127] EC.element_to_be_clickable((By.ID, 'proceed-link')) )

[0128] proceed_link.click()

[0129] except Exception as e:

[0130] # If the element cannot be found or the click fails, record the error log

[0131] logging.error(f"Certificate error page not found or could not be bypassed: {e}")

[0132] Specifically, for step S250, use webdriver to switch to the automatic login page, find the id of the button in the page structure, and then data can be automatically filled in for login. In a specific embodiment, the following is the automatic login code:

[0133] def login(ip_address, login_url, chromedriver_path, username, password):

[0134] chrome_options = setup_chrome_options() # Configure Chrome options

[0135] service = Service(chromedriver_path) # Create a ChromeDriver service

[0136] driver = webdriver.Chrome(service=service, options=chrome_options) # Start the Chrome browser

[0137] try:

[0138] # Access the login page

[0139] driver.get("https: / / "+ip_address + login_url)

[0140] handle_certificate_error(driver) # Handle certificate errors

[0141] # Enter the username and password

[0142] driver.find_element(By.ID, "account").send_keys(username)

[0143] driver.find_element(By.ID, "loginPwd").send_keys(password)

[0144] driver.find_element(By.ID, "btLogin").click() # Click the login button

[0145] driver.maximize_window() # Maximize the browser window

[0146] time.sleep(2) # Wait for 2 seconds

[0147] return driver # Return the browser driver object

[0148] except Exception as e:

[0149] # If the login fails, record the error log and quit the browser

[0150] logging.error(f"Failed to log in to {ip_address}: {e}")

[0151] driver.quit()

[0152] raise # Raise the exception

[0153] Specifically, for step S260, perform page jumps one by one according to the configured route. If there is a configured click event, after switching to the corresponding page, the click event needs to be executed to ensure that the folded hardware information can be opened.

[0154] Refer to Figures 6 to 8 , there are two cases here. One is that the screenshot can be taken on this page, and the other is that the components on this page need to be clicked to be displayed, and the screenshot can only be taken after clicking. This is the page after clicking. The click event needs to be simulated, click this button, and then take the screenshot. The following is a code example:

[0155] # Access the specified URL

[0156] driver.get("https: / / "+ip_address + url_info['url'])

[0157] handle_certificate_error(driver) # Handle certificate error

[0158] # If there is a "storage" property, click the relevant element

[0159] if url_info.get("storage"):

[0160] click_elements(driver, url_info["storage"])

[0161] else:

[0162] click_elements(driver, [url_info])

[0163] time.sleep(2) # Wait for the page to load

[0164] The core code for the automatic click event is as follows:

[0165] def click_elements(driver, elements):

[0166] for element_info in elements: # Traverse the element list

[0167] # Determine whether to use ID or CLASS_NAME for location based on element information

[0168] locator_type = By.ID if element_info.get("id") else By.CLASS_NAME

[0169] key = element_info.get("id") or element_info.get("class") # Get the ID or CLASS of the element

[0170] try:

[0171] # Wait for the element to be clickable and click

[0172] element = WebDriverWait(driver, 5).until(

[0173] EC.element_to_be_clickable((locator_type, key)) )

[0174] element.click()

[0175] # If the element has an "other" attribute, click the additional element

[0176] if element_info.get("other"):

[0177] other_element = WebDriverWait(driver, 5).until(

[0178] EC.element_to_be_clickable((By.ID, element_info["other"])) )

[0180] other_element.click()

[0181] time.sleep(5) # Wait for 5 seconds after clicking

[0182] except Exception as e:

[0183] # If the click fails, record the error log

[0184] logging.error(f"Failed to click element {key}: {e}")

[0185] Specifically, for step S270, the following is the code for the desktop screenshot:

[0186] screenshot = ImageGrab.grab() # Take a screenshot of the screen

[0187] # Detect the screenshot

[0188] if detect_success(screenshot,model): # Call the detection function

[0189] # If the detection is successful, save the screenshot to the specified path

[0190] save_path = os.path.join(screenshot_dir, ip_address.replace("https: / / ", ""), url_info['description'] + '.png')

[0191] os.makedirs(os.path.dirname(save_path), exist_ok=True) # Create directory if it doesn't exist

[0192] screenshot.save(save_path) # Save the screenshot

[0193] logging.info(f'Successfully created screenshot at {save_path}') # Log the success

[0194] return # Detection successful, exit the function

[0195] else:

[0196] # Detection failed, log and wait 1 second before retrying

[0197] logging.info(f"Detection failed, retrying... (Attempt{retries + 1} / {max_retries})")

[0198] retries += 1

[0199] time.sleep(1) # Wait for 1 second

[0200] Furthermore, in step S230:

[0201] The host information at least includes the IP address, account, password, and serial number SN of the host BMC;

[0202] The host information file contains at least one piece of host information.

[0203] Furthermore, in step S400,

[0204] Perform page jumps in sequence according to the configured route. If a simulated event click is configured, after switching to the corresponding page, perform the simulated event click again.

[0205] The simulated event click includes being able to take a screenshot on the current page and needing to click on the current page to take a screenshot.

[0206] Further, before step S300, it also includes cropping the screenshot file to intercept the central region of a preset size of the screenshot.

[0207] Specifically, referring to Figure 15 , in step S300, the trained yolov5 recognition model best.pt is loaded through the path, and the code is as follows:

[0208] # Load the detection loading model

[0209] try:

[0210] model = torch.hub.load('ultralytics / yolov5', 'custom', path=model_path)

[0211] except Exception as e:

[0212] logging.error(f"Failed to load model from {model_path}:{e}")

[0213] return

[0214] According to the logic of central cropping, crop the intercepted image data at the central position. Here, the position of 300*300 is a large range, and it only needs to ensure that the loaded loading effect image can be included in this range.

[0215] def crop_center_region(screenshot, crop_width=300, crop_height=300):

[0216] """Crop the central region image from the screenshot"""

[0217] width, height = screenshot.size

[0218] left = (width - crop_width) / / 2

[0219] top = (height - crop_height) / / 2

[0220] right = (width + crop_width) / / 2

[0221] bottom = (height + crop_height) / / 2

[0222] cropped_image = screenshot.crop((left, top, right, bottom))

[0223] return cropped_image

[0224] Detection logic implementation: Use the model to detect whether there is a loading image. If there is, return True; if not, return False

[0225] def detect_loading_spinner(cropped_image, model, conf_thres=0.5):

[0226] results = model(cropped_image)

[0227] # Get the detection results

[0228] detections = results.pandas().xyxy[0]

[0229] # Check if there are any detections above the confidence threshold

[0230] if len(detections) > 0 and any(detections['confidence'] > conf_thres):

[0231] return True # If a loading animation is detected, return True

[0232] return False # If no loading animation is detected, return False

[0233] Execute the detection: Call the cropping function and the detection function inside the detect_success function, and finally perform a logical judgment. When the detection is True, it means there is still a loading graphic effect, so return False; in the other case, when no Loading is detected, return True, indicating that the data has been loaded successfully.

[0234] def detect_success(screenshot, model):

[0235] """Detect whether there is a loading animation in the screenshot and only return the detection result"""

[0236] try:

[0237] # Crop the image of the central region

[0238] cropped_image = crop_center_region(screenshot)

[0239] # Use the model to detect the loading animation

[0240] loading_spinner_detected = detect_loading_spinner(cropped_image, model)

[0241] if loading_spinner_detected:

[0242] # If the loading animation is detected, it means the page has not been fully loaded, return False

[0243] logging.warning("Loading spinner detected, page may not be fully loaded.")

[0244] return False

[0245] else:

[0246] # If the loading animation is not detected, it is considered that the detection is successful, return True

[0247] logging.info("No loading spinner detected, detection successful.")

[0248] return True

[0249] except Exception as e:

[0250] logging.error(f"Error in detect_success: {e}")

[0251] return True # By default, it is considered that the detection fails when an error occurs (i.e., the page has not been fully loaded)

[0252] Complete the screenshot capture of a single IP: Repeat the above steps to complete all the screenshots in an IP and save all the qualified screenshots.

[0253] Further, in step S300, the yolov5 recognition model is built and trained through the following steps:

[0254] S301. Configure the Python running environment and use the pip command to install the dependent packages of torch, torchvision, opencv-python, and matplotlib;

[0255] S302. Use the git command to clone the yolov5 repository to obtain the yolov5 model, and use pip to install all the dependent packages;

[0256] S303. For the feature extraction BackBone part of the yolov5 model, insert a CBAM module after each downsampling layer Conv. The CBAM module includes a channel attention part and a spatial attention part;

[0257] S304. For the multi-scale feature fusion Neck part of the yolov5 model, the top-down path is to fuse the high-level large-object features and the low-level small-object features through upsampling (Upsample), and the bottom-up path is to fuse the low-level small-object features and the high-level large-object features through downsampling. After all the fusions, connect to the C3 module;

[0258] S305. For the detection head Head part of the yolov5 model, each detection head includes a bounding box prediction part and a classification prediction part;

[0259] S306. Based on the labeled dataset, train the adjusted yolov5 model, generate a dataset configuration file, and finally obtain the trained yolov5 model.

[0260] Specifically, in a specific embodiment, the yolov5 recognition model is built and trained through the following steps:

[0261] 1) Install dependencies: Use the pip command to install the dependent packages.

[0262] pip install torch torchvision opencv-python matplotlib

[0263] 2) Clone the repository: Use the git command to clone, then enter the yolov5 directory, and use pip to install all the dependent packages

[0264] git clone https: / / github.com / ultralytics / yolov5

[0265] cd yolov5

[0266] pip install -r requirements.txt

[0267] 3) Modify the yolov5 model structure:

[0268] Among them, referring to Figure 10 , the key modification to the feature extraction BackBone part of the model is to insert the CBAM module after each downsampling layer (after Conv), and the order is:

[0269] Conv → CBAM → C3 Block (for example, P1 / 2 → CBAM1 → P2 / 4 → C3_128x3).

[0270] The CBAM structure includes channel attention (global average pooling + MLP) and spatial attention (7x7 convolution + Sigmoid).

[0271] Among them, referring to Figure 11 , the key modification to the multi-scale feature fusion Neck part is:

[0272] 1) Top-down path: Fuse high-level features (large objects) and low-level features (small objects) through upsampling (Upsample), such as: P5 / 32 → Upsample → Concat with P4 / 16.

[0273] 2) Bottom-up path: Fuse low-level features and high-level features through downsampling (Conv 3x3 stride2).

[0274] Such as: P3 / 8 → Downsample → Concat with P4 / 16.

[0275] 3) After all fusions, connect to the C3 module (without residual connection, False parameter) to further integrate features

[0276] Among them, referring to Figure 12 , for the detection head Head part:

[0277] Function of the detection head Head part: Detect the feature maps of the three scales of P3 / 8, P4 / 16, and P5 / 32 output by the Neck respectively. Therefore, each detection head Head contains:

[0278] Bounding box prediction (4 coordinates + 1 confidence),

[0279] Classification prediction (probabilities of nc classes);

[0280] The output dimension is:

[0281] P3 / 8: 80x80x(5 + nc) (suitable for small targets),

[0282] P4 / 16: 40x40x(5 + nc) (suitable for medium targets),

[0283] P5 / 32: 20x20x(5 + nc) (suitable for large targets).

[0284] Therefore, referring to Figure 13 , the complete model flow chart includes:

[0285] 1) Backbone: Extract features and insert CBAM (enhance key features).

[0286] 2) Neck: Bidirectionally fuse multi-scale features (FPN + PAN structure).

[0287] 3) Head: Independently predict bounding boxes and classes for feature maps of each scale.

[0288] Referring to Figure 10 and Table 1, the modules of the BackBone part of the modified yolov5 are as follows:

[0289] Table 1:

[0290] Module Diagram Chinese Name Parameter Description Function Description Conv 64, 6x6, stride2 Downsampling Convolution Layer 1 64 output channels, 6x6 convolution kernel, stride 2, padding = 2 Downsample the image from 640x640 to 320x320, and expand the number of channels from 3 (RGB) to 64 CBAM 64 Attention Module 1 Input 64 channels, channel attention reduction ratio 16 Enhance important features through channel + spatial attention mechanism C3 128x3 Cross-Stage Module 1 128 input / output channels, containing 3 Bottleneck structures Fuse features with different receptive fields, reduce computational complexity while maintaining feature representation ability Conv 128, 3x3, stride2 Downsampling Convolution Layer 2 128 output channels, 3x3 convolution kernel, stride 2 Downsample the feature map from 160x160 to 80x80 CBAM 128 Attention Module 2 Input 128 channels Apply attention weighting to middle-level features C3 256x6 Cross-Stage Module 2 256 channels, 6 Bottlenecks Deepen the network depth and extract more complex features CBAM 256 Attention Module 3 Input 256 channels Apply attention at the key feature layer (P3 / 8) Conv 512, 3x3, stride2 Downsampling Convolution Layer 4 512 output channels Output 40x40 feature map (P4 / 16) C3 512x9 Cross-Stage Module 3 512 channels, 9 Bottlenecks Deep feature extraction, with a large number of parameters CBAM 512 Attention Module 4 Input 512 channels Enhance high-level semantic features Conv 1024, 3x3, stride2 Downsampling Convolution Layer 5 1024 output channels Output 20x20 minimum feature map (P5 / 32) C3 1024x3 Cross-Stage Module 4 1024 channels, 3 Bottlenecks Final feature refinement CBAM 1024 Attention Module 5 Input 1024 channels Apply attention on the highest-level features

[0291] Referring to Figure 11 and Table 2, the modules of the Neck part of the modified yolov5 are as follows:

[0292] Table 2:

[0293] Module Diagram Chinese Name Parameter Description Function Description Conv 512, 1x1 Feature Compression Layer 1 1x1 convolution kernel, 512 output channels Compress 1024 channels to 512 channels to reduce computational complexity Upsample 2x Upsampling Layer 2x nearest neighbor interpolation Enlarge the feature map size by 2 times (20x20 → 40x40) Concat with CBAM4 Feature Concatenation Operation 1 Concatenate along the channel dimension Fuse high-level semantic features (upsampling result) and low-level detail features (CBAM4 output) C3 512_False Feature Fusion Layer 1 512 channels, residual connection disabled Fuse the concatenated 1024-channel features and output 512 channels Conv 256, 1x1 Feature Compression Layer 2 1x1 convolution kernel, 256 output channels Further compress the number of channels (512→256) Concat with CBAM3 Feature concatenation operation 2 Concatenate along the channel dimension Fuse middle-level features (80x80 resolution) C3 256_False Feature fusion layer 2 256 channels, residual connection turned off Process the concatenated 512-channel features Conv 256,3x3,stride2 Downsampling layer 1 3x3 convolution, stride 2 Downsample the feature map from 80x80 to 40x40 Concat with N1 output Feature concatenation operation 3 Cross-level concatenation Fuse 40x40 resolution features at different stages C3 512_False_2 Feature fusion layer 3 512 channels, residual connection turned off Process the mixed 768-channel features Conv 512,3x3,stride2 Downsampling layer 2 3x3 convolution, stride 2 Downsample the feature map from 40x40 to 20x20 Concat with CBAM5 Feature concatenation operation 4 Concatenate along the channel dimension Finally fuse the lowest-level features (20x20 resolution) C3 1024_False Feature fusion layer 4 1024 channels, residual connection turned off Output the final multi-scale fused features

[0294] Referring to Figure 13 and Table 3, the modules of the Head part of the modified yolov5 are as follows:

[0295] Table 3:

[0296] Module diagram Chinese name Parameter description Functional description Neck P3 Neck feature P3 80×80×256 feature map Highest resolution feature from Neck, retaining small object details Neck P4 Neck feature P4 40×40×512 feature map Middle-level features, balancing details and semantic information Neck P5 Neck feature P5 20×20×1024 feature map Deepest low-resolution feature, containing strong semantic information Detect Head (P3) P3 detection head 3×(5+nc) output channels Each grid predicts 3 anchors, outputting coordinates + confidence + classification Detect Head (P4) P4 detection head 3×(5+nc) output channels Medium resolution detection, suitable for medium-scale objects Detect Head (P5) P5 detection head 3×(5+nc) output channels Low resolution detection, suitable for large-scale objects Predictions (P3) P3 prediction output 80×80×3×(5+nc) Highest resolution, dedicated to small object detection Predictions (P4) P4 prediction output 40×40×3×(5+nc) Medium resolution, general object detection Predictions (P5) P5 prediction output 20×20×3×(5+nc) Lowest resolution, dedicated to large object detection

[0297] Referring to Figures 9 to 13 , in a specific embodiment, the example code is as follows:

[0298] Introduce the CBAM module and add the CBAM module in models / common.py:

[0299] import torch

[0300] import torch.nn as nn

[0301] class CBAM(nn.Module):

[0302] def __init__(self, channels, reduction_ratio=16):

[0303] """Initialization of the CBAM module.

[0304] Parameters: channels (int): The number of channels of the input feature map.

[0305] reduction_ratio (int): The reduction ratio in the channel attention module, default is 16.

[0306] """

[0307] super(CBAM, self).__init__()

[0308] # Channel attention module

[0309] self.channel_attention = nn.Sequential(

[0310] nn.AdaptiveAvgPool2d(1), # Global average pooling, compresses the spatial dimension of the feature map to 1x1

[0311] nn.Conv2d(channels, channels / / reduction_ratio, kernel_size=1), # 1x1 convolution, reduces the number of channels

[0312] nn.ReLU(inplace=True), # ReLU activation function

[0313] nn.Conv2d(channels / / reduction_ratio, channels, kernel_size=1), # 1x1 convolution, restores the number of channels

[0314] nn.Sigmoid() # Sigmoid activation function, generates channel attention weights )

[0315] # Spatial attention module

[0316] self.spatial_attention = nn.Sequential(

[0317] nn.Conv2d(channels, 1, kernel_size=7, padding=3), # 7x7 convolution, generating a spatial attention map

[0318] nn.Sigmoid() # Sigmoid activation function, generating spatial attention weights )

[0319] def forward(self, x):

[0320] """ Forward propagation function.

[0321] Parameters: x (torch.Tensor): Input feature map, with shape

[0322] [batch_size, channels, height, width]

[0323] Returns: torch.Tensor: Feature map weighted by channel attention and spatial attention.

[0324] """

[0325] # Channel attention

[0326] channel_att = self.channel_attention(x) # Calculate channel attention weights

[0327] x = x * channel_att # Apply channel attention weights to the input feature map

[0328] # Spatial attention

[0329] spatial_att = self.spatial_attention(x) # Calculate spatial attention weights

[0330] x = x * spatial_att # Apply spatial attention weights to the input feature map

[0331] return x # Return the weighted feature map

[0332] Modify the C3 module in models / yolov5s.yaml, add CBAM, and insert the CBAM module after each C3 module to process feature maps of different scales. The number of input channels of the CBAM module is the same as the number of output channels of the corresponding C3 module.

[0333] For the Backbone part, the following is an example code:

[0334] # [from, number, module, args]

[0335] [[-1, 1, Conv, [64, 6, 2, 2]], # 0-P1 / 2

[0336] [-1, 1, CBAM,

[64] ], # 1-CBAM

[0337] [-1, 1, Conv, [128, 3, 2]], # 2-P2 / 4

[0338] [-1, 3, C3,

[128] ],

[0339] [-1, 1, CBAM,

[128] ], # 4-CBAM

[0340] [-1, 1, Conv, [256, 3, 2]], # 5-P3 / 8

[0341] [-1, 6, C3,

[256] ],

[0342] [-1, 1, CBAM,

[256] ], # 7-CBAM

[0343] [-1, 1, Conv, [512, 3, 2]], # 8-P4 / 16

[0344] [-1, 9, C3,

[512] ],

[0345] [-1, 1, CBAM,

[512] ], # 10-CBAM

[0346] [-1, 1, Conv, [1024, 3, 2]], # 11-P5 / 32

[0347] [-1, 3, C3,

[1024] ],

[0348] [-1, 1, CBAM,

[1024] ], # 13-CBAM ​

[0350] Example code for the multi-scale feature fusion part is as follows:

[0351] Add a multi-feature fusion layer to models / yolov5s.yaml.

[0352] Upsampling and Concat: Use nn.Upsample to upsample the feature map to match the size of the higher-resolution feature map. Use the Concat operation to concatenate feature maps of different scales together.

[0353] C3 module: The concatenated feature map further extracts features through the C3 module.

[0354] Multi-scale feature fusion: Combine high-resolution shallow features (such as P3) with low-resolution deep features (such as P5) to enhance the model's detection ability for small targets.

[0355] neck:

[0356] [[-1, 1, Conv, [512, 1, 1]],

[0357] [-1, 1, nn.Upsample, [None, 2, 'nearest']],

[0358] [[-1, 6], 1, Concat, [1]], # cat backbone P4

[0359] [-1, 3, C3, [512, False]], # 13

[0360] [-1, 1, Conv, [256, 1, 1]],

[0361] [-1, 1, nn.Upsample, [None, 2, 'nearest']],

[0362] [[-1, 4], 1, Concat, [1]], # cat backbone P3

[0363] [-1, 3, C3, [256, False]], # 17 (P3 / 8-small)

[0364] [-1, 1, Conv, [256, 3, 2]],

[0365] [[-1, 14], 1, Concat, [1]], # cat head P4

[0366] [-1, 3, C3, [512, False]], # 20 (P4 / 16-medium)

[0367] [-1, 1, Conv, [512, 3, 2]],

[0368] [[-1, 10], 1, Concat, [1]], # cat head P5

[0369] [-1, 3, C3, [1024, False]], # 23 (P5 / 32-large)

[0371] Regarding the training model, prepare the dataset: The dataset contains images of the loading effect and is labeled using LabelImg to generate the dataset configuration file data.yaml.

[0372] train: path / to / train / images

[0373] val: path / to / val / images

[0374] nc: 1

[0375] names: ['loading']

[0376] Start training

[0377] python train.py --img 640 --batch 16 --epochs 100 --data data.yaml --cfg yolov5s_laoding.yaml --weights yolov5s.pt

[0378] Finally, obtain the best.pt model.

[0379] Furthermore, in step S300,

[0380] Detecting whether the screenshot file meets the requirements is to detect whether there is a loading image through a pre-built and pre-trained yolov5 recognition model.

[0381] Specifically, refer to Figure 14 ​, in step S300, there is a problem with the automated BMC page screenshot. That is, it is difficult to determine the time when the BMC page loads data. Due to various reasons such as the performance, quantity, and bus BUS communication of the hardware, the page loading speed is relatively slow. Therefore, it may cause the automated screenshot to take a screenshot operation before the hardware data on the page is fully loaded. Thus, the present invention cleverly uses an object detection model in computer vision and uses AI to detect whether the loading graphic exists, thereby ensuring that the data on the page is fully loaded and obtaining complete data (excluding the loading animation effect).

[0382] Normally, the logic of page loading is that in the first step, a loading graphic (dynamic spinning circle or arc) appears, in the second step, data is requested, in the third step, the data is obtained, in the fourth step, the page is rendered, and in the fifth step, the loading graphic is cancelled. Therefore, if the loading graphic always exists, it means that the data on the page has not been fully loaded and the rendering has not been completed. At this time, yolov5 can be used to train based on the intercepted and labeled loading graphic, so as to detect the loading graphic in the picture to determine whether the data has been fully loaded, and further judge whether the picture is qualified.

[0383] Furthermore, the present invention also proposes a BMC page automatic screenshot device for executing the BMC page automatic screenshot method. The BMC page automatic screenshot device is installed on the upper computer. A Python installation and operation environment is set on the upper computer. The upper computer is electrically connected to at least one server through a network device. The BMC page automatic screenshot device includes:

[0384] A configuration module for configuring and storing the host information file that requires BMC page screenshots;

[0385] A simulated event click module for executing simulated click events in the browser;

[0386] A screenshot module for executing page screenshot operations to obtain screenshot files;

[0387] A detection control module for cropping and detecting the screenshot file until a qualified screenshot file is obtained;

[0388] A storage module for storing qualified screenshot files;

[0389] The configuration module, the simulated event click module, the screenshot module, and the storage module are respectively electrically connected to the detection control module.

[0390] In a specific embodiment, since the BMC page automatic screenshot device is a software or other application program and needs to be configured into different host computers or servers, the size of the software affects the installation and debugging efficiency of on-site workers. It is necessary to package the Python program into a final exe file for installation and debugging. Therefore, the present invention optimizes the software or other application program of the BMC page automatic screenshot device, specifically including the following steps:

[0391] 1. Reduce the version of selenuim to a small-size version;

[0392] 2. Use the pyexcel + plugin package method to replace the pandas plugin package;

[0393] 3. Some of the code is as follows:

[0394] import pyexcel # Replace the original pandas reading method

[0395] try:

[0396] # Read the Excel file

[0397] records = pyexcel.get_records(file_name = excel_file)

[0398] for row in records:

[0399] ip_address = row['ip']

[0400] username = row['username']

[0401] password = row['password']

[0402] The terminal display information is:

[0403] PS E:\Python\screen> pyinstaller –onefile –icon = 1.ico –hidden-import pyexcel_xlsx –hidden-import pyexcel_xls screen.py

[0404] INFO: Pyinstaller:5.13.2

[0405] INFO: Python 3.10.11

[0406] Note: The parameters --hidden-import pyexcel_xlsx and --hidden-import pyexcel_xls are to prevent the loss of plugins.

[0407] 4. Use upx to optimize the package size. Compared with step 2, add the parameter --upx-dir=E:\upx-5.0.0-win64\, and add the upx parameter during compilation for packaging. The terminal display information is as follows:

[0408] Pyinstaller –onefile –upx-dir=E:\upx-5.0.0-win64\ –icon=1.ico –hidden-import pyexcel_xlsx –hidden-import pyexcel_xls screen.py

[0409] Through the above steps, the size of this software package has been optimized from 2.37G to only 23M, and the functions are exactly the same, which greatly facilitates subsequent use and customized development.

[0410] Furthermore, the present invention also proposes a computer-readable storage medium, on which program instructions are stored, and when the program instructions are executed by a processor, the BMC page automatic screenshot method described above is implemented.

[0411] The above is only a preferred embodiment of the present invention. The present invention is not limited to the above implementation manner. As long as it achieves the technical effects of the present invention by the same means, any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present disclosure shall be included within the scope of protection of the present disclosure. Within the scope of protection of the present invention, its technical solutions and / or implementation manners can have various different modifications and changes.

Claims

1. A method for automatically taking screenshots of a BMC page, characterized in that: The BMC page automatic screenshot method comprises the following steps: S100, jump to the host BMC page that needs to be screenshot through routing; S200, performing simulated click and automatic screenshot operations on the page to obtain a screenshot file; S300, based on the pre-built and pre-trained yolov5 recognition model, detecting whether the screenshot file meets the requirements, if the screenshot file does not meet the requirements, discarding the current screenshot file and repeating steps S200 and S300 until a qualified screenshot file is obtained, and saving the qualified screenshot file; S400: If it is necessary to take screenshots of other BMC pages of the current host, repeat steps S100 to S300 until all screenshots of the BMC pages of the current host are completed; S500: If it is necessary to take screenshots of the BMC pages of other hosts, repeat steps S100 to S400 until the screenshots of the BMC pages of all hosts are completed.

2. The BMC page automatic screenshot method according to claim 1, characterized in that: In step S100, Adjust the page zoom ratio before the first configuration, keep the browser open until the screenshot process is completed, and load different routes for screenshots.

3. The BMC page automatic screenshot method according to claim 1, characterized in that: Step S200 includes: S210, if the web application test environment selenium does not exist on the local machine, configure the web application test environment selenium on the local machine by installing the Chrome browser, the corresponding version of the Google browser driver Chromedriver, the Python runtime environment, the selenium dependency library, and the pytorch dependency library on the local machine; S220, configure the Google browser driver Chromedriver path, the network address URL to be screenshotted, the image name to be screenshotted, the BMC IP address and SN list through the json configuration file; S230, pointing to a host information file for storing host information for which a BMC page screenshot is required through a json configuration file; S240, if a simulated event click is required, execute the simulated event click and jump to the automatic login interface, and perform the automated self-signed https certificate security verification operation; S250, switch to the login page based on webdriver, find the button ID in the page structure, automatically fill in the data to perform the login operation; S260, executing page jumps in sequence according to the configured routes. If a click event is configured, it is necessary to execute the click event after switching to the corresponding page; S270, executing a desktop screenshot operation to obtain a screenshot file.

4. The BMC page automatic screenshot method according to claim 3, characterized in that: In step S230: The host information includes at least the IP address, account number, password and serial number SN of the host BMC; The host information file contains at least one host information.

5. The BMC page automatic screenshot method according to claim 1, characterized in that: In step S400, The pages are redirected in sequence according to the configured routes. If a simulated event click is configured, the simulated event click needs to be executed after switching to the corresponding page. The simulated event click includes that the current page can be screenshotted and the current page needs to be clicked before the screenshot can be taken.

6. The BMC page automatic screenshot method according to claim 1, characterized in that: Before step S300, the process also includes cutting out the screenshot file to capture a central area of ​​the screenshot of a preset size.

7. The BMC page automatic screenshot method according to claim 1, characterized in that: In step S300, the yolov5 recognition model is built and trained through the following steps: S301. Configure the Python runtime environment and use the pip command to install the torch, torchvision, opencv-python, and matplotlib dependency packages; S302. Use the git command to clone the yolov5 repository, obtain the yolov5 model, and use pip to install all dependent packages; S303, for the feature extraction BackBone part of the yolov5 model, after each downsampling layer Conv, insert a CBAM module, wherein the CBAM module includes a channel attention part and a spatial attention part; S304, for the multi-scale feature fusion Neck part of the yolov5 model, the top-down path is to fuse the high-level large target features and the bottom-level small target features through upsampling (Upsample), and the bottom-up path is to fuse the bottom-level small target features and the high-level large target features through downsampling. All the fusions are connected to the C3 module; S305, for the detection head part of the yolov5 model, each detection head includes a bounding box prediction part and a classification prediction part; S306, training the adjusted yolov5 model based on the labeled data set, generating a data set configuration file, and finally obtaining the trained yolov5 model.

8. The BMC page automatic screenshot method according to claim 1, characterized in that: In step S300, The detection of whether the screenshot file meets the requirements is to detect whether there is a loaded image through a pre-built and pre-trained yolov5 recognition model.

9. A BMC page automatic screenshot device, used to execute the BMC page automatic screenshot method according to any one of claims 1 to 8, characterized in that: The BMC page automatic screenshot device is installed on a host computer, the host computer is provided with a Python installation and operation environment, the host computer is electrically connected to at least one server through a network device, and the BMC page automatic screenshot device includes: Configuration module, used to configure and store the host information file that needs the BMC page screenshot host information; The simulated event click module is used to execute simulated click events in the browser; Screenshot module, used to perform page screenshot operation and obtain screenshot file; The detection control module is used to cut out and detect the screenshot file until a qualified screenshot file is obtained; A storage module, used to store qualified screenshot files; The configuration module, the simulated event click module, the screenshot module, and the storage module are electrically connected to the detection control module respectively.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images