Machine room operation management system and management method

Abstract

The invention discloses a machine room operation management system and management method. The system comprises a data acquisition module, a voice interaction module and an intelligent analysis module, and the data acquisition module is connected with the voice interaction module and the intelligent analysis module. The data acquisition module is used for acquiring image information of an operation area acquired by the wearable operation equipment and sending the image information to the intelligent analysis module; the intelligent analysis module is used for detecting the image information by adopting a pre-trained target detection model, determining an operation action displayed in the image information, generating indication information under the condition that the operation action accords with an operation standard, and sending the indication information to the voice interaction module, the operation standard is used for representing a standard operation action, and the voice interaction module is used for receiving the indication information. The indication information is used for indicating the next operation action needing to be executed by the operator; and the voice interaction module is used for generating first voice information based on the indication information.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and more specifically, to a computer room operation management system and management method. Background Art

[0002] Currently, the management methods for computer room personnel operations are limited. Management primarily relies on on-site personnel or surveillance cameras within the computer room, resulting in significant waste of labor and time. It also fails to provide timely response or feedback to irregular on-site operations or the operating environment, making real-time management of operations and the environment impossible. Furthermore, due to non-standardized operating procedures, complex and unclear maintenance procedures, or a lack of relevant maintenance knowledge, maintenance personnel struggle to strictly follow standardized procedures. This can lead to duplication of effort and inefficient operations, and even equipment crashes, service interruptions, and data loss, increasing the difficulty of troubleshooting and the time required to resolve issues. Insufficient traceability of operating procedures. Routine maintenance operations often only include simple paper records of maintenance content, without recording or even visually documenting the entire operating process. When problems arise, it is difficult to trace historical operations or the root cause of the problem, increasing the risk of safety incidents. Summary of the Invention

[0003] The embodiments of the present application provide a computer room operation management system and management method to at least solve the technical problem in the related art that computer room operation management mainly relies on monitoring images and manual work, resulting in low management efficiency.

[0004] According to one aspect of an embodiment of the present application, a computer room operation management system is provided, comprising: a data acquisition module, a voice interaction module and an intelligent analysis module, wherein the data acquisition module is connected to the voice interaction module and the intelligent analysis module respectively; the data acquisition module is used to collect image information of the operation area acquired by a wearable operation device, and send the image information to the intelligent analysis module; the intelligent analysis module is used to detect the image information using a pre-trained target detection model, determine the operation action displayed in the image information, and generate instruction information when the operation action meets the operation standard, and send the instruction information to the voice interaction module, wherein the operation standard is used to characterize the standard operation action, and the instruction information is used to indicate the next operation action that the operator needs to perform; the voice interaction module is used to generate a first voice message based on the instruction information.

[0005] Optionally, the intelligent analysis module includes: a target recognition unit and a data analysis unit; the target recognition unit is used to use a pre-trained target detection model to detect the target object in the image information, obtain a detection result, and send the detection result to the data analysis unit, wherein the target object includes at least one of the following: operating equipment, equipment port, equipment label, operating behavior, and wearing tool; the detection result is used to represent the detected target object; the data analysis unit is used to analyze the target object corresponding to the detection result to obtain an analysis result, and the analysis result is used to indicate whether the operating action meets the operating standard.

[0006] Optionally, the target recognition unit is further used to deploy the pre-trained target detection model, wherein the loss function of the target detection model is determined based on a standard intersection-over-union ratio and a positional relationship between vertices of the predicted box and vertices of the true box, and the standard intersection-over-union ratio is used to represent the intersection area of ​​the predicted box and the true box divided by the union area of ​​the predicted box and the true box.

[0007] Optionally, the data analysis unit is used to extract the image of the area where the target object is located from the image corresponding to the image information to obtain the target image, and convert the color model of the target image from the RGB model to the HSV model to obtain the converted image, and analyze the converted image to obtain an analysis result; the data analysis unit is also used to generate exception information when the analysis result indicates that the operation action does not meet the operation standard.

[0008] Optionally, the system also includes: an information management module and an exception reporting module, wherein the information management module is connected to the intelligent analysis module, and the exception reporting module is connected to the intelligent analysis module and the information management module respectively; the information management module is used to store the operating standards, operating personnel and operating equipment information; the exception reporting module is used to receive the exception information, generate alarm information based on the exception information, and send the exception information to the information management module, and send the alarm information to the voice interaction module.

[0009] Optionally, the data acquisition module includes: a voice acquisition unit and an image acquisition unit, wherein the voice acquisition unit is used to acquire second voice information input through the operating equipment, and the image acquisition unit is used to acquire image information of the operating area.

[0010] Optionally, the exception reporting module includes: an exception notification unit and a log recording unit, wherein the exception notification unit is used to send the exception information, and the log recording unit is used to record the exception information.

[0011] Optionally, the information management module includes: an operation information unit, a resource information unit and a user information unit, wherein the operation information unit is used to store the operation standards, operation applications and operation records, the resource information unit is used to store operation equipment information, the port of the operation equipment, the opposite device information and the location information of the operation equipment; the user information unit is used to store operation personnel information.

[0012] According to another aspect of an embodiment of the present application, a computer room operation management method is also provided, including: collecting image information of the operation area acquired by the operation equipment; using a pre-trained target detection model to detect the image information, determine the operation action displayed in the image information, and generate instruction information when the operation action meets the operation standard, wherein the operation standard is used to characterize the standard operation action, and the instruction information is used to indicate the next operation action that the operator needs to perform; generating a first voice message based on the instruction information.

[0013] Optionally, a pre-trained target detection model is used to detect the image information to determine the work action displayed in the image information, including: using a pre-trained target detection model to detect the target object in the image information to obtain a detection result, wherein the target object includes at least one of the following: work equipment, equipment port, equipment label, work behavior, and wearing tool, and the detection result is used to represent the detected target object. The loss function of the target detection model is determined based on the standard intersection-over-union ratio and the positional relationship between the vertices of the predicted box and the vertices of the real box. The standard intersection-over-union ratio is used to represent the intersection area of ​​the predicted box and the real box divided by the union area of ​​the predicted box and the real box; the target object corresponding to the detection result is analyzed to obtain an analysis result, and the analysis result is used to indicate whether the work action meets the work standard.

[0014] Optionally, the target object corresponding to the detection result is analyzed to obtain an analysis result, including: cutting out an image of the area where the target object is located from the image corresponding to the image information to obtain a target image, and converting the color model of the target image from an RGB model to an HSV model to obtain a converted image, and analyzing the converted image to obtain an analysis result.

[0015] Optionally, the method further includes: generating abnormal information when the analysis result indicates that the operation action does not meet the operation standard; storing the abnormal information, and generating alarm information.

[0016] According to another aspect of the embodiment of the present application, a computer device is also provided, including: a memory and a processor, wherein the memory is used to store program instructions; the processor is connected to the memory and is used to execute the above-mentioned computer room operation management method.

[0017] According to another aspect of the embodiments of the present application, a computer program product is provided, including computer instructions, which implement the above-mentioned computer room operation management method when executed by a processor.

[0018] In the embodiment of the present application, a data acquisition module, a voice interaction module and an intelligent analysis module are used, wherein the data acquisition module is connected to the voice interaction module and the intelligent analysis module respectively; the data acquisition module is used to collect image information of the working area obtained by the wearable working equipment, and send the image information to the intelligent analysis module; the intelligent analysis module is used to detect the image information using a pre-trained target detection model, determine the working action displayed in the image information, and generate instruction information when the working action meets the working standard, and send the instruction information to the voice interaction module, wherein the working standard Used to represent standard operating actions, the instruction information is used to indicate the next operating action that the operator needs to perform; the voice interaction module is used to generate a first voice message based on the instruction information, detect the image information of the operating area obtained by the wearable operating device through a pre-trained target detection model, and use the intelligent analysis module to analyze the detected image, and complete the standardized management of the computer room operation based on the analysis results, thereby achieving the purpose of efficient management of the computer room operation process, thereby achieving the technical effect of improving the efficiency of computer room operation management, and thus solving the technical problem of low management efficiency in related technologies that mainly rely on monitoring images and manual computer room operation management. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 This is a structural block diagram of a computer room operation management system according to an embodiment of the present application;

[0021] Figure 2 is a structural block diagram of a wearable working device according to an embodiment of the present application;

[0022] Figure 3 is a schematic diagram of identifying a target object according to an embodiment of the present application;

[0023] Figure 4 is a flow chart of analyzing a target object by a data analysis unit according to an embodiment of the present application;

[0024] Figure 5 This is a flow chart of a fiber replacement operation according to an embodiment of the present application;

[0025] Figure 6 This is a flowchart of an operation process detection according to an embodiment of the present application;

[0026] Figure 7 This is a schematic diagram of an operation process detection interface according to an embodiment of the present application;

[0027] Figure 8 This is a hardware structure block diagram of a computer terminal for implementing a computer room operation management method according to an embodiment of the present application;

[0028] Figure 9 This is a flow chart of a computer room operation management method according to an embodiment of the present application;

[0029] Figure 10 This is a structural diagram of a computer room operation management device according to an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] The information collected in the embodiments of the present application is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with the relevant laws, regulations and standards of the relevant regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or reject the automated decision results; if the user chooses to reject, the expert decision-making process will be entered.

[0033] In order to solve the problems existing in the related art, the embodiment of the present application provides a computer room operation management system, such as Figure 1 As shown, it includes: a data acquisition module, a voice interaction module and an intelligent analysis module, wherein the data acquisition module is connected to the voice interaction module and the intelligent analysis module respectively; the data acquisition module is used to collect image information of the working area obtained by the wearable working equipment, and send the image information to the intelligent analysis module; the intelligent analysis module is used to use a pre-trained target detection model to detect the image information, determine the working action displayed in the image information, and generate instruction information when the working action meets the working standard, and send the instruction information to the voice interaction module, wherein the working standard is used to characterize the standard working action, and the instruction information is used to indicate the next working action that the operator needs to perform; the voice interaction module is used to generate a first voice message based on the instruction information.

[0034] It should be noted that in actual application scenarios, after the image information of the working area is acquired by the wearable working device, the data acquisition module collects the image information from the wearable working device.

[0035] Take wearable working equipment as head-mounted intelligent working equipment as an example. Figure 2 A schematic diagram of the structure of a head-mounted intelligent working device is shown in FIG. Figure 2 As shown, the head-mounted intelligent operation device includes: a microcontroller development board (for example: ESP32-CAM), a camera module, a speaker module, a microphone module, a 4G module (communication module), and the head-mounted intelligent operation device is connected to the back-end server and the network server (Web server) through the Internet of Things platform.

[0036] It is understandable that the head-mounted intelligent working device obtains image information of the working area through the camera module and plays the first voice information, such as alarm information and prompt information, through the speaker module.

[0037] In some embodiments of the present application, the intelligent analysis module includes: a target recognition unit and a data analysis unit; the target recognition unit is used to use a pre-trained target detection model to detect the target object in the image information, obtain a detection result, and send the detection result to the data analysis unit, wherein the target object includes at least one of the following: operating equipment, equipment port, equipment label, operating behavior, and wearing tool; the detection result is used to represent the detected target object; the data analysis unit is used to analyze the target object corresponding to the detection result to obtain an analysis result, and the analysis result is used to indicate whether the operating action meets the operating standard.

[0038] In some embodiments of the present application, the target recognition unit is further used to deploy the pre-trained target detection model, wherein the loss function of the target detection model is determined based on a standard intersection-over-union ratio and a positional relationship between the vertices of the predicted box and the vertices of the real box, and the standard intersection-over-union ratio is used to represent the intersection area of ​​the predicted box and the real box divided by the union area of ​​the predicted box and the real box.

[0039] Optionally, the data analysis unit is used to extract the image of the area where the target object is located from the image corresponding to the image information to obtain the target image, and convert the color model of the target image from the RGB model to the HSV model to obtain the converted image, and analyze the converted image to obtain an analysis result; the data analysis unit is also used to generate abnormal information when the analysis result indicates that the operation action does not meet the operation standard.

[0040] The system further includes: an information management module and an exception reporting module, wherein the information management module is connected to the intelligent analysis module, and the exception reporting module is connected to the intelligent analysis module and the information management module respectively; the information management module is used to store the operating standards, operating personnel and operating equipment information; the exception reporting module is used to receive the exception information, generate alarm information based on the exception information, and send the exception information to the information management module and the alarm information to the voice interaction module. The data acquisition module includes: a voice acquisition unit and an image acquisition unit, wherein the voice acquisition unit is used to collect the second voice information input through the operating equipment, and the image acquisition unit is used to collect image information of the operating area. The exception reporting module includes: an exception notification unit and a log recording unit, wherein the exception notification unit is used to send the exception information, and the log recording unit is used to record the exception information. The information management module includes: an operation information unit, a resource information unit and a user information unit, wherein the operation information unit is used to store the operation standards, operation applications and operation records, the resource information unit is used to store the operation equipment information, the port of the operation equipment, the opposite device information and the location information of the operation equipment; the user information unit is used to store the operation personnel information.

[0041] It should be noted that the second voice information may be voice information generated in the working area during the working process, for example, voice information of the operator.

[0042] Specifically, the object detection model is trained using a pre-acquired dataset, which includes multiple images containing target objects, such as equipment racks, equipment ports, equipment labels, personnel behavior, tools worn, and potential hazards. Labeling tools are then used to annotate the target objects' feature points, such as rack identifiers, port identifiers, equipment labels, key points on the operator's hands, tools worn, and potential hazards. Potential hazards include, but are not limited to, improper equipment operation, such as when operators fail to follow proper procedures or use inappropriate tools when replacing optical fibers or plugging or unplugging equipment. Lack of personal protective equipment: In the computer room, operators should wear appropriate protective equipment, such as insulating gloves and goggles. If operators fail to wear tools correctly, such behavior is marked as a potential hazard. Hazardous objects near equipment: For example, improperly placed items in the computer room, such as flammable and sharp objects, can pose a safety threat. These items, if detected in prohibited locations, are classified as potential hazards. Environmental issues, such as the environmental conditions within the computer room, which are critical to equipment operation, such as excessive humidity and temperature, excessive dust, and insufficient lighting, can also be detected through image analysis and become potential hazards. Non-operating personnel: Non-operating personnel entering the computer room or attempting to access equipment are also potential hazards. Damaged or abnormal equipment: The system can also identify the physical condition of equipment within the computer room. Damaged equipment, abnormal indicator lights, and haphazardly connected cables are also potential hazards.

[0043] The dataset is divided into training set, validation set, and test set in a ratio of 8:1:1, and the target confidence threshold is set to ensure the accuracy of the detection results and build a target detection model. Figure 3 A schematic diagram of a target detection model for identifying image information is shown, Figure 3 As shown, Figure 3 The range shown is the image framed by the minimum bounding box. Figure 3 The box with the middle phase position in the upper right corner is the prediction box. Figure 3 The box with the relative position at the lower left is the real box. The distance d1 between the upper left vertex and the distance d2 between the lower right vertex of the predicted box and the real box are calculated respectively. The loss function of the target detection model is determined based on d1 and d2. The loss function of the target detection model can be determined in the following way:

[0044]

[0045] L MPDIoU =1-MPDIoU;

[0046] Where A represents the area of ​​the predicted box, B represents the area of ​​the real box, A∩B represents the intersection area of ​​the predicted box and the real box, A∪B represents the union area of ​​the predicted box and the real box, d1 is the Euclidean distance between the upper left vertex of the predicted box and the real box, d2 is the Euclidean distance between the lower right vertex of the predicted box and the real box, w and h are the width and height of the minimum bounding box, Indicates the horizontal coordinate of the upper left vertex of the prediction box, Indicates the vertical coordinate of the upper left vertex of the prediction box, Indicates the horizontal coordinate of the upper left vertex of the real box, Indicates the vertical coordinate of the upper left vertex of the real box, Indicates the horizontal coordinate of the lower right vertex of the prediction box, Indicates the horizontal coordinate of the lower right vertex of the real box, Indicates the vertical coordinate of the lower right vertex of the prediction box, Indicates the vertical coordinate of the lower right vertex of the real box, L MPDIoU represents the loss function.

[0047] It can be understood that the minimum bounding box is a rectangular box with the smallest area that can completely cover multiple target objects.

[0048] Figure 4 The process of analyzing the target object by the data analysis unit is shown in FIG. Figure 4 As shown, the target recognition unit receives image information containing the target object and analyzes it using an open-source computer vision library, such as OpenCV (Open Source Computer Vision Library), to determine whether the personnel's behavior, the equipment they are operating, the tools they are wearing, and the environment they are in are consistent with the current operation type and meet the specifications. If any of these items do not match, a corresponding alarm and abnormality information reporting module is sent. For example, in the intelligent analysis module, the target recognition unit first receives the raw image from the image acquisition unit. It then uses an object detection model, such as the YOLOv8 algorithm, to perform object detection on the image, identifying features such as racks, ports, equipment labels, personnel behavior, tools they are wearing, and potential hazards. To improve accuracy and detection performance, the MPDIoU loss function is used. This step ensures the accuracy of target detection. Next, for targets requiring detailed information, such as rack identification, port identification, and equipment labels, the system uses OpenCV to extract the relevant areas and convert them to the HSV color space to prepare for subsequent text recognition. After preliminary image processing and object recognition, the target area (such as the device label) captured by OpenCV is sent to the PaddleOCR system for optical character recognition (OCR). PaddleOCR analyzes the text information in the captured area, which is important for understanding device details, confirming port numbers, etc.

[0049] Once the data analysis unit finds any mismatch - whether it is a mismatch between behavior and expected work processes, a discrepancy between the operating equipment and recorded information, or a safety hazard in the environment, it will immediately pass the abnormal information to the abnormal reporting module. The abnormal reporting module will then notify the management personnel and record the relevant events in the system log for subsequent investigation and improvement measures. If the data analysis unit determines that the work behavior and equipment operation are in compliance with the specifications, it will continue to convey the next operation instructions to the operator through the voice interaction module (using Pyttsx3 to convert text to speech) to ensure the smooth progress of the work process.

[0050] The conversion process of converting the RGB model to the HSV model is shown in the following formula:

[0051] V←max(R,G,B);

[0052]

[0053] IfH<0thenH←H+360.Onoutput0≤V≤1,0≤S≤1,0≤H≤360.

[0054] In the formula, V represents brightness, S represents saturation, and H represents hue. V←max(R,G,B) means that the maximum value of the three components in RGB is determined as V in HSV, min(R,G,B) represents the minimum value of the three components in RGB, IfH<0thenH←H+360.On means that if H is less than 0, then H is added by 360 degrees to ensure that H falls between 0 and 360 degrees. output0≤V≤1,0≤S≤1,0≤H≤360. means that in the output result, V and S are real numbers in the range of 0 to 1, and H is an angle between 0 and 360 degrees.

[0055] Taking the fiber replacement operation as an example, the specific process is as follows: Figure 5As shown, Step 1: Move to the relevant equipment: The operator must first reach the specific equipment location where the fiber needs to be replaced. This step ensures that the operator is working on the correct target equipment and avoids the risks of operating the wrong device. Step 2: Wear an insulating wristband: Before beginning the operation, the operator must wear an insulating wristband or appropriate insulating protective equipment. This is for safety reasons, preventing injuries from static electricity or electrical current when operating live equipment or interfaces. Step 3: Remove the relevant optical fiber: After confirming the optical fiber to be replaced, the operator must safely remove the old optical fiber according to regulations. This typically requires the use of appropriate tools, such as a fiber optic insertion and extraction tool, to avoid damaging the optical fiber interface or the optical fiber itself. Step 4: Replace the relevant optical fiber: After removing the old optical fiber, the operator should carefully insert the new optical fiber. Ensure that the optical fiber is correctly inserted into the corresponding port and inserted in accordance with regulations to avoid twisting or incomplete connection, thereby ensuring stable and efficient network connections.

[0056] The intelligent analysis module uses target recognition and data analysis to confirm whether the operator's behavior complies with the above process and whether the operation is performed on the correct device and port. If deviations from the standard behavior or operational errors are detected, the system immediately triggers an exception report, notifies management and records the event, and either stops or instructs the operator to correct the operation. The voice interaction module provides voice guidance during the operation to ensure that operators follow standardized procedures. For example, it reminds operators to wear insulating wristbands, confirms that they have reached the correct equipment, and provides confirmation and next step instructions at each operation step, thereby reducing human error and improving operation accuracy and safety.

[0057] Figure 6 Shows a detection process of an operation process, such as Figure 6As shown, the process includes the following steps: Step 1: Hand keypoint detection. The object detection model is used to identify key points on the hand (including but not limited to finger joints and fingertips), especially the first key points of the thumb and index finger (including but not limited to the joints and fingertips of the thumb and index finger), as these are often the primary contact points for fine manipulation. Step 2: Determine the location of the key points and analyze whether they are located in the fiber area. If the detection results show that the first key points of the thumb and index finger are indeed located in the fiber area, this indicates that the operator is performing fiber-related operations. Step 3: Calculate the intersection over union (IOU). The intersection over union (IOU) is calculated between the fiber area and multiple possible port areas. IOU is a metric that measures the degree of overlap between two areas, with a value between 0 and 1. If there is no overlap at all, the IOU is 0; if there is complete overlap, the IOU is 1. Step 4: Filter the port area set. When the IOU is not equal to 0, it indicates that at least one port area overlaps with the currently operated fiber area. All port areas that intersect with the fiber area are filtered out. Step 5: Determine the operation port. In the selected port area set, identify which port areas contain the most hand key points, which means that the operator's hand is pointing to or touching one of these port areas. Finally, the port area touched the most times by the hand key points will be determined as the current operation port. Figure 7 As shown, a worker is operating the equipment. The inspection interface diagram includes a rack plate, a device label, and the worker's hand. Different colored dots represent key points on the worker's fingers. The first key points of the thumb and index finger are concentrated on port 07, confirming that the port currently being operated is port 07. Step 6: Operation Information Verification: The identified current operation port information is compared with the operation information in the information management module to check for consistency. The operation information may include the device ID and port number of the planned fiber replacement. Step 7: Process Control and Exception Handling: If the identified port matches the target port in the operation information, the system uses the voice interaction module to guide the worker to the next step, such as confirming that the fiber is securely inserted. If the identified port does not match the operation information, the system deems an abnormal operation to have occurred and triggers an alarm mechanism. This may include notifying on-site supervisors or the remote monitoring center through the exception reporting module and issuing a voice warning to the worker, requiring them to reconfirm or adjust their operation.

[0058] In order to solve the problems existing in the related art, the embodiment of the present application provides a computer room operation management method, which can be run on Figure 8 In the computer terminal shown, the computer terminal is explained below.

[0059] The computer room operation management method embodiment provided in the embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 8 The hardware structure block diagram of a computer terminal for implementing the computer room operation management method is shown in FIG. Figure 8 As shown, the computer terminal 10 may include one or more (illustrated by 102a, 102b, ..., 102n in the figure) processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions connected via a wired and / or wireless network. In addition, it may also include: a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, and a BUS bus. It will be understood by those skilled in the art that Figure 8 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 8 More or fewer components than shown, or with Figure 8 Different configurations shown.

[0060] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry." The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuitry may be a single, independent processing module, or may be incorporated in whole or in part into any of the other components of the computer terminal 10. As described in the embodiments of the present application, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0061] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the computer room operation management method in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implementing the above-mentioned computer room operation management method. The memory 104 may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely located relative to the processor, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0062] The transmission module 106 is configured to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by the communications provider of the computer terminal 10. In one embodiment, the transmission module 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission module 106 may be a radio frequency (RF) module, which is configured to communicate with the Internet wirelessly.

[0063] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 .

[0064] It should be noted that, in some optional embodiments, the above Figure 8 The computer terminal shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. Figure 8 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computer terminal described above.

[0065] In the above-mentioned operating environment, an embodiment of the present application provides an embodiment of a computer room operation management method. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that here.

[0066] Figure 9 A method for managing computer room operations is shown. Figure 9 Shown, including:

[0067] Step S901, collecting image information of the working area acquired by the working equipment;

[0068] Step S902: Detect the image information using a pre-trained object detection model to determine the work action displayed in the image information. If the work action meets the work standard, generate instruction information, where the work standard is used to represent the standard work action, and the instruction information is used to indicate the next work action that the operator needs to perform.

[0069] Step S903: Generate first voice information based on the instruction information.

[0070] The above method detects the image information of the working area obtained by the wearable working equipment through the pre-trained target detection model, and uses the intelligent analysis module to analyze the detected image, and completes the standardized management of the computer room operation based on the analysis results, thereby achieving the purpose of efficient management of the computer room operation process, thereby achieving the technical effect of improving the efficiency of computer room operation management, and further solving the technical problem of low management efficiency in related technologies that mainly rely on monitoring images and manual work for computer room operation management.

[0071] In some embodiments of the present application, a pre-trained target detection model is used to detect the image information, and the specific steps for determining the work action displayed in the image information are as follows: a pre-trained target detection model is used to detect the target object in the image information to obtain a detection result, wherein the target object includes at least one of the following: work equipment, equipment port, equipment label, work behavior, and wearing tool. The detection result is used to represent the detected target object. The loss function of the target detection model is determined based on the standard intersection-over-union ratio and the positional relationship between the vertices of the predicted box and the vertices of the real box. The standard intersection-over-union ratio is used to represent the intersection area of ​​the predicted box and the real box divided by the union area of ​​the predicted box and the real box; the target object corresponding to the detection result is analyzed to obtain an analysis result, and the analysis result is used to indicate whether the work action meets the work standard.

[0072] In some embodiments of the present application, the specific steps of analyzing the target object corresponding to the detection result to obtain the analysis result are as follows: extracting an image of the area where the target object is located from the image corresponding to the image information to obtain a target image, converting the color model of the target image from an RGB model to an HSV model to obtain a converted image, and analyzing the converted image to obtain an analysis result. If the analysis result indicates that the operation action does not meet the operation standard, generating exception information; storing the exception information, and generating an alarm information.

[0073] Figure 10 A computer room operation management device is shown, which includes:

[0074] The acquisition module 110 is used to acquire image information of the working area obtained by the working equipment;

[0075] The detection module 112 is configured to detect the image information using a pre-trained object detection model, determine the work action displayed in the image information, and generate instruction information if the work action meets the work standard, wherein the work standard is used to represent the standard work action, and the instruction information is used to indicate the next work action that the operator needs to perform;

[0076] The generating module 114 is configured to generate first voice information based on the instruction information.

[0077] The above-mentioned computer room operation management device detects the image information of the operation area obtained by the wearable operation equipment through a pre-trained target detection model, and uses an intelligent analysis module to analyze the detected image, and completes the standardized management of the computer room operation based on the analysis results, thereby achieving the purpose of efficient management of the computer room operation process, thereby achieving the technical effect of improving the efficiency of computer room operation management, and further solving the technical problem of low management efficiency in related technologies that mainly rely on monitoring images and manual work for computer room operation management.

[0078] It should be noted that Figure 10 The computer room operation management device shown is used to execute Figure 2 The computer room operation management method shown in the figure, therefore, the relevant explanations in the above-mentioned computer room operation management method are also applicable to the computer room operation management device, and will not be repeated here.

[0079] An embodiment of the present application also provides a computer device, including: a memory and a processor, wherein the memory is used to store program instructions; the processor is connected to the memory and is used to execute the above-mentioned computer room operation management method.

[0080] The computer room operation management method executed by the above-mentioned computer equipment detects the image information of the operation area obtained by the wearable operation equipment through a pre-trained target detection model, and uses an intelligent analysis module to analyze the detected image, and completes the standardized management of the computer room operation based on the analysis results, thereby achieving the purpose of efficient management of the computer room operation process, thereby achieving the technical effect of improving the efficiency of computer room operation management, and further solving the technical problem of low management efficiency in related technologies that mainly relies on monitoring images and manual work for computer room operation management.

[0081] An embodiment of the present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the computer room operation management method in the present application.

[0082] The computer room operation management method implemented by the above-mentioned computer program product detects the image information of the operation area obtained by the wearable operation equipment through a pre-trained target detection model, and uses an intelligent analysis module to analyze the detected image. Based on the analysis results, the standardized management of the computer room operation is completed, thereby achieving the purpose of efficient management of the computer room operation process, thereby achieving the technical effect of improving the efficiency of computer room operation management, and further solving the technical problem of low management efficiency in related technologies that mainly rely on monitoring images and manual work for computer room operation management.

[0083] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0084] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0085] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0086] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0087] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0088] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0089] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A computer room operation management system, characterized in that: include: A data acquisition module, a voice interaction module and an intelligent analysis module, wherein the data acquisition module is connected to the voice interaction module and the intelligent analysis module respectively; The data acquisition module is used to collect image information of the working area obtained by the wearable working device and send the image information to the intelligent analysis module; The intelligent analysis module is configured to detect the image information using a pre-trained target detection model, determine the work action displayed in the image information, and generate instruction information if the work action meets the work standard, and send the instruction information to the voice interaction module, wherein the work standard is used to represent the standard work action, and the instruction information is used to indicate the next work action that the operator needs to perform; The voice interaction module is used to generate first voice information based on the indication information.

2. The system according to claim 1, wherein: The intelligent analysis module includes: a target recognition unit and a data analysis unit; The target recognition unit is configured to detect a target object in the image information using a pre-trained target detection model, obtain a detection result, and send the detection result to the data analysis unit, wherein the target object includes at least one of the following: operating equipment, equipment port, equipment label, operating behavior, and wearing tools; and the detection result is used to represent the detected target object; The data analysis unit is used to analyze the target object corresponding to the detection result to obtain an analysis result, and the analysis result is used to indicate whether the operation action meets the operation standard.

3. The system according to claim 2, characterized in that The target recognition unit is further used to deploy the pre-trained target detection model, wherein the loss function of the target detection model is determined based on a standard intersection-over-union ratio and a positional relationship between vertices of the predicted box and vertices of the true box, and the standard intersection-over-union ratio is used to represent the intersection area of ​​the predicted box and the true box divided by the union area of ​​the predicted box and the true box.

4. The system according to claim 2, wherein: The data analysis unit is configured to extract an image of the area where the target object is located from the image corresponding to the image information to obtain a target image, convert a color model of the target image from a red-green RGB model to an HSV model to obtain a converted image, and analyze the converted image to obtain an analysis result; The data analysis unit is further configured to generate abnormal information when the analysis result indicates that the operation action does not meet the operation standard.

5. The system according to claim 4, characterized in that The system further comprises: an information management module and an abnormality reporting module, wherein the information management module is connected to the intelligent analysis module, and the abnormality reporting module is connected to the intelligent analysis module and the information management module respectively; The information management module is used to store the operation standards, operation personnel and operation equipment information; The abnormality reporting module is used to receive the abnormality information, generate alarm information based on the abnormality information, send the abnormality information to the information management module, and send the alarm information to the voice interaction module.

6. The system according to claim 1, wherein: The data acquisition module includes: a voice acquisition unit and an image acquisition unit, wherein the voice acquisition unit is used to acquire second voice information input through the operating equipment, and the image acquisition unit is used to acquire image information of the operating area.

7. The system according to claim 5, characterized in that The abnormality reporting module includes: an abnormality notification unit and a log recording unit, wherein the abnormality notification unit is used to send the abnormality information, and the log recording unit is used to record the abnormality information.

8. The system according to claim 5, wherein: The information management module includes: an operation information unit, a resource information unit and a user information unit, wherein the operation information unit is used to store the operation standards, operation applications and operation records, the resource information unit is used to store the operation equipment information, the port of the operation equipment, the opposite device information and the location information of the operation equipment; the user information unit is used to store the operation personnel information.

9. A computer room operation management method, characterized in that: include: Collect image information of the working area obtained by the working equipment; Using a pre-trained object detection model to detect the image information, determine the work action displayed in the image information, and generate instruction information if the work action meets the work standard, wherein the work standard is used to represent the standard work action, and the instruction information is used to indicate the next work action that the operator needs to perform; First voice information is generated based on the instruction information.

10. The method according to claim 9, characterized in that The image information is detected using a pre-trained object detection model to determine the work action displayed in the image information, including: A pre-trained target detection model is used to detect target objects in the image information to obtain a detection result, wherein the target object includes at least one of the following: operating equipment, equipment port, equipment label, operating behavior, and worn tools. The detection result is used to represent the detected target object. The loss function of the target detection model is determined based on a standard intersection-over-union ratio and a positional relationship between vertices of a predicted box and vertices of a true box. The standard intersection-over-union ratio is used to represent the intersection area of ​​the predicted box and the true box divided by the union area of ​​the predicted box and the true box. The target object corresponding to the detection result is analyzed to obtain an analysis result, and the analysis result is used to indicate whether the operation action meets the operation standard.

11. The method according to claim 10, characterized in that Analyze the target object corresponding to the detection result to obtain an analysis result, including: The image of the area where the target object is located is cut out from the image corresponding to the image information to obtain the target image, and the color model of the target image is converted from the RGB model to the HSV model to obtain the converted image, and the converted image is analyzed to obtain the analysis result.

12. The method according to claim 10, characterized in that The method further comprises: generating abnormal information when the analysis result indicates that the operation action does not meet the operation standard; The abnormal information is stored and an alarm information is generated.

13. A computer device, characterized in that: include: A memory and a processor, wherein the memory is used to store program instructions; The processor is connected to the memory and is used to execute the computer room operation management method described in any one of claims 9 to 12.

14. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by the processor, the computer room operation management method described in any one of claims 9 to 12 is implemented.

Images