A device-oriented multi-source heterogeneous data identification and detection method

Abstract

The present disclosure provides a device-oriented multi-source heterogeneous data recognition and detection method and device, relating to the technical field of artificial intelligence. The method comprises collecting multi-source heterogeneous data; using a labeling tool to process the multi-source heterogeneous data to generate a multi-source heterogeneous data training set; training a YOLOv5 learning model through the multi-source heterogeneous data training set to obtain a target detection model and a model weight file; inputting the multi-source heterogeneous data into the target detection model to output detection information and obtain the attached information of the multi-source heterogeneous data. The technical scheme of the present disclosure can widely and real-timely detect multi-scene multi-source heterogeneous image data, reduce detection cost, combine big data iteration and upgrading, and realize controllable, safe and multi-source heterogeneous data detection application with different needs for different users through power and domain division in intelligent terminal application.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a method for identifying and detecting multi-source heterogeneous data from devices. Background Technology

[0002] Image recognition has provided people with increasingly rich practical functions, and its emergence has greatly changed the lifestyles, work methods, and leisure activities of modern people. People are paying more and more attention to the enormous convenience that artificial intelligence provides to life and work. However, the current application and practical functions of image recognition, detection, and other technologies can only be compatible with the recognition of mainstream items, and do not provide people with diverse and customized intelligent recognition and classification options. When recognizing non-mainstream and complex items, manual processing of the images to be detected and recognized is required in the early stages to standardize them and prepare for subsequent detection and recognition work. Image recognition, detection, and other technologies cannot achieve a complete image feature library classification index, let alone meet the recognition requirements of multi-source heterogeneous classification data. In particular, products or systems for efficient asset inventory and organization, as well as for using big data to capture, analyze, and confirm assets and provide derivative information, are currently still in the blank stage.

[0003] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0004] This disclosure provides a method and related equipment for identifying and detecting multi-source heterogeneous data of devices, which at least to a certain extent realizes the use of artificial intelligence technology to complete the classification index of pattern feature library, and meets the identification and detection requirements of multi-source heterogeneous classification data.

[0005] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0006] According to one aspect of this disclosure, a method for identifying and detecting multi-source heterogeneous data of devices is provided, comprising: collecting multi-source heterogeneous data; processing the multi-source heterogeneous data using an annotation tool to generate a multi-source heterogeneous data training set; training a YOLOv5 learning model using the multi-source heterogeneous data training set to obtain a target detection model and a model weight file; inputting the multi-source heterogeneous data into the target detection model and outputting detection information while simultaneously acquiring auxiliary information of the multi-source heterogeneous data.

[0007] Furthermore, the aforementioned multi-source heterogeneous data consists of pattern data of various classified devices such as servers, routers, switches, disks, U ports, cabinets, cables, and serial ports, collected from different devices.

[0008] Furthermore, the process of using annotation tools to process multi-source heterogeneous data includes using the image annotation tool Labellmg to annotate the multi-source heterogeneous data. The annotation information mainly includes the category information and coordinate location information of the multi-source heterogeneous data.

[0009] Furthermore, the YOLOv5 learning model includes an input layer, a backbone layer, a neck layer, and a prediction layer. The input layer employs Mosaic data augmentation, including adaptive anchor box calculation and adaptive image scaling. The backbone layer includes a Focus structure and two CSP (CrossStage Partial) structures to aggregate the multi-source heterogeneous data and form image features. The neck layer uses FPN (Feature Pyramid Networks) and PAN (Path Aggregation Network) structures to extract image features from the multi-source heterogeneous data and pass these features to the prediction layer. The prediction layer uses GIOU_Loss (Generalized intersectionover union Loss) as the bounding box loss function to predict the multi-source heterogeneous data to be detected based on the image features.

[0010] Furthermore, training the YOLOv5 learning model involves using a multi-scale sliding window with different anchors to train a weight file that meets the requirements.

[0011] Furthermore, the step of inputting multi-source heterogeneous data into the target detection model and outputting detection information while simultaneously acquiring auxiliary information of the multi-source heterogeneous data includes: the mobile terminal acquiring multi-source heterogeneous data in the real scene via a camera, compressing it to a size of 1*3*640*640; inputting the processed multi-source heterogeneous data into the mobile terminal containing the target detection model for recognition and detection; the mobile terminal outputting recognition and detection information, drawing the detection box in the multi-source heterogeneous data of the target asset, and acquiring auxiliary information of the multi-source heterogeneous data via a communication unit.

[0012] According to another aspect of this disclosure, a device for identifying and detecting multi-source heterogeneous data from devices is provided, comprising:

[0013] Multi-source heterogeneous data acquisition module, used to acquire multi-source heterogeneous data;

[0014] The multi-source heterogeneous data processing module is used to label and classify multi-source heterogeneous data to generate a multi-source heterogeneous data training set.

[0015] The multi-source heterogeneous data training module is used to build a YOLOv5 learning model, train and identify multi-source heterogeneous data training sets, and thus obtain a weight file.

[0016] The multi-source heterogeneous data detection module is used to identify and detect multi-source heterogeneous data to be detected using a target detection model, and outputs detection information and auxiliary information of the multi-source heterogeneous data.

[0017] Furthermore, the device for identifying and detecting multi-source heterogeneous data of equipment may further include the following modules:

[0018] The storage module is used to store detection information and ancillary information of multi-source heterogeneous data;

[0019] The control module is used to control the operation of the multi-source heterogeneous data acquisition module, the multi-source heterogeneous data processing module, the multi-source heterogeneous data training module, the multi-source heterogeneous data detection module, and the storage module.

[0020] According to another aspect of this disclosure, a device for identifying and detecting multi-source heterogeneous data from devices is provided, comprising:

[0021] At least one processor and a memory communicatively connected to the at least one processor; the memory stores commands executable by the at least one processor to enable the at least one processor to perform a device-oriented multi-source heterogeneous data identification and detection method as described in the first aspect above.

[0022] According to another aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements a device-oriented multi-source heterogeneous data identification and detection method as described in the first aspect above.

[0023] According to another aspect of this disclosure, a computer program product is provided, comprising a computer program stored on a computer-readable storage medium, the computer program including program instructions that, when executed by a processor, cause the processor to perform a device-oriented multi-source heterogeneous data identification and detection method as described in the first aspect above.

[0024] This disclosure provides a method and apparatus for identifying and detecting multi-source heterogeneous data in devices, comprising: collecting multi-source heterogeneous data; processing the multi-source heterogeneous data using an annotation tool to generate a multi-source heterogeneous data training set; training a YOLOv5 learning model using the multi-source heterogeneous data training set to obtain a target detection model and a model weight file; inputting the multi-source heterogeneous data into the target detection model and outputting detection information while simultaneously acquiring supplementary information of the multi-source heterogeneous data. Compared with existing technical solutions, this disclosure can automatically identify and detect a wide range of real-time collected multi-source heterogeneous image data from multiple scenarios and acquire relevant information from the multi-source heterogeneous data, saving the manual processing step of multi-source heterogeneous data. Combined with big data iterative upgrades and through weighted and domain-based access in smart terminal applications, it enables controllable, secure, and customized multi-source heterogeneous data identification and detection applications for different users with varying needs.

[0025] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0026] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0027] Figure 1 This is a flowchart of an exemplary embodiment of a method for identifying and detecting multi-source heterogeneous data of a device provided in this disclosure;

[0028] Figure 2 This is a schematic diagram of the overall framework structure of a method for identifying and detecting multi-source heterogeneous data of devices provided in an exemplary embodiment of this disclosure;

[0029] Figure 3 This is a schematic diagram of the structure of a YOLOv5 learning model for a device multi-source heterogeneous data identification and detection method provided in an exemplary embodiment of this disclosure;

[0030] Figure 4 This is a schematic diagram of the Focus structure of a method for identifying and detecting multi-source heterogeneous data of devices provided in an exemplary embodiment of this disclosure;

[0031] Figure 5 This is a schematic diagram of the CSP1_X structure of a method for identifying and detecting multi-source heterogeneous data of devices provided in an exemplary embodiment of this disclosure;

[0032] Figure 6 This is a schematic diagram of the CSP2_X structure of a method for identifying and detecting multi-source heterogeneous data of devices provided in an exemplary embodiment of this disclosure;

[0033] Figure 7 This is a schematic diagram of the CBL (two-dimensional convolution operation Conv + normalization BatchNormalization + nonlinear activation function Leaky_relu) and the Resunit re-aggregation module in the CSP structure of an exemplary embodiment of the present disclosure for a device multi-source heterogeneous data identification and detection method;

[0034] Figure 8 This is a schematic diagram of the process of constructing a YOLOv5 learning model for a device multi-source heterogeneous data identification and detection method provided by an exemplary embodiment of this disclosure;

[0035] Figure 9 This is a schematic diagram of an apparatus for identifying and detecting multi-source heterogeneous data of a device, provided in an exemplary embodiment of this disclosure;

[0036] Figure 10 This disclosure provides an exemplary embodiment of a device for identifying and detecting multi-source heterogeneous data. Detailed Implementation

[0037] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0038] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0039] The solution provided in this application can widely and in real-time detect heterogeneous image data from multiple sources across various scenarios, reducing detection costs. Combined with big data iterative upgrades, and through weighted and domain-based processing in smart terminal applications, it enables controllable, secure, and customized heterogeneous data detection applications for different users with varying needs. For ease of understanding, the following explanations of several terms used in this application are provided first.

[0040] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0041] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0042] Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in tasks such as target recognition, tracking, and measurement, and further performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR (Optical Character Recognition), video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D (3D) technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0043] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0044] The solutions provided in this application involve technologies such as image recognition in artificial intelligence, and are specifically illustrated through the following embodiments:

[0045] Figure 1 This is a flowchart of an exemplary embodiment of a method for identifying and detecting multi-source heterogeneous data from devices, comprising the following steps:

[0046] Step S1: Collect multi-source heterogeneous data;

[0047] Step S2: Use annotation tools to process multi-source heterogeneous data to generate a multi-source heterogeneous data training set. The annotation tools can use the dataset annotation software Labellmg to annotate the multi-source heterogeneous data. The annotation information mainly includes the category information and coordinate location information of the multi-source heterogeneous data.

[0048] Step S3: Train the YOLOv5 learning model using the multi-source heterogeneous data training set to obtain the object detection model and obtain the model weight file;

[0049] Step S4: Input the multi-source heterogeneous data into the target detection model and output the detection information while obtaining the auxiliary information of the multi-source heterogeneous data.

[0050] Figure 2 This application provides an exemplary embodiment of a method for identifying and detecting multi-source heterogeneous data from devices, along with a schematic diagram of its overall framework for practical applications, including the following steps:

[0051] Step S21: The mobile terminal obtains multi-source heterogeneous data from the real scene via the camera;

[0052] Step S22: The multi-source heterogeneous data is compressed to a size of 1*3*640*640;

[0053] Step S23: Input the processed multi-source heterogeneous data into a mobile terminal containing a target detection model for recognition and detection;

[0054] Step S24: The mobile terminal outputs recognition and detection information, draws the detection box on the multi-source heterogeneous data of the target asset, and obtains the auxiliary information of the multi-source heterogeneous data through the communication unit.

[0055] Figure 3 This application provides an exemplary embodiment of a YOLOv5 learning model in a device multi-source heterogeneous data identification and detection method, comprising an input end, a backbone end, a neck end, and a prediction end. The backbone end is used to aggregate the multi-source heterogeneous data and form image features. The neck end is used to extract the image features of the multi-source heterogeneous data and pass the image features to the prediction layer. The prediction end is used to predict the multi-source heterogeneous data to be detected based on the image features.

[0056] Input: Mosaic data augmentation references the CutMix data augmentation method and theoretically shares some similarities. While CutMix uses two images for stitching, Mosaic uses four. The implementation of Mosaic data augmentation involves: reading four images at a time; flipping, scaling, and changing the color gamut of each image; arranging them according to four directions; and combining the images and bounding boxes. Random scaling, cropping, and arrangement for stitching result in good detection performance for small targets. Mosaic data augmentation has several advantages: 1. Enriched dataset: Randomly using four images, scaling them, and then randomly distributing them for stitching greatly enriches the detection dataset. Random scaling, in particular, adds many small targets, improving the network's robustness. 2. Reduced GPU requirements: During training, Mosaic augmentation can directly compute data from four images, reducing the need for a large mini-batch size; a single GPU can achieve good results.

[0057] Adaptive Anchor Box Calculation: In the YOLOv5 algorithm, initial anchor boxes with set widths and heights are used for different datasets. During network training, the network outputs predicted boxes based on the initial anchor boxes, compares them with the ground truth boxes, calculates the difference, and then updates the network parameters iteratively. The YOLOv5 network adaptively calculates the optimal anchor box values ​​for different training sets during each training iteration.

[0058] Adaptive Image Scaling: In common object detection algorithms, images vary in length and width. Therefore, a common approach is to scale the original images to a standard size before feeding them into the detection network. In the YOLOv5 code, the `letterbox` function in the `datasets.py` file is modified to adaptively add minimal black borders to the original images. YOLOv5 commonly uses sizes such as 416*416 and 608*608; for example, scaling an 800*600 image. The specific steps are as follows:

[0059] Step 1: Calculate the scaling factor. The original scaled size is 416*416. Dividing both by the original image size gives two scaling factors: 0.52 and 0.69. Choose the smaller scaling factor.

[0060] Step 2: Calculate the scaled dimensions. Multiply both the width and height of the original image by the minimum scaling factor of 0.52, resulting in a width of 416 and a height of 312.

[0061] Step 3: Calculate the black border fill value. Subtract 312 from 416 to get 104, thus obtaining the original height that needs to be filled. Then, using NumPy's `np.mod` modulo operation, we get 8 pixels, and divide by 2 to obtain the fill value needed at both ends of the image height.

[0062] Backbone end: Focus structure diagram as shown below Figure 4 As shown, the key is the slicing operation. Taking the YOLOv5s structure as an example, the original 608*608*3 image is input into the Focus structure. Using the slicing operation, it is first transformed into a 304*304*12 feature map, and then after a convolution operation with 32 convolution kernels, it is finally transformed into a 304*304*32 feature map.

[0063] Another key structure in the backbone of the YOLOv5 detection algorithm is the CSP: YOLOv5 designs two CSP structures. Taking the YOLOv5 network as an example, the CSP1_X structure diagram is shown below. Figure 5 The diagram shown is for another CSP2_X architecture applied to the backbone network at the backbone end. Figure 6 The example shown applies to the Neck. The CBL and Resunit modules in the CSP structure, such as... Figure 7 As shown.

[0064] Neck: YOLOv5's Neck uses an FPN+PAN structure and adopts the CSP2 structure, which is inspired by CSPnet, to enhance the ability to fuse network features.

[0065] Prediction side:

[0066] Bounding box loss function: YOLOv5 uses GIOU_Loss as the loss function for bounding boxes. In object detection tasks, the evaluation metric for bounding boxes is IoU, which ranges between (0,1), has scale invariance, and can measure the degree of matching for various shapes.

[0067] definition:

[0068] Where A is the intersection of the prediction box and the ground truth box; B is the union of the prediction box and the ground truth box; and C is the minimum bounding rectangle of the prediction box and the target box.

[0069] GIoU loss focuses not only on overlapping regions but also on non-overlapping regions, solving the problem of not being able to assess the difference between non-overlapping boxes. When the predicted box and the target box completely overlap: GIoU = IoU = 1; when the predicted box and the target box do not overlap, GIoU decreases with increasing distance, approaching -1.

[0070] Non-maximum suppression (NMS): In the post-processing of object detection, NMS operations are often required for filtering many bounding boxes. YOLOv4 uses DIOU_nms based on DIOU_Loss, while YOLOv5 uses weighted NMS.

[0071] Before using the YOLOv5 detection algorithm to output the skin disease type, the YOLOv5 detection network should be built first. The specific process is as follows: Figure 8 As shown, the steps are as follows:

[0072] Step 1: Build the YOLOv5 framework;

[0073] Step 2: Install the relevant dependencies and configure the virtual environment;

[0074] Step 3: Prepare a multi-source heterogeneous dataset for training and obtain the required weight file. Specifically, modify the YAML file for the multi-source heterogeneous dataset and network parameters; modify the parameters in train.py; execute the train.py file to start training, obtain the object detection model and the model weight file.

[0075] Figure 9This application provides an exemplary embodiment of a device multi-source heterogeneous data identification and detection device 900, which includes, but is not limited to, the following modules: multi-source heterogeneous data acquisition module 910, multi-source heterogeneous data processing module 920, multi-source heterogeneous data training module 930, and multi-source heterogeneous data detection module 940.

[0076] Among them, the multi-source heterogeneous data processing module 910 is used to collect multi-source heterogeneous data;

[0077] The multi-source heterogeneous data processing module 920 is used to label and classify multi-source heterogeneous data to generate a multi-source heterogeneous data training set.

[0078] The multi-source heterogeneous data training module 930 is used to build a YOLOv5 learning model, train and identify multi-source heterogeneous data training sets, and thus obtain a weight file.

[0079] The multi-source heterogeneous data detection module 940 is used to identify and detect multi-source heterogeneous data to be detected using a target detection model, and output detection information and auxiliary information of the multi-source heterogeneous data.

[0080] Preferably, the device 900 for identifying and detecting multi-source heterogeneous data of equipment may further include a storage module 950 for storing detection information and ancillary information of multi-source heterogeneous data;

[0081] Optionally, the device 900 for identifying and detecting multi-source heterogeneous data may further include a control module 960 for controlling the operation of the multi-source heterogeneous data acquisition module 910, the multi-source heterogeneous data processing module 920, the multi-source heterogeneous data training module 930, the multi-source heterogeneous data detection module 940, and the storage module 950.

[0082] It should be noted that since the device for identifying and detecting multi-source heterogeneous data of a device in this embodiment is based on the same inventive concept as the method for identifying and detecting multi-source heterogeneous data of a device in any of the above embodiments, the corresponding content in the method embodiment is also applicable to this system embodiment, and will not be described in detail here.

[0083] Figure 10This application provides an exemplary embodiment of a device-oriented multi-source heterogeneous data identification and detection device 1000. The device-oriented multi-source heterogeneous data identification and detection device 1000 can be any type of terminal, such as a mobile phone, game console, tablet computer, e-book reader, smart glasses, MP4 (Moving Picture Experts Group Audio Layer IV) player, smart home device, AR (Augmented Reality) device, VR (Virtual Reality) device, and other mobile terminals. Alternatively, the device-oriented multi-source heterogeneous data identification and detection device 1000 can also be a personal computer (PC), such as a laptop computer and a desktop computer, etc.

[0084] Among them, the device 1000 for identifying and detecting multi-source heterogeneous data can be equipped with an application for providing…

[0085] Preferably, the device 1000 for identifying and detecting multi-source heterogeneous data includes: one or more processors 1010 and a memory 1020. Figure 10 Take the 1010 processor as an example.

[0086] Processor 1010 and memory 1020 can be connected via a bus or other means. Figure 10 Take a bus connection as an example.

[0087] Memory 1020, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the device 1000 for multi-source heterogeneous data identification and detection in this embodiment of the invention. Figure 9 The diagram shows a multi-source heterogeneous data acquisition module 910, a multi-source heterogeneous data processing module 920, a multi-source heterogeneous data training module 930, and a multi-source heterogeneous data detection module 940. The processor 1010 executes various functional applications and data processing of the device multi-source heterogeneous data identification and detection device 900 by running non-transitory software programs, instructions, and modules stored in the memory 1020, thereby implementing a device multi-source heterogeneous data identification and detection method according to the above method embodiment.

[0088] The memory 1020 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the device-oriented multi-source heterogeneous data identification and detection device 900. Furthermore, the memory 1020 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1020 may optionally include memory remotely located relative to the processor 1010, and these remote memories may be connected to the device-oriented multi-source heterogeneous data identification and detection device 900 via a network.

[0089] Optionally, the network connection described above can be a wireless network or a wired network using standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to a Local Area Network (LAN), Metropolitan Area Network (MAN), Wide Area Network (WAN), mobile, wired or wireless network, private network, or any combination of virtual private network. In some embodiments, technologies and / or formats including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network. Furthermore, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), and Internet Protocol Security (IPsec) can be used to encrypt all or some links. In other embodiments, custom and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0090] The one or more units are stored in the memory 1020, and when executed by the one or more processors 1010, they perform a device-oriented multi-source heterogeneous data identification and detection method according to any of the above method embodiments. For example, the method described above is executed. Figure 1 Method steps S1 to S4 in the text, Figure 2 Method steps S21 to S24 are implemented. Figure 9 The functions of modules 910-960 in the document.

[0091] This invention also provides a computer-readable storage medium storing computer-executable instructions that are executed by one or more processors, for example, by... Figure 10 One or more processors 1010 execute the above-described method embodiment of a device-oriented multi-source heterogeneous data identification and detection method, for example, executing the method described above. Figure 1 Method steps S1 to S4 in the text, Figure 2 Method steps S21 to S24 are implemented. Figure 9 The functions of modules 910-960 in the document.

[0092] This invention also provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions that, when executed by a computer, for example, are... Figure 10 One of the processors 1010 executes, causing the computer to perform a device-oriented multi-source heterogeneous data identification and detection method as described above. For example, executing the above-described method... Figure 1 Method steps S1 to S4 in the text, Figure 2 Method steps S21 to S24 are implemented. Figure 9 The functions of modules 910-960 in the document.

[0093] A program product for implementing the above-described method according to embodiments of the present invention is described. This product may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0094] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0095] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0096] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0097] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0098] Optionally, the application clients installed on different devices 900 may be the same, or the application clients installed on two devices 900 may be clients of the same type of application from different control system platforms. Depending on the terminal platform, the specific form of the application client may also differ; for example, the application client may be a mobile client, a PC client, or a World Wide Web (WWW) client.

[0099] Those skilled in the art will understand that the number of the aforementioned devices 900 can be more or less. For example, there may be only one terminal, or there may be dozens or hundreds of terminals, or even more. This application does not limit the number of terminals or the type of devices.

[0100] It should be noted that the above figures are merely illustrative representations of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is also readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0101] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: entirely hardware implementations, entirely software implementations (including firmware, microcode, etc.), or implementations combining hardware and software aspects, collectively referred to herein as “circuits,” “modules,” or “systems.”

[0102] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0103] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0104] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0105] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0106] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A method for identifying and detecting multi-source heterogeneous data from devices, characterized in that, include: Collect multi-source heterogeneous data; the multi-source heterogeneous data refers to the pattern data of servers, routers, switches, disks, U ports, cabinets, cables, serial ports and other classified devices collected from different devices; Annotation tools are used to process multi-source heterogeneous data to generate a multi-source heterogeneous data training set; The YOLOv5 learning model is trained using the multi-source heterogeneous data training set to obtain an object detection model and a model weight file. After inputting multi-source heterogeneous data into the target detection model, the model outputs detection information and simultaneously acquires auxiliary information from the multi-source heterogeneous data. The ancillary information is derived information of the identified classification device, including ancillary products or systems; The steps of inputting multi-source heterogeneous data into the target detection model and then outputting detection information while simultaneously acquiring auxiliary information of the multi-source heterogeneous data include: The mobile terminal acquires multi-source heterogeneous data from the real scene via its camera, and then compresses it into 1. 3 640 Size 640; The processed multi-source heterogeneous data is input into a mobile terminal containing a target detection model for recognition and detection; The mobile terminal outputs recognition and detection information, draws the detection box on the multi-source heterogeneous data of the target asset, and obtains the auxiliary information of the multi-source heterogeneous data through the communication unit.

2. The method for identifying and detecting multi-source heterogeneous data of devices according to claim 1, characterized in that, The process of using annotation tools to process multi-source heterogeneous data includes using the image annotation tool Labellmg to annotate the multi-source heterogeneous data. The annotation information includes the category information and coordinate location information of the multi-source heterogeneous data.

3. The method for identifying and detecting multi-source heterogeneous data of devices according to claim 1, characterized in that, The YOLOv5 learning model includes an input end, a backbone network end, a bottleneck end, and a prediction end; The input terminal uses a mosaic data augmentation method, including adaptive anchor box calculation and adaptive image scaling; The backbone network includes a focus structure and two cross-stage partial CSP structures, which are used to aggregate the multi-source heterogeneous data and form image features; The bottleneck Neck adopts a structure of Feature Pyramid Network (FPN) and Path Aggregation Network (PAN) to extract image features from the multi-source heterogeneous data and pass the image features to the prediction layer. The prediction end uses the generalized intersection loss GIOU_Loss as the bounding box loss function to predict the multi-source heterogeneous data to be detected based on image features.

4. The method for identifying and detecting multi-source heterogeneous data of devices according to claim 1, wherein training the YOLOv5 learning model includes using a multi-scale sliding window with different anchors to train a weight file that meets the requirements.

5. A device for identifying and detecting multi-source heterogeneous data from equipment, characterized in that, Includes the following modules: The multi-source heterogeneous data acquisition module is used to acquire multi-source heterogeneous data; the multi-source heterogeneous data refers to the pattern data of servers, routers, switches, disks, U ports, cabinets, cables, serial ports and other classified devices acquired by different devices. The multi-source heterogeneous data processing module is used to label and classify multi-source heterogeneous data to generate a multi-source heterogeneous data training set. The multi-source heterogeneous data training module is used to build a YOLOv5 learning model, train and identify multi-source heterogeneous data training sets, and thus obtain a weight file. The multi-source heterogeneous data detection module is used to identify and detect the multi-source heterogeneous data to be detected using a target detection model, and output detection information and auxiliary information of the multi-source heterogeneous data. The auxiliary information is derived information of the identified classification device, including auxiliary products or systems. The multi-source heterogeneous data detection module is used by mobile terminals to acquire multi-source heterogeneous data from real-world scenes via cameras, and then compresses it into 1... 3 640 The size is 640; the processed multi-source heterogeneous data is input into a mobile terminal containing a target detection model for recognition and detection; the mobile terminal outputs recognition and detection information, draws the detection box on the multi-source heterogeneous data of the target asset, and obtains the auxiliary information of the multi-source heterogeneous data through the communication unit.

6. The device for identifying and detecting multi-source heterogeneous data of equipment according to claim 5, characterized in that, It also includes the following modules: The storage module is used to store detection information and ancillary information of multi-source heterogeneous data; The control module is used to control the operation of the multi-source heterogeneous data acquisition module, the multi-source heterogeneous data processing module, the multi-source heterogeneous data training module, the multi-source heterogeneous data detection module, and the storage module.

7. An electronic device for identifying and detecting multi-source heterogeneous data from equipment, characterized in that, include: At least one processor and a memory for communicatively connecting to said at least one processor; The memory stores commands that can be executed by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the device-oriented multi-source heterogeneous data identification and detection method according to any one of claims 1 to 4.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the device-oriented multi-source heterogeneous data identification and detection method according to any one of claims 1 to 4.

Images