Sample library construction method, device and equipment of electric power vision large model and medium

Abstract

The invention discloses a sample library construction method, device and equipment of an electric power vision large model and a medium. The method comprises the following steps: detecting an inspection image through a detection model to obtain a first segmentation sample set; performing fine tuning training on a general image segmentation model according to the first segmentation sample set to obtain a special detection segmentation model; carrying out iterative optimization on the special detection segmentation model; and after iterative optimization is completed, segmenting and pasting scarce samples according to the special detection segmentation model to obtain expanded samples of the scarce samples. According to the embodiment of the invention, the number of samples of the electric power vision large model can be increased, and sample categories are balanced.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, equipment and medium for constructing a sample library of a large electric power vision model. Background Art

[0002] Large AI models usually need to process large amounts of complex data to achieve high performance and accuracy. As a key link in image recognition technology, the quality and scale of the data set are crucial to model training and performance improvement. Image recognition data sets refer to data sets used to train and test image recognition models. Data sets usually have the following concerns: First, the size and quality of the data set: A good data set should have a large enough scale and high-quality image data. Only in this way can the model be exposed to more scenes and more complex images, thereby improving its accuracy and generalization ability. Second, the categories and distribution of images in the data set: Images of different categories and distributions have different effects on model training and testing.

[0003] The power line images obtained from drone inspections in the power industry usually have problems such as low data quality, uneven category distribution, and huge amounts of labeled data in the data set, and cannot be directly used as samples for the power vision big model. Before training the power vision big model, it is necessary to balance and expand the categories of the images, which is very important for improving the accuracy and generalization ability of the power vision big model. Summary of the invention

[0004] The present invention provides a method, device, equipment and medium for constructing a sample library of a large electric power vision model, so as to expand the number of samples of the large electric power vision model and balance the sample categories.

[0005] According to one aspect of the present invention, a method for constructing a sample library of a large electric power visual model is provided, comprising:

[0006] The inspection image is detected by the detection model to obtain a first segmentation sample set;

[0007] Fine-tune and train a general image segmentation model according to the first segmentation sample set to obtain a dedicated detection segmentation model;

[0008] Iteratively optimizing the dedicated detection and segmentation model;

[0009] After the iterative optimization is completed, the scarce class samples are segmented and pasted according to the dedicated detection segmentation model to obtain expanded samples of the scarce class samples.

[0010] According to another aspect of the present invention, there is provided a sample library construction device for a large electric power visual model, comprising:

[0011] A segmentation sample acquisition module is used to detect the inspection image through a detection model to obtain a first segmentation sample set;

[0012] A general model fine-tuning module, used for fine-tuning the general image segmentation model according to the first segmentation sample set to obtain a dedicated detection segmentation model;

[0013] A dedicated model optimization module, used for iteratively optimizing the dedicated detection and segmentation model;

[0014] The scarce sample expansion module is used to segment and paste the scarce class samples according to the dedicated detection segmentation model to obtain expanded samples of the scarce class samples after the iterative optimization is completed.

[0015] According to another aspect of the present invention, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the method for constructing a sample library of a large electric power vision model according to any embodiment of the present invention is implemented.

[0016] According to another aspect of the present invention, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the sample library construction method of the power vision large model described in any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the sample library construction method of the electric power vision large model described in any embodiment of the present invention when executed.

[0018] The embodiment of the present invention proposes a hybrid detection system of multiple detection models and special detection segmentation models to achieve multi-model complementarity, continuously iterate to improve the performance of special detection segmentation models, and realize efficient research and development of special detection segmentation models and segmentation data sets; at the same time, with the help of high-performance special detection segmentation models, efficient and accurate scarce target segmentation and pasting are performed, the sample expansion efficiency is improved, and the number of expanded samples is greatly increased to meet the requirements of algorithm robustness.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 is a flow chart of a method for constructing a sample library of a large electric power visual model provided according to an embodiment of the present invention;

[0022] Figure 2A is a flow chart of a method for constructing a sample library of a large electric power visual model according to another embodiment of the present invention;

[0023] Figure 2B is a schematic diagram of a data processing flow of a first sub-packet provided according to another embodiment of the present invention

[0024] Figure 2C is a schematic diagram of a segmentation sample set generation process provided according to another embodiment of the present invention;

[0025] Figure 2D is a schematic diagram of a data processing flow of a second sub-packet provided according to another embodiment of the present invention;

[0026] Figure 3 It is a structural schematic diagram of a method for constructing a sample library of a large electric power visual model provided according to another embodiment of the present invention;

[0027] Figure 4 It is a schematic diagram of the structure of an electronic device implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] Figure 1 This is a flow chart of a method for constructing a sample library of a large electric power vision model provided by an embodiment of the present invention. This embodiment is applicable to the case where the samples required for training the large electric power vision model are expanded before training the large electric power vision model. The method can be executed by a sample library construction device for the large electric power vision model. The device can be implemented in the form of hardware and / or software. The device can be configured in an electronic device with corresponding data processing capabilities, such as a background server. Figure 1 As shown, the method includes:

[0031] S110 , detecting the inspection image using a detection model to obtain a first segmentation sample set.

[0032] S120. Fine-tune and train a general image segmentation model according to the first segmentation sample set to obtain a dedicated detection segmentation model.

[0033] S130: Iteratively optimize the dedicated detection and segmentation model.

[0034] S140 , after the iterative optimization is completed, the scarce class samples are segmented and pasted according to the dedicated detection segmentation model to obtain expanded samples of the scarce class samples.

[0035] Among them, the detection targets of the detection model include defects and equipment in the image; the detection and segmentation targets of the dedicated detection and segmentation model include defects and equipment in the image.

[0036] Specifically, the inspection images taken by the drone on the inspection route and after arriving at the site are collected as the data source for model training and sample expansion. The inspection images are inspected using the existing (equipment, defect) inspection models in the industry to determine the equipment and defects in the inspection images, and the inspection images are annotated and segmented according to the inspection results to obtain the first segmented sample set.

[0037] Existing open source segmentation detection models are usually suitable for conventional targets, such as people, vehicles, objects, etc., and have poor detection and recognition performance for industry-specific data. As an open source model, the general image segmentation model (Segment Anything Model) cannot detect and segment equipment and defects in accurate inspection images. It is necessary to first fine-tune the general image segmentation model based on the first segmentation sample set to obtain a dedicated detection segmentation model that can detect equipment and defects.

[0038] The dedicated detection and segmentation model is the core of subsequent sample expansion. Therefore, after obtaining the dedicated detection and segmentation model, it is necessary to compare the detection results of the dedicated detection and segmentation model with those of the existing detection models on the same inspection images, and perform iterative optimization accordingly to obtain a dedicated detection and segmentation model with detection accuracy that meets expectations.

[0039] When training a deep learning model, a series of methods are used to amplify the training data, but the number of augmentations for the scarce class samples still does not meet the robustness requirements of the detection algorithm, resulting in a low accuracy rate for the detection of scarce sample targets. After the iteration is completed, the present invention uses a dedicated detection segmentation model to segment the scarce class samples (images) to obtain rare defects and / or rare equipment in the scarce class samples, and then pastes them onto the sample images that originally did not have rare defects and / or rare faults to obtain expanded samples of the scarce class samples. By using a dedicated detection segmentation model to perform efficient and accurate defect and equipment segmentation, the sample expansion efficiency can be improved, and the number of expanded samples can be greatly increased to meet the robustness requirements of the algorithm.

[0040] The embodiment of the present invention proposes a hybrid detection system of multiple detection models and special detection segmentation models to achieve multi-model complementarity, continuously iterate to improve the performance of special detection segmentation models, and realize efficient research and development of special detection segmentation models and segmentation data sets; at the same time, with the help of high-performance special detection segmentation models, efficient and accurate scarce target segmentation and pasting are performed, the sample expansion efficiency is improved, and the number of expanded samples is greatly increased to meet the requirements of algorithm robustness.

[0041] Figure 2A This is a flowchart of a method for constructing a sample library of a large electric power visual model provided by another embodiment of the present invention. This embodiment is optimized and improved on the basis of the above embodiment. Figure 2A As shown, the method includes:

[0042] S210, sub-packaging the inspection image to obtain at least two sub-packets; detecting the inspection image in the first sub-packet using at least two detection models to obtain a first detection result.

[0043] S220: Input the first detection result and the inspection image in the first sub-package into a general image segmentation model to obtain a first segmentation sample set.

[0044] Among them, the training frameworks of the detection models are different.

[0045] Specifically, Figure 2B As shown, the inspection image is randomly divided into n sub-packets D = {D1, D2...D n}, n≥2, each sub-package contains a number of inspection images. The inspection images in the first sub-package D1 are selected, and the inspection images are detected using the existing landmark detection model, which includes the detection model Ma and the detection model Mb obtained by training with two different frameworks.

[0046] If the same target appears at the same position in a single image and the confidence level is greater than or equal to the first threshold (for example, 0.5), it is initially assumed to be the target and automatically annotated to form an automatically annotated data set B. If the same defect appears at the same position in a single image and the second threshold (for example, 0.3) is less than the confidence level and less than the first threshold, manual review and fine-tuning are performed, and the samples after review and fine-tuning are added to the annotated sample data set A. If the same target appears at the same position in a single image and the confidence level is less than or equal to the second threshold, it is placed in the inspection data set d. Figure 2C As shown, data sets A and B are integrated, and the overall data set is input into the general image segmentation model, which is prompted to segment the image according to the annotation information. The segmentation results are then manually reviewed and fine-tuned to form the first segmentation sample set. By building a semi-automatic and efficient data annotation system for a large visual model, designing mixed detection of multiple existing detection models, and coordinating confidence screening strategies to achieve automatic screening of targets, target screening and annotation can be quickly achieved.

[0047] S230: Fine-tune and train a general image segmentation model according to the first segmentation sample set to obtain a dedicated detection segmentation model.

[0048] S240. Detect the inspection image in the second sub-package using the at least two detection models to obtain a second detection result; and detect the inspection image in the second sub-package using the dedicated detection segmentation model to obtain a third detection result.

[0049] S250, completing the third detection result according to the second detection result; inputting the three detection results and the inspection image in the second sub-package into a dedicated detection segmentation model to obtain a second segmentation sample set.

[0050] S260: Iteratively optimize the dedicated detection segmentation model according to the second segmentation sample set and the first segmentation sample set.

[0051] Specifically, Figure 2DAs shown, the detection model Ma, the detection model Mb and the dedicated segmentation detection model Mc are used simultaneously to perform target detection on the inspection image in the second sub-package. The second detection result (Bbox_ab) and the third detection result (Segment_c) are obtained. The recognition accuracy of the dedicated segmentation detection model is limited at this time, and it is very likely that the target will be missed. It is necessary to supplement the third detection result with the second detection result. The supplemented third detection result is then marked on the inspection image, and the inspection image is input into the segmentation model, prompting it to segment the image according to the marked information, and then the segmentation result is manually reviewed and fine-tuned to form a second segmentation sample set. The dedicated detection segmentation model is fine-tuned and trained using the second segmentation sample set and the first segmentation sample set to complete one iteration of the dedicated detection segmentation model.

[0052] Subsequent iterations and upgrades are performed in sequence using the inspection images in other sub-packages (D3, ..., Dn). The specific iteration process refers to the iteration process using the D2 sub-package, which will not be repeated here. When the iteration reaches a certain level, a dedicated detection segmentation model is used to predict the segmentation of the automatically labeled segmentation data set. Compared with the previous segmentation data set, non-existent targets will be supplemented and manually verified. After multiple rounds of data supplementation and annotation, a high-quality segmentation data set is finally formed, and the dedicated detection segmentation model is further iterated on the updated data set to obtain a high-performance target detection segmentation model.

[0053] Design multiple existing detection models for hybrid detection, and use confidence screening strategies to achieve automated screening and labeling of targets. With the general segmentation model, only one annotation is needed to completely segment instances, and no additional training or fine-tuning is required, so it can be used on a new dataset. This semi-automatic labeling system can greatly reduce the workload of manual labeling. Some data require manual verification, which improves the efficiency of the entire data processing process. In the later stage, optimize the semi-automatic labeling system of the segmentation dataset, use the segmentation dataset to fine-tune the general segmentation model, and form the first generation of dedicated detection segmentation model. Use existing detection models and dedicated detection segmentation models for hybrid detection to upgrade the semi-automatic labeling system, achieve multi-model complementarity, and continuously iterate to improve the detection performance of the dedicated detection segmentation model, realizing efficient research and development and processing of target detection segmentation models and segmentation datasets.

[0054] Optionally, the completing the third detection result according to the second detection result includes:

[0055] If a target exists in the first detection result but does not exist in the second detection result and the target meets the completion condition, the target is placed in the second detection result.

[0056] For details, please refer to Figure 2D, compare the target Bbox_ab of the same type and same position in the first detection result with the detection confidence >= the first threshold with the target Segment_c in the second detection result with the detection confidence >= the first threshold. Put the target Bbox_ab that does not exist in Segment_c into Segment_c.

[0057] S270, selecting defect-free or minority-class defect samples as background samples; performing detection and segmentation on the scarce class samples through a dedicated detection and segmentation model to obtain scarce targets.

[0058] S280, detecting and segmenting the background samples through a dedicated detection and segmentation model to obtain a background image; pasting the scarce target to the background image to obtain an expanded sample of the scarce class sample.

[0059] Specifically, select scarce class samples with sparse sample distribution and use the model for segmentation prediction, select defect-free or few-class defect samples as background sample sets and use the model for segmentation prediction. The former obtains scarce targets, including scarce equipment and / or scarce defects, and the latter obtains a background image. Randomly select 1 scarce target, randomly select a background image with no device area covering special equipment, and then paste the scarce target at a specific position of the background image to form a new expanded sample. Repeat this process many times to eventually achieve the generation of a large number of scarce samples and balance between sample classes. In addition, before pasting the scarce target to the background image, it can also be randomly processed, including random set transformation, random color transformation, and random noise addition, to increase the richness of the scarce target.

[0060] Optionally, before pasting the scarce target to the background image to obtain an expanded sample of the scarce class sample, the method further includes:

[0061] Based on one or more preset random processing methods, the scarce target is randomly processed; wherein the random processing method includes: random set transformation, random color transformation and random noise addition.

[0062] Specifically, random set transformations include horizontal flipping, special types of vertical flipping, and jitter scaling; random color transformations include brightness changes, contrast changes, and saturation changes; random noise includes: Gaussian noise, salt and pepper noise, Poisson noise, speckle noise, exponential noise, and uniform noise.

[0063] The embodiment of the present invention proposes a method for autonomously generating new sample data of a large electric power vision model, forming a large visual model data generation system. It realizes automatic image segmentation targets, and automatically completes background filling and new sample generation. Fundamentally, a systematic data generation and augmentation is formed for scarce samples, starting from the samples to ultimately achieve the diversity and balanced distribution of large model algorithm samples, thereby improving the robustness of downstream models and improving target detection rates.

[0064] Based on the above embodiment, optionally, the method further includes:

[0065] Inputting the unlabeled inspection image into the dedicated detection segmentation model to obtain the detection result output by the detection segmentation model;

[0066] The unclassified inspection image is marked according to the detection result.

[0067] The marking content includes the device type and / or defect type.

[0068] Specifically, in addition to being used to segment scarce samples, the dedicated detection segmentation model can also be used to mark inspection images because it has high-precision detection capabilities. Unclassified inspection images are input into the dedicated detection segmentation model, which detects the equipment and defects in the inspection images and outputs the corresponding detection results. The detection results record whether there are equipment or defects in the inspection images, as well as the specific equipment type or defect type. Subsequently, the information of the detection results is marked into the inspection images to complete the automatic marking of the inspection images, reducing the huge workload caused by manual labeling, improving the efficiency of the data closed-loop processing process, and the efficient development of dedicated detection segmentation models.

[0069] Figure 3 A schematic diagram of a sample library construction device for a large electric power visual model provided by another embodiment of the present invention. Figure 3 As shown, the device comprises:

[0070] The segmentation sample acquisition module 310 is used to detect the inspection image through the detection model to obtain a first segmentation sample set;

[0071] A general model fine-tuning module 320, configured to perform fine-tuning training on a general image segmentation model according to the first segmentation sample set to obtain a dedicated detection segmentation model;

[0072] A dedicated model optimization module 330, used for iteratively optimizing the dedicated detection segmentation model;

[0073] The scarce sample expansion module 340 is used to segment and paste the scarce class samples according to the dedicated detection segmentation model to obtain the expanded samples of the scarce class samples after the iterative optimization is completed.

[0074] The sample library construction device of the electric power visual large model provided in the embodiment of the present invention can execute the sample library construction method of the electric power visual large model provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0075] Optionally, the segmentation sample acquisition module 310 includes:

[0076] A subpackaging unit, used for subpackaging the inspection image to obtain at least two subpacks;

[0077] A first detection unit is used to detect the inspection image in the first sub-packet by using at least two detection models to obtain a first detection result; the training frameworks of the detection models are different from each other;

[0078] The first segmentation unit is used to input the first detection result and the inspection image in the first sub-packet into a general image segmentation model to obtain a first segmentation sample set.

[0079] Optionally, the dedicated model optimization module 330 includes:

[0080] A second detection unit, configured to detect the inspection image in the second sub-package by using the at least two detection models to obtain a second detection result;

[0081] The third detection unit detects the inspection image in the second sub-package by using the dedicated detection segmentation model to obtain a third detection result;

[0082] A completion unit, configured to complete the third detection result according to the second detection result;

[0083] A second segmentation unit, used for inputting the three detection results and the inspection image in the second sub-package into a dedicated detection segmentation model to obtain a second segmentation sample set;

[0084] An optimization unit is used to iteratively optimize the dedicated detection segmentation model according to the second segmentation sample set and the first segmentation sample set.

[0085] Optionally, the scarce sample expansion module 340 includes:

[0086] A background selection unit is used to select defect-free or defect-less samples as background samples;

[0087] The target segmentation unit is used to detect and segment the scarce class samples through a dedicated detection and segmentation model to obtain the scarce targets;

[0088] A background segmentation unit, used to detect and segment the background sample through a dedicated detection and segmentation model to obtain a background image;

[0089] The target pasting unit is used to paste the scarce target to the background image to obtain an expanded sample of the scarce class sample.

[0090] Optionally, the scarce sample expansion module 340 further includes:

[0091] A random processing unit is used to perform random processing on the scarce target based on one or more preset random processing methods; wherein the random processing methods include: random set transformation, random color transformation and random noise.

[0092] Optionally, the detection targets of the detection model include defects and equipment in the image.

[0093] Optionally, the detection and segmentation targets of the dedicated detection and segmentation model include defects and devices in the image.

[0094] Optionally, the device further comprises:

[0095] A detection module, used for inputting an unlabeled inspection image into the dedicated detection segmentation model to obtain a detection result output by the detection segmentation model;

[0096] A marking module is used to mark the unclassified inspection image according to the detection result.

[0097] Optionally, the tag content includes device type and / or defect type.

[0098] The sample library construction device for the electric power visual large model further described can also execute the sample library construction method for the electric power visual large model provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0099] Figure 4 A schematic diagram of the structure of an electronic device 40 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0100] like Figure 4As shown, the electronic device 40 includes at least one processor 41, and a memory connected to the at least one processor 41, such as a read-only memory (ROM) 42, a random access memory (RAM) 43, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 41 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 42 or the computer program loaded from the storage unit 48 to the random access memory (RAM) 43. In the RAM 43, various programs and data required for the operation of the electronic device 40 can also be stored. The processor 41, the ROM 42, and the RAM 43 are connected to each other through a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0101] A number of components in the electronic device 40 are connected to the I / O interface 45, including: an input unit 46, such as a keyboard, a mouse, etc.; an output unit 47, such as various types of displays, speakers, etc.; a storage unit 48, such as a disk, an optical disk, etc.; and a communication unit 49, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 49 allows the electronic device 40 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0102] The processor 41 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 41 executes the various methods and processes described above, such as a sample library construction method for a large power vision model.

[0103] In some embodiments, the sample library construction method of the electric power visual large model can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 48. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 40 via the ROM 42 and / or the communication unit 49. When the computer program is loaded into the RAM 43 and executed by the processor 41, one or more steps of the sample library construction method of the electric power visual large model described above can be performed. Alternatively, in other embodiments, the processor 41 can be configured to execute the sample library construction method of the electric power visual large model in any other appropriate manner (for example, by means of firmware).

[0104] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0105] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0106] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0107] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0108] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0109] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0110] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0111] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for constructing a sample library of a large electric power visual model, characterized in that: The method comprises: The inspection image is detected by the detection model to obtain a first segmentation sample set; Fine-tune and train a general image segmentation model according to the first segmentation sample set to obtain a dedicated detection segmentation model; Iteratively optimizing the dedicated detection and segmentation model; After the iterative optimization is completed, the scarce class samples are segmented and pasted according to the dedicated detection segmentation model to obtain expanded samples of the scarce class samples.

2. The method according to claim 1, characterized in that The detecting of the inspection image by the detection model to obtain the first segmentation sample set comprises: Sub-packaging the inspection image to obtain at least two sub-packs; The inspection images in the first sub-package are detected by at least two detection models to obtain a first detection result; the training frameworks of the detection models are different from each other; The first detection result and the inspection image in the first sub-package are input into a general image segmentation model to obtain a first segmentation sample set.

3. The method according to claim 2, characterized in that The iterative optimization of the dedicated detection segmentation model comprises: Detecting the inspection images in the second sub-package using the at least two detection models to obtain a second detection result; Detect the inspection image in the second sub-package by using the dedicated detection segmentation model to obtain a third detection result; Completing the third detection result according to the second detection result; Input the three detection results and the inspection image in the second sub-package into a dedicated detection segmentation model to obtain a second segmentation sample set; The dedicated detection segmentation model is iteratively optimized according to the second segmentation sample set and the first segmentation sample set.

4. The method according to claim 3, characterized in that The completing the third detection result according to the second detection result comprises: If a target exists in the first detection result but does not exist in the second detection result and the target meets the completion condition, the target is placed in the second detection result.

5. The method according to claim 1, characterized in that The step of segmenting and pasting the scarce class samples according to the dedicated detection segmentation model to obtain the expanded samples of the scarce class samples includes: Select samples without defects or with few defects as background samples; Use a dedicated detection and segmentation model to detect and segment the scarce class samples to obtain the scarce targets; The background sample is detected and segmented by a dedicated detection and segmentation model to obtain a background image; Paste the scarce target to the background image to obtain an expanded sample of the scarce class sample.

6. The method according to claim 5, characterized in that Before pasting the scarce target to the background image to obtain an expanded sample of the scarce class sample, the method further includes: Based on one or more preset random processing methods, the scarce target is randomly processed; wherein the random processing method includes: random set transformation, random color transformation and random noise.

7. The method according to any one of claims 1 to 6, characterized in that: The detection targets of the detection model include defects and devices in the image.

8. The method according to claim 7, characterized in that The detection and segmentation targets of the dedicated detection and segmentation model include defects and devices in the image.

9. The method according to claim 8, characterized in that The method further comprises: Inputting the unlabeled inspection image into the dedicated detection segmentation model to obtain the detection result output by the detection segmentation model; The unclassified inspection image is marked according to the detection result.

10. The method according to claim 9, characterized in that The marking content includes the device type and / or defect type.

11. A sample library construction device for a large electric power visual model, characterized in that: The device comprises: A segmentation sample acquisition module is used to detect the inspection image through a detection model to obtain a first segmentation sample set; A general model fine-tuning module, used for fine-tuning the general image segmentation model according to the first segmentation sample set to obtain a dedicated detection segmentation model; A dedicated model optimization module, used for iteratively optimizing the dedicated detection and segmentation model; The scarce sample expansion module is used to segment and paste the scarce class samples according to the dedicated detection segmentation model to obtain expanded samples of the scarce class samples after the iterative optimization is completed.

12. The device according to claim 11, characterized in that The segmentation sample acquisition module comprises: A subpackaging unit, used for subpackaging the inspection image to obtain at least two subpacks; A first detection unit is used to detect the inspection image in the first sub-packet by using at least two detection models to obtain a first detection result; the training frameworks of the detection models are different from each other; The first segmentation unit is used to input the first detection result and the inspection image in the first sub-packet into a general image segmentation model to obtain a first segmentation sample set.

13. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for constructing a sample library of a large electric vision model according to any one of claims 1 to 10.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for constructing a sample library of a large electric power vision model according to any one of claims 1 to 10 when executed.

Images