Image segmentation model training method and device applied to substation equipment
Through automated adjustment of prompt points and image enhancement processing, the problem of low training efficiency of image segmentation models in substation equipment and key components scenarios is solved, and efficient model training and adaptability improvement is achieved.
Patent Information
- Application Number
- CN202410695243.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-31
- Publication Date
- 2024-11-08
AI Technical Summary
In the prior art, the image segmentation model has low training efficiency in substation equipment and key components scenarios, especially in large-scale data sets or complex scenarios, and requires manual determination of prompt points, which is not efficient.
By automatically adjusting the prompt points, obtaining the image sample set of substation equipment, image enhancement processing is performed based on the equipment attribute characteristics, initial prompt points are determined, prompt points are corrected based on the initial segmentation results and sample annotation information, and finally training the image segmentation model based on the correction prompt points.
The refined training efficiency of the image segmentation model in substation equipment and key components scenarios is improved, and the automated training process without manual participation is realized, and the adaptability and generalization ability of the model is improved.
Smart Images

Figure CN118918322A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data processing, and in particular to an image segmentation model training method and device applied to substation equipment. Background Art
[0002] Substations are a key link in the power system, and their safe and stable operation is crucial to ensuring the overall efficiency of the power grid. As the power system transforms towards intelligence and automation, it is very important to automatically perceive and detect the equipment and equipment components in the substation scene. Therefore, it is necessary to fine-tune the image segmentation model so that the trained image segmentation model can identify specific equipment or specific equipment components.
[0003] In the process of fine-tuning the training of the image segmentation model, it is necessary to input the cue points while inputting the training sample images, so as to guide the image segmentation model to focus on specific areas in the image based on the cue points. Currently, cue points are usually determined manually, however, this static manual cueing method has the problem of low efficiency when facing large-scale data sets or complex scenes. Summary of the invention
[0004] The embodiment of the present invention provides an image segmentation model training method and device applied to substation equipment, so as to improve the refined training efficiency of the image segmentation model in the application scenario of segmenting substation equipment and key components of the equipment by automatically adjusting prompt points.
[0005] In a first aspect, an embodiment of the present invention provides an image segmentation model training method applied to substation equipment, the method comprising:
[0006] Acquire a substation equipment image sample set; wherein the substation equipment image sample set includes multiple groups of equipment image samples, each group of equipment image samples includes an equipment image and sample annotation information corresponding to the equipment image, and the equipment image includes at least one substation equipment;
[0007] For each of the equipment images, based on the attribute characteristics of the substation equipment in the equipment image, the equipment image is enhanced to obtain an image to be used corresponding to each of the equipment images;
[0008] For each of the images to be used, determining at least one initial cue point corresponding to the current image to be used, inputting the current image to be used and each of the initial cue points into the image segmentation model to be trained, and obtaining an initial segmentation result corresponding to the current image to be used;
[0009] Based on the initial segmentation result and the sample annotation information, adjusting the at least one initial prompt point corresponding to the current image to be used to obtain a modified prompt point;
[0010] The image segmentation model to be trained is trained based on each of the images to be used and the corresponding correction prompt points. When the segmentation result output by the image segmentation model to be trained meets the preset training end condition, the training of the image segmentation model to be trained is completed, and an image segmentation model corresponding to the substation equipment is obtained.
[0011] In a second aspect, an embodiment of the present invention further provides an image segmentation model training device applied to substation equipment, the device comprising:
[0012] A sample set determination module, used to obtain a substation equipment image sample set; wherein the substation equipment image sample set includes multiple groups of equipment image samples, each group of equipment image samples includes an equipment image and sample annotation information corresponding to the equipment image, and the equipment image includes at least one substation equipment;
[0013] An enhancement processing module, used to perform enhancement processing on each of the equipment images based on the attribute characteristics of the substation equipment in the equipment image, so as to obtain an image to be used corresponding to each of the equipment images;
[0014] An initial segmentation module, used to determine, for each of the images to be used, at least one initial cue point corresponding to the current image to be used, input the current image to be used and each of the initial cue points into the image segmentation model to be trained, and obtain an initial segmentation result corresponding to the current image to be used;
[0015] A cue point correction module, configured to adjust the at least one initial cue point corresponding to the current image to be used based on the initial segmentation result and the sample annotation information to obtain a corrected cue point;
[0016] The segmentation model training module is used to train the image segmentation model to be trained based on each of the images to be used and the corresponding correction prompt points. When the segmentation result output by the image segmentation model to be trained meets the preset training end condition, the training of the image segmentation model to be trained is completed, and an image segmentation model corresponding to the substation equipment is obtained.
[0017] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:
[0018] one or more processors;
[0019] a storage device for storing one or more programs,
[0020] When one or more programs are executed by one or more processors, the one or more processors implement an image segmentation model training method applied to substation equipment as described in any of the embodiments of the present invention.
[0021] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute an image segmentation model training method applied to substation equipment as described in any of the embodiments of the present invention.
[0022] The technical solution of the embodiment of the present invention is to obtain a substation equipment image sample set, wherein the substation equipment image sample set includes multiple groups of equipment image samples, each group of equipment image samples includes equipment images and sample annotation information corresponding to the equipment images, and the equipment images include at least one substation equipment, so that for each equipment image, based on the attribute characteristics of the substation equipment in the equipment image, the equipment image is enhanced to obtain a to-be-used image corresponding to each equipment image, and further, for each to-be-used image, at least one initial prompt point corresponding to the current to-be-used image is determined, and the current to-be-used image and each initial prompt point are input into the to-be-trained image segmentation model to obtain an initial segmentation result corresponding to the current to-be-used image, and then, based on the initial segmentation result and the sample annotation information, at least one initial prompt point corresponding to the current to-be-used image is adjusted to obtain a revised prompt point, and finally, based on each to-be-used image and the corresponding revised prompt point, the to-be-trained image segmentation model is trained, and when the segmentation result output by the to-be-trained image segmentation model meets the preset training end condition, the training of the to-be-trained image segmentation model is terminated to obtain an image segmentation model corresponding to the substation equipment. The technical solution of the present invention does not require human intervention throughout the entire process, and improves the efficiency of refined training of image segmentation models in application scenarios of segmenting substation equipment and key components of the equipment through automatic adjustment of prompt points. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required to describe the embodiments. Obviously, the drawings introduced are only drawings of a part of the embodiments to be described in the present invention, rather than all the drawings. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0024] Figure 1 A flow chart of an image segmentation model training method applied to substation equipment provided by an embodiment of the present invention;
[0025] Figure 2 A flowchart of another image segmentation model training method applied to substation equipment provided by an embodiment of the present invention;
[0026] Figure 3 A schematic diagram of the structure of an image segmentation model training device applied to substation equipment provided by an embodiment of the present invention;
[0027] Figure 4 The present invention is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0029] It should be noted that the terms "first preset condition", "second preset condition", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0030] Embodiment 1
[0031] Figure 1 A flowchart of an image segmentation model training method for substation equipment provided by an embodiment of the present invention is provided. This embodiment can be applied to the situation of fine-tuning the image segmentation model. The method can be executed by an image segmentation model training device for substation equipment, which can be implemented in the form of software and / or hardware, and the hardware can be an electronic device, such as a mobile terminal, a PC, or a server.
[0032] like Figure 1 As shown, the method includes:
[0033] S110: Obtain a substation equipment image sample set.
[0034] The substation equipment image sample set includes multiple groups of equipment image samples, each group of equipment image samples includes equipment images and sample annotation information corresponding to the equipment images, and the equipment images include at least one substation equipment. The equipment image can be a real-life image collected from an actual substation scene, or an image containing transformer equipment obtained from a specific database.
[0035] The sample annotation information may include the device overall annotation information with the substation device as a segmentation object and / or the device key component annotation information with the substation device key components as a segmentation object. For example, if a device image includes two substation devices, one of the substation devices as a segmentation area can be used as a mask image of the substation device as a whole, and the mask image of the substation device as a whole can be called the device overall annotation information; and several key components in the other substation device can be used as a segmentation area, and the mask images corresponding to these key components can be called the device key component annotation information.
[0036] S120 , for each device image, based on the attribute characteristics of the substation equipment in the device image, the device image is enhanced to obtain an image to be used corresponding to each device image.
[0037] The attribute characteristics of the substation equipment can be understood as the inherent attributes of the substation equipment. For example, the attribute characteristics of the substation equipment may include: substation equipment with complex spatial structure, substation equipment with simple spatial structure, substation equipment with a large number of metal reflective surfaces, etc. The image to be used is an image obtained after enhancing the equipment image.
[0038] In this embodiment, by enhancing the device image, the presentation image of the device under special conditions such as different viewing angles, occlusions, and different lighting environments can be simulated. The segmentation model is trained using the enhanced device image to improve the adaptability and generalization ability of the segmentation model.
[0039] Specifically, different enhancement processing methods can be set in advance for different attribute features, so as to determine the correspondence between the attribute features and the enhancement processing methods. On the basis of obtaining the device image, according to the attribute features corresponding to the substation equipment contained in the device image, the enhancement processing method corresponding to the attribute features is determined from the pre-set correspondence, and the device image is enhanced based on this enhancement processing method, so as to obtain the image to be processed corresponding to each device image. Among them, different enhancement processing methods correspond to different enhancement methods and different enhancement parameters. Different enhancement methods may include but are not limited to: rotation, shearing, adjusting brightness, adjusting contrast, adjusting saturation and adjusting hue, etc.
[0040] Exemplarily, if the substation equipment in the equipment sample image is a complex structure equipment attribute feature, rotation enhancement processing can be first performed based on a predetermined first target rotation angle, and then shear enhancement processing can be performed based on a predetermined first target shear parameter; if the substation equipment in the equipment sample image is a simple structure equipment attribute feature, rotation enhancement processing can be first performed based on a predetermined second target rotation angle, and then shear enhancement processing can be performed based on a predetermined second target shear parameter.
[0041] As another example, if the substation equipment in the equipment image is a metal material equipment attribute feature, the enhancement processing method includes: performing brightness adjustment enhancement processing based on the target brightness value, performing contrast adjustment enhancement processing based on the target contrast value, performing saturation adjustment enhancement processing based on the target saturation value, and performing hue adjustment enhancement processing based on the target hue value.
[0042] S130, for each image to be used, determine at least one initial cue point corresponding to the current image to be used, input the current image to be used and each initial cue point into the image segmentation model to be trained, and obtain an initial segmentation result corresponding to the current image to be used.
[0043] Among them, the current image to be used is the image to be used that is currently being processed, and any one of the images to be used can be used as the current image to be used. The initial prompt point is different from the modified prompt point. The number of initial prompt points can be one or more. The image segmentation model to be trained is specifically a segmentation everything model (SegmentAnything Model, SAM) that needs further fine-tuning and training. The initial segmentation result is the segmentation result output by the image segmentation model to be trained after the image to be used and the initial prompt point are processed by the image segmentation model to be trained. Among them, the initial segmentation result may include the segmentation result of the entire device and / or the segmentation result of at least one key component of the device.
[0044] In this embodiment, the specific implementation method of determining at least one initial prompt point corresponding to the current image to be used may include: processing the current image to be used based on a key point detection algorithm to obtain suspected points of the device and / or suspected points of key components of the device in the current image to be used; using the suspected points of the device and / or suspected points of key components of the device as at least one initial prompt point corresponding to the current image to be used.
[0045] Among them, the suspected point of the equipment is a pixel point or a set of pixel points that may be a substation equipment in the current image to be used. The suspected point of the key component of the equipment is a pixel point or a set of pixel points that may be a key component of the equipment in the current image to be used.
[0046] In the specific implementation process, the current image to be used can be input into the key point detection algorithm, and the key point detection algorithm outputs the suspected points of the device and / or the suspected points of the key components of the device in the current image to be used, which are the initial prompt points. In this embodiment, by placing prompt points at different scales, for example, setting prompt points on the overall outline of the device and the details of the key components, it helps the image segmentation model understand the image content at different levels, which is conducive to the fine segmentation of complex structure devices.
[0047] In addition, the initial cue point can be determined by using image preprocessing techniques (such as edge detection algorithms) or heuristic rules based on prior knowledge of the device.
[0048] Specifically, the processing method for each image to be used is the same, and the current image to be processed is used as an example for explanation. Based on the one or more initial cue points corresponding to the current image to be processed determined based on the above method, the current image to be processed and all the initial cue points are used as inputs of the image segmentation model to be trained, and the image segmentation model to be trained can output the initial segmentation result corresponding to the current image to be used. Optionally, all the images to be used and the corresponding initial cue points can be batch-input into the image segmentation model to be trained, and the image segmentation model to be trained can output the initial segmentation results corresponding to all the images to be processed.
[0049] S140: Based on the initial segmentation result and the sample annotation information, at least one initial prompt point corresponding to the current image to be used is adjusted to obtain a modified prompt point.
[0050] Among them, adjusting the initial prompt point can be specifically understood as adding or deleting a prompt point. After adjusting at least one initial prompt point, there are situations where newly added prompt points, deleted prompt points, and unchanged prompt points exist. Based on this, the set of prompt points after all initial points are adjusted is called modified prompt points. Based on this, the number of modified prompt points can be one or more.
[0051] The sample labeling information includes: the equipment labeling information with the substation equipment as a whole and the equipment key component labeling information with the substation equipment key component as a segmentation object. The initial segmentation result includes the equipment segmentation result and / or at least one equipment key component segmentation result.
[0052] In this embodiment, based on the initial segmentation result and the sample annotation information, the specific implementation method of adjusting at least one initial prompt point corresponding to the current image to be used may include: for each substation equipment in the current image to be used, if the sample annotation information corresponding to the current substation equipment is the overall equipment annotation information, and the initial segmentation result is the segmentation result of at least one key component of the equipment, then the initial prompt points located in the image area of the current substation equipment are deleted until only one is retained; if the sample annotation information corresponding to the current substation equipment is the key component annotation information of the equipment, and the initial segmentation result is the overall equipment segmentation result, then prompt points are added in the image area corresponding to at least one key component of the current substation equipment.
[0053] Specifically, since the current image to be used may include one or more substation equipment, for each substation equipment, if the initial segmentation result and the sample annotation information are consistent, the initial prompt point corresponding to the substation equipment does not need to be adjusted; if the initial segmentation result and the sample annotation information are inconsistent, the initial prompt point corresponding to the substation equipment is adjusted. Specifically, if the sample annotation information corresponding to a substation equipment is the overall annotation information of the equipment, and the initial segmentation result is the segmentation result of at least one key component of the equipment, and the image area of the substation equipment includes more than one initial prompt point, the initial prompt point located in the image area of the substation equipment is deleted until only one is retained. If the sample annotation information corresponding to a substation equipment is one or more key component annotation information of the equipment, and the initial segmentation result is the overall segmentation result of the equipment, a prompt point is added in the image area corresponding to at least one key component of the substation equipment, and the adjustment of the initial prompt point is achieved in the above manner.
[0054] S150, training the image segmentation model to be trained based on each image to be used and the corresponding correction prompt points. When the segmentation result output by the image segmentation model to be trained meets the preset training end condition, the training of the image segmentation model to be trained is completed, and an image segmentation model corresponding to the substation equipment is obtained.
[0055] Specifically, each image to be used and the correction prompt points corresponding to each image to be processed can be used as the input of the image segmentation model to be trained again, and the image segmentation model to be trained can be trained, and the image segmentation model to be trained can be corrected according to the segmentation result and sample annotation information output by the image segmentation model to be trained. When the segmentation result output by the image segmentation model to be trained meets the preset training end condition (for example, the loss function determined according to the segmentation result and sample annotation information converges, or the number of training times reaches the maximum number of iterations, etc.), the training process of the image segmentation model to be trained is stopped, and at this time, the image segmentation model corresponding to the substation equipment can be obtained.
[0056] It should be noted that the acquisition, storage, use, and processing of data in the technical solution of this application are in compliance with the relevant provisions of national laws and regulations. For example, network requests are intercepted and processed with the authorization of the user.
[0057] The technical solution of the embodiment of the present invention is to obtain a substation equipment image sample set, wherein the substation equipment image sample set includes multiple groups of equipment image samples, each group of equipment image samples includes equipment images and sample annotation information corresponding to the equipment images, and the equipment images include at least one substation equipment, so that for each equipment image, based on the attribute characteristics of the substation equipment in the equipment image, the equipment image is enhanced to obtain a to-be-used image corresponding to each equipment image, and further, for each to-be-used image, at least one initial prompt point corresponding to the current to-be-used image is determined, and the current to-be-used image and each initial prompt point are input into the to-be-trained image segmentation model to obtain an initial segmentation result corresponding to the current to-be-used image, and then, based on the initial segmentation result and the sample annotation information, at least one initial prompt point corresponding to the current to-be-used image is adjusted to obtain a revised prompt point, and finally, based on each to-be-used image and the corresponding revised prompt point, the to-be-trained image segmentation model is trained, and when the segmentation result output by the to-be-trained image segmentation model meets the preset training end condition, the training of the to-be-trained image segmentation model is terminated to obtain an image segmentation model corresponding to the substation equipment. The technical solution of the present invention does not require human intervention throughout the entire process, and improves the efficiency of refined training of image segmentation models in application scenarios of segmenting substation equipment and key components of the equipment through automatic adjustment of prompt points.
[0058] Embodiment 2
[0059] Figure 2 This is a schematic diagram of an image segmentation model training method for substation equipment provided by an embodiment of the present invention. Based on the above embodiment, S120 is further described. The specific implementation method can refer to the technical solution of this embodiment. The technical terms that are the same or corresponding to the above embodiment are not repeated here.
[0060] like Figure 2 As shown, the method specifically comprises the following steps:
[0061] S210. Obtain a substation equipment image sample set; wherein the substation equipment image sample set includes multiple groups of equipment image samples, each group of equipment image samples includes an equipment image and sample annotation information corresponding to the equipment image, and the equipment image includes at least one substation equipment.
[0062] S220: If the substation equipment in the equipment image has a complex structure equipment attribute feature, the enhancement processing method includes: performing rotation enhancement processing based on a predetermined target rotation angle and performing shear enhancement processing based on a predetermined target shear parameter.
[0063] In this embodiment, if the substation equipment in the equipment sample image is a complex structure equipment attribute feature, rotation enhancement processing may be performed based on a predetermined target rotation angle, and then shear enhancement processing may be performed based on a predetermined target shear parameter.
[0064] Next, how to determine the target rotation angle and target shearing parameters is explained in detail.
[0065] Optionally, specific implementation steps of determining the target rotation angle and target shearing parameters include:
[0066] (1) determining a first number of preset rotation parameters within an angular range based on a preset angle step, and determining a second number of preset shearing parameters based on different preset sizes;
[0067] Exemplarily, the preset angle step is 10°, and 36 preset rotation parameters can be determined within the 360° angle range. Different preset sizes can be determined by: multiple different image sizes determined based on expert experience are preset sizes, or the length and width are randomly determined within the range of the length and width of the device image to obtain multiple different preset sizes. For example, presetting 20 different preset sizes is 20 preset shearing parameters.
[0068] (2) Based on the first number of preset rotation parameters and the second number of preset shearing parameters, determining a third number of combination parameters; wherein the third number is the product of the first number and the second number.
[0069] Specifically, different preset rotation parameters and different preset shearing parameters are combined to determine a plurality of combination parameters. Based on the above exemplary embodiment, 720 combination parameters can be determined according to 36 preset rotation parameters and 20 preset shearing parameters.
[0070] (3) Determine the target rotation angle and target shearing parameters from the combined parameters.
[0071] In this embodiment, it is determined from all the combination parameters which combination parameter corresponds to a better image segmentation model performance, and the preset parameters included in the combination parameter are the target parameters.
[0072] Specifically, determining the target rotation angle and the target shearing parameter from the combination parameters may include: obtaining an image of the equipment to be processed in which the substation equipment is a complex structure equipment; performing corresponding enhancement processing on the image of the equipment to be processed based on each combination parameter, and obtaining an enhanced processed image corresponding to each combination parameter; inputting each enhanced processed image into a preset image segmentation model, determining the segmentation accuracy corresponding to each enhanced processed image, and determining the combination parameter corresponding to the maximum segmentation accuracy as the target combination parameter; using the preset rotation parameter in the target combination parameter as the target rotation parameter, and using the preset shearing parameter in the target combination parameter as the target shearing parameter.
[0073] Exemplarily, a device image containing a complex structure device can be obtained as the device image to be processed, and then the device image to be processed is traversed through each combination parameter. That is, for each combination parameter, the current combination parameter is applied to the device image to be processed to obtain the current enhanced processed image, and the current enhanced processed image is input into the preset image segmentation model, and the preset image segmentation model outputs the current segmentation result. The current segmentation accuracy is determined according to the current segmentation result and the actual annotation information. For example, the segmentation accuracy can be determined by calculating the intersection over union (IoU) index. Based on the same method, the segmentation accuracy corresponding to each combination parameter can be determined. Thus, the size of each segmentation accuracy is compared, and the combination parameter with the largest segmentation accuracy value is determined as the target combination parameter, and the preset rotation parameter in the target combination parameter is used as the target rotation parameter, and the preset shear parameter in the target combination parameter is used as the target shear parameter.
[0074] In addition, determining the target rotation angle and the target shearing parameter may also include: obtaining a large number of device images containing complex structure devices, for example, obtaining 5000 images. For each device image, a preset number of combination parameters to be applied are randomly matched to the device image from the combination parameters, for example, 10 combination parameters to be applied are determined from 720 combination parameters for each device image; then, each combination parameter to be applied is used to perform corresponding enhancement processing on the device image, and an enhanced processed image corresponding to each combination parameter to be applied is obtained; then, each enhanced processed image is input into a preset image segmentation model to determine the segmentation accuracy corresponding to each enhanced processed image; thus, the combination parameters to be applied applied to the enhanced processed image corresponding to the maximum segmentation accuracy are used as the optimal combination parameters of the device image, so that 5000 optimal combination parameters can be determined; further, the target combination parameters are determined from the optimal combination parameters. This method uses a random strategy to determine different combination parameters, that is, randomly selecting preset rotation parameters and preset shearing parameters in each training iteration, which can ensure that the data distribution is balanced overall and avoid overfitting a specific type of enhancement.
[0075] S230. If the substation equipment in the equipment image has metal material equipment attribute characteristics, the enhancement processing method includes: brightness adjustment enhancement processing based on the target brightness value, contrast adjustment enhancement processing based on the target contrast value, saturation adjustment enhancement processing based on the target saturation value, and hue adjustment enhancement processing based on the target hue value.
[0076] In this embodiment, the specific implementation steps of determining the target brightness value, the target contrast value, the target saturation value, and the target hue value may include: (1) determining a plurality of preset brightness values within a brightness value range based on a first preset step size, determining a plurality of preset contrast values within a contrast value range based on a second preset step size, determining a plurality of preset saturation values within a saturation value range based on a third preset step size, and determining a plurality of preset hue values within a hue value range based on a fourth preset step size; (2) determining a plurality of combination parameters based on a plurality of preset brightness values, a plurality of preset contrast values, a plurality of preset saturation values, and a plurality of preset hue values; (3) obtaining a plurality of preset parameters of a substation equipment; (4) performing corresponding enhancement processing on the image of the device to be processed based on each combination parameter, and obtaining an enhanced processed image corresponding to each combination parameter; (5) inputting each enhanced processed image into a preset image segmentation model, determining the segmentation accuracy corresponding to each enhanced processed image, and determining the combination parameter corresponding to the maximum segmentation accuracy as the target combination parameter; (6) using the preset brightness value in the target combination parameter as the target brightness value, the preset contrast value in the target combination parameter as the target contrast value, the preset saturation value as the target saturation value, and the preset hue value as the target hue value.
[0077] S240, for each image to be used, determine at least one initial cue point corresponding to the current image to be used, input the current image to be used and each initial cue point into the image segmentation model to be trained, and obtain an initial segmentation result corresponding to the current image to be used.
[0078] S250: Based on the initial segmentation result and the sample annotation information, at least one initial prompt point corresponding to the current image to be used is adjusted to obtain a modified prompt point.
[0079] S260, training the image segmentation model to be trained based on each image to be used and the corresponding correction prompt points. When the segmentation result output by the image segmentation model to be trained meets the preset training end condition, the training of the image segmentation model to be trained is completed, and an image segmentation model corresponding to the substation equipment is obtained.
[0080] It is understandable that as the performance of image segmentation models on specific substation equipment gradually matures, knowledge distillation technology can be used to transfer this part of the learning results to the segmentation tasks of other equipment, reducing the dependence on additional labeled data.
[0081] The technical solution of the embodiment of the present invention is that in the process of enhancing the device image, if the substation equipment in the device image is a complex structure device attribute feature, the enhancement processing method includes: rotation enhancement processing based on a predetermined target rotation angle and shearing enhancement processing based on a predetermined target shearing parameter; if the substation equipment in the device image is a metal material device attribute feature, the enhancement processing method includes: brightness adjustment enhancement processing based on a target brightness value, contrast adjustment enhancement processing based on a target contrast value, saturation adjustment enhancement processing based on a target saturation value, and hue adjustment enhancement processing based on a target hue value. By performing personalized enhancement processing on the device images of complex structure device attribute features and metal material device attribute features, the presentation images of the device images of complex structure device attribute features under different viewing angles and in the presence of occlusion, and the presentation images of the device images of metal material device attribute features under specific lighting environments can be simulated. The segmentation model is trained using the device images after enhancement processing, ensuring that high-precision abnormal behavior recognition and segmentation effects can be maintained even in complex and changeable monitoring environments, thereby improving the adaptability and generalization ability of the image segmentation model.
[0082] Embodiment 3
[0083] Figure 3 A structural schematic diagram of an image segmentation model training device for substation equipment provided in an embodiment of the present invention, the device includes: a sample set determination module 310, an enhancement processing module 320, an initial segmentation module 330, a prompt point correction module 340 and a segmentation model training module 350.
[0084] The sample set determination module 310 is used to obtain a substation equipment image sample set; wherein the substation equipment image sample set includes multiple groups of equipment image samples, each group of equipment image samples includes an equipment image and sample annotation information corresponding to the equipment image, and the equipment image includes at least one substation equipment;
[0085] An enhancement processing module 320 is used to perform enhancement processing on each of the equipment images based on the attribute characteristics of the substation equipment in the equipment image to obtain an image to be used corresponding to each of the equipment images;
[0086] The initial segmentation module 330 is used to determine, for each of the images to be used, at least one initial cue point corresponding to the current image to be used, input the current image to be used and each of the initial cue points into the image segmentation model to be trained, and obtain an initial segmentation result corresponding to the current image to be used;
[0087] A cue point correction module 340, configured to adjust the at least one initial cue point corresponding to the current image to be used based on the initial segmentation result and the sample annotation information to obtain a corrected cue point;
[0088] The segmentation model training module 350 is used to train the image segmentation model to be trained based on each of the images to be used and the corresponding correction prompt points. When the segmentation result output by the image segmentation model to be trained meets the preset training end condition, the training of the image segmentation model to be trained is completed, and an image segmentation model corresponding to the substation equipment is obtained.
[0089] Based on the above device, optionally, the initial segmentation module 330 includes: an initial prompt point determination unit;
[0090] Among them, the initial prompt point determination unit is used to process the current image to be used based on the key point detection algorithm to obtain the suspected points of the device and / or the suspected points of the key components of the device in the current image to be used; and use the suspected points of the device and / or the suspected points of the key components of the device as at least one initial prompt point corresponding to the current image to be used.
[0091] On the basis of the above device, optionally, the sample annotation information includes the device overall annotation information with the substation device as a segmentation object and the device key component annotation information with the substation device key component as a segmentation object, and the prompt point correction module 340 includes: a prompt point deletion unit and a prompt point addition unit;
[0092] The prompt point deletion unit is used for deleting the initial prompt points located in the image area of the current substation equipment until only one is retained, for each substation equipment in the current image to be used, if the sample annotation information corresponding to the current substation equipment is the overall annotation information of the equipment, and the initial segmentation result is the segmentation result of at least one key component of the equipment;
[0093] A prompt point adding unit is used to add prompt points in the image area corresponding to at least one key component of the current substation equipment if the sample annotation information corresponding to the current substation equipment is the annotation information of the key component of the equipment and the initial segmentation result is the overall segmentation result of the equipment.
[0094] On the basis of the above device, optionally, the attribute characteristics of the substation equipment include attribute characteristics of complex structure equipment and the attribute characteristics of the substation equipment include attribute characteristics of metal material equipment, and the enhanced processing module 320 includes: a first type of enhanced unit and a second type of enhanced unit;
[0095] Among them, the first type of enhancement unit is used for if the substation equipment in the equipment image is a complex structure equipment attribute feature, the enhancement processing method includes: rotation enhancement processing based on a predetermined target rotation angle and shear enhancement processing based on a predetermined target shear parameter.
[0096] The second type of enhancement unit is used for if the substation equipment in the equipment image is a metal material equipment attribute feature, the enhancement processing method includes: brightness adjustment enhancement processing based on the target brightness value, contrast adjustment enhancement processing based on the target contrast value, saturation adjustment enhancement processing based on the target saturation value, and hue adjustment enhancement processing based on the target hue value.
[0097] On the basis of the above device, optionally, the image segmentation model training device further includes an enhanced parameter determination unit; the enhanced parameter determination unit includes: a parameter preset subunit, a combined parameter determination subunit and an enhanced parameter determination subunit;
[0098] A parameter preset subunit, configured to determine a first number of preset rotation parameters within a circumferential angle range based on a preset angle step, and to determine a second number of preset shearing parameters based on different preset sizes;
[0099] A combination parameter determination subunit, configured to determine a third number of combination parameters based on the first number of preset rotation parameters and the second number of preset shear parameters; wherein the third number is the product of the first number and the second number;
[0100] The enhanced parameter determination subunit is used to determine the target rotation angle and the target shearing parameter from the combined parameters.
[0101] On the basis of the above-mentioned device, optionally, an enhanced parameter determination subunit is specifically used to obtain an image of a device to be processed in which the substation equipment is a complex structure device; after performing corresponding enhancement processing on the image of the device to be processed based on each of the combined parameters, an enhanced processed image corresponding to each of the combined parameters is obtained; each of the enhanced processed images is input into a preset image segmentation model, the segmentation accuracy corresponding to each of the enhanced processed images is determined, and the combined parameters corresponding to the maximum segmentation accuracy are determined as target combined parameters; the preset rotation parameters in the target combined parameters are used as target rotation parameters, and the preset shearing parameters in the target combined parameters are used as target shearing parameters.
[0102] The technical solution of the embodiment of the present invention is to obtain a substation equipment image sample set, wherein the substation equipment image sample set includes multiple groups of equipment image samples, each group of equipment image samples includes equipment images and sample annotation information corresponding to the equipment images, and the equipment images include at least one substation equipment, so that for each equipment image, based on the attribute characteristics of the substation equipment in the equipment image, the equipment image is enhanced to obtain a to-be-used image corresponding to each equipment image, and further, for each to-be-used image, at least one initial prompt point corresponding to the current to-be-used image is determined, and the current to-be-used image and each initial prompt point are input into the to-be-trained image segmentation model to obtain an initial segmentation result corresponding to the current to-be-used image, and then, based on the initial segmentation result and the sample annotation information, at least one initial prompt point corresponding to the current to-be-used image is adjusted to obtain a revised prompt point, and finally, based on each to-be-used image and the corresponding revised prompt point, the to-be-trained image segmentation model is trained, and when the segmentation result output by the to-be-trained image segmentation model meets the preset training end condition, the training of the to-be-trained image segmentation model is terminated to obtain an image segmentation model corresponding to the substation equipment. The technical solution of the present invention does not require human intervention throughout the entire process, and improves the efficiency of refined training of image segmentation models in application scenarios of segmenting substation equipment and key components of the equipment through automatic adjustment of prompt points.
[0103] The image segmentation model training device for substation equipment provided by an embodiment of the present invention can execute the image segmentation model training method for substation equipment provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.
[0104] It is worth noting that the various units and modules included in the above system are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present invention.
[0105] Embodiment 4
[0106] Figure 4 The present invention is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 4 A block diagram of an exemplary electronic device 40 suitable for implementing exemplary embodiments of the present invention is shown. Figure 4 The electronic device 40 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.
[0107] like Figure 4As shown, the electronic device 40 is in the form of a general computing device. The components of the electronic device 40 may include but are not limited to: one or more processors or processing units 401, a system memory 402, and a bus 403 connecting different system components (including the system memory 402 and the processing unit 401).
[0108] Bus 403 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.
[0109] The electronic device 40 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 40, including volatile and non-volatile media, removable and non-removable media.
[0110] System memory 402 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 404 and / or cache memory 405. Electronic device 40 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 406 may be used to read and write non-removable, non-volatile magnetic media ( Figure 4 not shown, usually called a "hard drive"). Although Figure 4 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 403 via one or more data medium interfaces. Memory 402 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of various embodiments of the present invention.
[0111] A program / utility 408 having a set (at least one) of program modules 407 may be stored, for example, in memory 402, such program modules 407 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 407 generally perform the functions and / or methods of the embodiments described herein.
[0112] The electronic device 40 may also communicate with one or more external devices 409 (e.g., keyboards, pointing devices, displays 410, etc.), may communicate with one or more devices that enable a user to interact with the electronic device 40, and / or may communicate with any device that enables the electronic device 40 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface 411. Furthermore, the electronic device 40 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 412. As shown, the network adapter 412 communicates with other modules of the electronic device 40 via the bus 403. It should be understood that although Figure 4 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 40, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0113] The processing unit 401 executes various functional applications and page processing by running the program stored in the system memory 402, such as implementing the image segmentation model training method applied to substation equipment provided in the embodiment of the present invention.
[0114] Embodiment 5
[0115] An embodiment of the present invention further provides a storage medium containing computer executable instructions, which, when executed by a computer processor, is used to execute an image segmentation model training method applied to substation equipment, the method comprising:
[0116] Acquire a substation equipment image sample set; wherein the substation equipment image sample set includes multiple groups of equipment image samples, each group of equipment image samples includes an equipment image and sample annotation information corresponding to the equipment image, and the equipment image includes at least one substation equipment;
[0117] For each of the equipment images, based on the attribute characteristics of the substation equipment in the equipment image, the equipment image is enhanced to obtain an image to be used corresponding to each of the equipment images;
[0118] For each of the images to be used, determining at least one initial cue point corresponding to the current image to be used, inputting the current image to be used and each of the initial cue points into the image segmentation model to be trained, and obtaining an initial segmentation result corresponding to the current image to be used;
[0119] Based on the initial segmentation result and the sample annotation information, adjusting the at least one initial prompt point corresponding to the current image to be used to obtain a modified prompt point;
[0120] The image segmentation model to be trained is trained based on each of the images to be used and the corresponding correction prompt points. When the segmentation result output by the image segmentation model to be trained meets the preset training end condition, the training of the image segmentation model to be trained is completed, and an image segmentation model corresponding to the substation equipment is obtained.
[0121] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.
[0122] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.
[0123] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
[0124] Computer program code for performing the operation of embodiments of the present invention may be written in one or more programming languages or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0125] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims
1. A method for training an image segmentation model for substation equipment, characterized in that: include: Acquire a substation equipment image sample set; wherein the substation equipment image sample set includes multiple groups of equipment image samples, each group of equipment image samples includes an equipment image and sample annotation information corresponding to the equipment image, and the equipment image includes at least one substation equipment; For each of the equipment images, based on the attribute characteristics of the substation equipment in the equipment image, the equipment image is enhanced to obtain an image to be used corresponding to each of the equipment images; For each of the images to be used, determining at least one initial cue point corresponding to the current image to be used, inputting the current image to be used and each of the initial cue points into the image segmentation model to be trained, and obtaining an initial segmentation result corresponding to the current image to be used; Based on the initial segmentation result and the sample annotation information, adjusting the at least one initial prompt point corresponding to the current image to be used to obtain a modified prompt point; The image segmentation model to be trained is trained based on each of the images to be used and the corresponding correction prompt points. When the segmentation result output by the image segmentation model to be trained meets the preset training end condition, the training of the image segmentation model to be trained is completed, and an image segmentation model corresponding to the substation equipment is obtained.
2. The method according to claim 1, characterized in that The step of determining at least one initial prompt point corresponding to the current image to be used includes: Processing the current image to be used based on a key point detection algorithm to obtain suspected points of the device and / or suspected points of key components of the device in the current image to be used; The device suspected point and / or the device key component suspected point are used as at least one initial prompt point corresponding to the image to be used currently.
3. The method according to claim 1, characterized in that The sample annotation information includes device overall annotation information with the substation device as a segmentation object and device key component annotation information with the substation device key components as a segmentation object, and adjusting the at least one initial prompt point corresponding to the current image to be used based on the initial segmentation result and the sample annotation information, including: For each substation equipment in the current image to be used, if the sample annotation information corresponding to the current substation equipment is the overall annotation information of the equipment, and the initial segmentation result is the segmentation result of at least one key component of the equipment, then the initial prompt point located in the image area of the current substation equipment is deleted until only one is retained; If the sample annotation information corresponding to the current substation equipment is the annotation information of the key component of the equipment, and the initial segmentation result is the overall segmentation result of the equipment, a prompt point is added in the image area corresponding to at least one key component of the current substation equipment.
4. The method according to claim 1, characterized in that: The attribute features of the substation equipment include attribute features of complex structure equipment, and the enhancing process of the equipment image based on the attribute features of the substation equipment in the equipment image includes: If the substation equipment in the equipment image has complex structural equipment attribute features, the enhancement processing method includes: performing rotation enhancement processing based on a predetermined target rotation angle and performing shear enhancement processing based on a predetermined target shear parameter.
5. The method according to claim 4, characterized in that The method further comprises: determining the target rotation angle and the target shearing parameter; The determining of the target rotation angle and the target shearing parameter comprises: Determining a first number of preset rotation parameters within a circumferential angle range based on a preset angle step, and determining a second number of preset shearing parameters based on different preset sizes; Determine a third number of combination parameters based on the first number of preset rotation parameters and the second number of preset shear parameters; wherein the third number is the product of the first number and the second number; The target rotation angle and the target shearing parameter are determined from the combined parameters.
6. The method according to claim 5, characterized in that Determining the target rotation angle and the target shearing parameter from the combined parameters includes: Acquire the equipment image to be processed of the substation equipment which is a complex structure equipment; After performing corresponding enhancement processing on the device image to be processed based on each of the combination parameters, an enhanced processed image corresponding to each of the combination parameters is obtained; Inputting each of the enhanced processed images into a preset image segmentation model, determining the segmentation accuracy corresponding to each of the enhanced processed images, and determining the combination parameter corresponding to the maximum segmentation accuracy as the target combination parameter; The preset rotation parameter in the target combination parameter is used as the target rotation parameter, and the preset shearing parameter in the target combination parameter is used as the target shearing parameter.
7. The method according to claim 1, characterized in that The attribute features of the substation equipment include metal material equipment attribute features, and the enhancing process of the equipment image based on the attribute features of the substation equipment in the equipment image includes: If the substation equipment in the equipment image is a metal material equipment attribute feature, the enhancement processing method includes: brightness adjustment enhancement processing based on the target brightness value, contrast adjustment enhancement processing based on the target contrast value, saturation adjustment enhancement processing based on the target saturation value, and hue adjustment enhancement processing based on the target hue value.
8. An image segmentation model training device applied to substation equipment, characterized in that: include: A sample set determination module, used to obtain a substation equipment image sample set; wherein the substation equipment image sample set includes multiple groups of equipment image samples, each group of equipment image samples includes an equipment image and sample annotation information corresponding to the equipment image, and the equipment image includes at least one substation equipment; An enhancement processing module, used to perform enhancement processing on each of the equipment images based on the attribute characteristics of the substation equipment in the equipment image, so as to obtain an image to be used corresponding to each of the equipment images; An initial segmentation module, used to determine, for each of the images to be used, at least one initial cue point corresponding to the current image to be used, input the current image to be used and each of the initial cue points into the image segmentation model to be trained, and obtain an initial segmentation result corresponding to the current image to be used; A cue point correction module, configured to adjust the at least one initial cue point corresponding to the current image to be used based on the initial segmentation result and the sample annotation information to obtain a corrected cue point; The segmentation model training module is used to train the image segmentation model to be trained based on each of the images to be used and the corresponding correction prompt points. When the segmentation result output by the image segmentation model to be trained meets the preset training end condition, the training of the image segmentation model to be trained is completed, and an image segmentation model corresponding to the substation equipment is obtained.
9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image segmentation model training method applied to substation equipment as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image segmentation model training method for substation equipment described in any one of claims 1 to 7 when executed.
Citation Information
Cited By
Image labeling method and apparatus, device, program product, and storage medium
WO2026123615A1
