A difficulty coordination remote sensing photovoltaic image segmentation training method and related device

Abstract

The application provides a remote sensing photovoltaic image segmentation training method and related device with difficulty coordination, wherein the corresponding sample proportion is determined according to the training round number n, the sample proportion includes the sample proportion of difficult types, ordinary types and simple types; samples are randomly extracted from the total sample set according to the sample proportion to obtain the nth round training sample set; the semantic segmentation task model is subjected to spatial weighted loss training by using the nth round training sample set and the spatial weight map of each sample in the nth round training sample set, wherein the spatial weight map includes the spatial weighting coefficient of each pixel in the sample. The semantic segmentation task model is trained by randomly extracting samples from the total sample set according to the sample proportion, the exposure proportion of difficult samples can be stably controlled, the pertinence of difficult area learning is improved, and the segmentation performance of the whole difficult area, difficult boundary and difficult sample can be improved without changing the main structure of the basic segmentation network.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more specifically, to a method and apparatus for training remote sensing photovoltaic image segmentation with difficulty coordination. Background Technology

[0002] With the widespread application of low-to-medium resolution remote sensing data such as Sentinel-2, semantic segmentation-based surface target extraction technology has been applied to scenarios such as photovoltaic power plant identification, expansion monitoring, and resource statistics. Compared with general segmentation tasks in natural images, remote sensing photovoltaic target extraction typically has the following characteristics: low foreground target area ratio, extreme imbalance between foreground and background categories; significant imaging differences under different regions, seasons, and land cover conditions, with obvious intra- and inter-domain offsets; strong confusion between photovoltaic targets and other land features such as bare land, building roofs, and water edges; and a long-tailed distribution of training sample difficulty, with a relatively low proportion of difficult samples that truly determine the upper limit of model performance.

[0003] Improving the semantic segmentation accuracy of remote sensing photovoltaic images has become a challenging problem of concern to those skilled in the art. Summary of the Invention

[0004] The purpose of this invention is to provide a remote sensing photovoltaic image segmentation training method and related apparatus with coordinated difficulty to improve the above-mentioned problems.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows: In a first aspect, embodiments of the present invention provide a remote sensing photovoltaic image segmentation training method with coordinated difficulty, the method comprising: The corresponding sample ratio is determined based on the number of training rounds n, wherein the sample ratio includes the proportion of samples of difficult, normal and easy types, and the difficult, normal and easy types represent the pixel segmentation and recognition difficulty in the samples. Samples are randomly drawn from the total sample set according to the stated sample ratio to obtain the training sample set for the nth round. The semantic segmentation task model is trained using a spatial weighted loss method using the nth round training sample set and the spatial weight map of each sample in the nth round training sample set. The spatial weight map includes the spatial weighting coefficients of each pixel in the sample.

[0006] Secondly, embodiments of the present invention provide a remote sensing photovoltaic image segmentation training device with coordinated difficulty, the device comprising: The first processing unit is used to determine the corresponding sample ratio according to the number of training rounds n, wherein the sample ratio includes the proportion of difficult, normal and easy types of samples, and the difficult, normal and easy types represent the pixel segmentation and recognition difficulty in the samples. The first processing unit is further configured to randomly select samples from the total sample set according to the sample ratio to obtain the training sample set for the nth round; The second processing unit is used to train the semantic segmentation task model using the nth round training sample set and the spatial weight map of each sample in the nth round training sample set, wherein the spatial weight map includes the spatial weighting coefficient of each pixel in the sample.

[0007] Thirdly, embodiments of the present invention provide a storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method.

[0008] Fourthly, embodiments of the present invention provide an electronic device, the electronic device comprising: a processor and a memory, the memory being used to store one or more programs; when the one or more programs are executed by the processor, the above-described method is implemented.

[0009] Compared to existing technologies, the present invention provides a remote sensing photovoltaic image segmentation training method and related apparatus with coordinated difficulty levels. The method determines the corresponding sample ratio based on the number of training rounds (n), including the proportions of difficult, normal, and easy types of samples. Samples are randomly selected from the total sample set according to this ratio to obtain the nth round training sample set. The semantic segmentation task model is trained using a spatially weighted loss algorithm on the nth round training sample set and the spatial weight maps of each sample within it. The spatial weight maps include the spatial weighting coefficients of each pixel in the sample. Randomly selecting samples from the total sample set according to the sample ratio to train the semantic segmentation task model can stably control the exposure ratio of difficult samples, improve the targeting of difficult region learning, and enhance the segmentation performance of difficult regions, difficult boundaries, and difficult samples as a whole without changing the main structure of the basic segmentation network.

[0010] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0013] Figure 2 This is one of the flowcharts illustrating the difficulty-coordinated remote sensing photovoltaic image segmentation training method provided in this embodiment of the invention.

[0014] Figure 3 This is the second flowchart illustrating the difficulty-coordinated remote sensing photovoltaic image segmentation training method provided in this embodiment of the invention.

[0015] Figure 4 This is a verification diagram of the ability to segment difficult samples provided in the embodiments of the present invention.

[0016] Figure 5 A schematic diagram of a unit for a remote sensing photovoltaic image segmentation training device with difficulty coordination provided in an embodiment of the present invention.

[0017] In the diagram: 10-Processor; 11-Memory; 12-Bus; 13-Communication interface; 701-First processing unit; 702-Second processing unit. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0019] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0020] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0021] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0022] In the description of this invention, it should be noted that the terms "upper," "lower," "inner," "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship in which the product of this invention is usually placed when in use. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0023] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set" and "connection" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0024] The following detailed description of some embodiments of the present invention is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0025] Currently, techniques such as class-balanced sampling, hard sample mining, FocalLoss, DiceLoss, online hard sample mining (OHEM), and uncertainty graph-assisted post-processing can be used to improve the model's learning ability in complex regions. However, current technologies still have significant shortcomings in training set difficulty modeling and training control mechanisms.

[0026] First, many existing hard sample learning methods directly use models already fitted to the training set to infer the difficulty of training samples. Since these samples have already participated in model training, the resulting difficulty cannot truly represent the generalization difficulty of the samples, easily leading to overconfidence in previously seen samples. Second, existing techniques typically enhance hard samples separately from the sampling or loss side, lacking a training loop built around a unified source of difficulty. This makes it difficult for difficulty information to permeate both data selection and parameter optimization, limiting the performance improvement of hard region segmentation. Third, existing methods rarely simultaneously consider both sample-level and pixel-level difficulty representation, making it difficult to simultaneously address the questions of "which samples need to be seen more" and "which locations within a sample need stronger constraints." Fourth, hard samples and hard pixels often contain noise and annotation uncertainty. If high-intensity hardness weights are continuously used throughout training, noise can be amplified in the later stages of training, compromising model convergence stability.

[0027] Therefore, it is necessary to propose a technical solution that can generate training difficulty unbiasedly, cover both sample-level and pixel-level granularity, and drive the dual-channel collaborative optimization of sampling and loss with a unified difficulty, so as to improve the segmentation capability of difficult samples of remote sensing photovoltaic targets.

[0028] This invention provides an electronic device, which may be a server device, a computer device, or a mobile phone device, etc. Please refer to... Figure 1 This is a schematic diagram of the structure of an electronic device. The electronic device includes a processor 10, a memory 11, and a bus 12. The processor 10 and the memory 11 are connected via the bus 12. The processor 10 is used to execute executable modules, such as computer programs, stored in the memory 11.

[0029] Processor 10 can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the difficulty-coordinated remote sensing photovoltaic image segmentation training method can be completed through integrated logic circuits in the hardware or software instructions within processor 10. Processor 10 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0030] The memory 11 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage.

[0031] Bus 12 can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. Figure 1 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus 12 or one type of bus 12.

[0032] The memory 11 is used to store programs, such as programs corresponding to a difficulty-coordinated remote sensing photovoltaic image segmentation training device. The difficulty-coordinated remote sensing photovoltaic image segmentation training device includes at least one software functional module that can be stored in the memory 11 in the form of software or firmware, or embedded in the operating system (OS) of the electronic device. Upon receiving an execution instruction, the processor 10 executes the program to implement the difficulty-coordinated remote sensing photovoltaic image segmentation training method.

[0033] The electronic device provided in this embodiment of the invention may further include a communication interface 13. The communication interface 13 is connected to the processor 10 via a bus.

[0034] It should be understood that, Figure 1 The structure shown is only a partial schematic diagram of the electronic device; the electronic device may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.

[0035] The remote sensing photovoltaic image segmentation training method with difficulty coordination provided in this embodiment of the invention can be applied to, but is not limited to, [various applications]. Figure 1 For the specific process of the electronic devices shown, please refer to [link / reference]. Figure 2 The training methods for remote sensing photovoltaic image segmentation with difficulty coordination include S10, S20 and S30, which are described in detail below.

[0036] S10, determine the corresponding sample ratio based on the number of training rounds n.

[0037] The sample proportions include the percentages of difficult, normal, and easy types of samples, which represent the pixel segmentation and recognition difficulty in the samples. Optionally, after the number of training rounds n reaches the proportion adjustment threshold (which can be approximately 60% of the total training progress), the proportion of difficult types of samples gradually decreases and the proportion of easy types of samples gradually increases as the number of training rounds increases.

[0038] Optionally, a higher proportion of hard type samples can be used in the early stages of training, for example, Hard:Medium:Easy = 0.40:0.35:0.25; a more balanced proportion can be used in the later stages of training, for example, Hard:Medium:Easy = 0.35:0.35:0.30. A smooth transition, rather than a hard switch, is used between the early and later proportions to address the problem of noise amplification and increased overfitting risk caused by continuously reinforcing hard pixels in the later stages of training.

[0039] The smooth transition mentioned above refers to the gradual shift in sampling strategy from emphasizing difficult samples to a more robust equilibrium strategy as training progresses. The purpose and advantages of a smooth transition are: 1. Appropriately increasing the proportion of difficult samples in the early stages of training can more quickly draw the model's attention to truly challenging scenarios, preventing the model from remaining stuck on superficial progress brought about by a large number of easy samples. 2. Gradually increasing the proportion of ordinary and easy samples in the later stages of training can prevent the model from being continuously dominated by difficult samples, thereby reducing the risk of repeated amplification of noisy, mislabeled, or extreme samples. 3. Using a smooth transition between the early and late stages, rather than a sudden switch, can prevent abrupt changes in the distribution of training data in a particular round, reducing oscillations in the optimization process and making convergence more stable. This method helps alleviate the problems of noise amplification and increased overfitting risk that may result from continuously reinforcing difficult samples in the later stages of training.

[0040] However, what's being adjusted here is the sampling ratio of difficult samples, which is essentially the reinforcement intensity at the sample level. Correspondingly, the later weight decay addresses the issue of continuously amplifying the weights of difficult pixels at the pixel level. Both aim to encourage more focus on difficult objects in the early stages of training, and then gradually return to a more robust state in the later stages; one operates at the sample level, and the other at the pixel level.

[0041] S20, randomly select samples from the total sample set according to the sample ratio to obtain the training sample set for the nth round.

[0042] It should be noted that this random sampling can be performed by a controlled sampler. By randomly sampling from the total sample set according to the sample proportion, the exposure frequency of difficult samples during training can be stably controlled.

[0043] S30, using the nth round training sample set and the spatial weight map of each sample in the nth round training sample set, the semantic segmentation task model is trained with spatial weighted loss, where the spatial weight map includes the spatial weighting coefficient of each pixel in the sample.

[0044] A unified source of difficulty drives both the sampling and loss channels, ensuring that difficulty information permeates both data selection and parameter optimization, forming a complete training loop. After N rounds of training, model training, inference, and evaluation are completed. Preferred evaluation metrics include overall IoU, boundary IoU, and hard sample IoU. Hard sample IoU can be calculated separately based on Hard-level samples, directly reflecting the technical effectiveness of this invention in hard sample learning.

[0045] In the remote sensing photovoltaic image segmentation training method with difficulty coordination provided in this embodiment of the invention, samples are randomly selected from the total sample set according to the sample ratio to train the semantic segmentation task model. This can stably control the exposure ratio of difficult samples, improve the targeting of learning difficult regions, and improve the segmentation performance of difficult regions, difficult boundaries, and difficult samples as a whole without changing the main structure of the basic segmentation network.

[0046] Building upon the preceding text, this embodiment of the invention also provides an optional implementation method for the content in S30. Please refer to the following: S30, using the nth round training sample set and the spatial weight graph of each sample within the nth round training sample set, performs spatial weighted loss training on the semantic segmentation task model, including: S31, S32, and S33, which are specifically described below.

[0047] S31, use the nth round training sample set to train the semantic segmentation task model, and obtain the recognition result of each pixel in each sample in the nth round training sample set output by the semantic segmentation task model. S32, construct the loss function for the nth round of training based on the spatial weight graph of each sample in the nth round of training sample set.

[0048] Optionally, the loss function includes a Focal component and a Dice component.

[0049] It should be understood that the Focus loss component focuses more on hard-to-classify pixels, while the Dice loss component focuses more on the reasonableness of overall region overlap. The Focus loss component is responsible for drawing attention to hard-to-classify pixels. The Dice loss component is responsible for maintaining the overall region shape and coverage consistency. Spatial weighting coefficients directly affect the Focus loss component, meaning that the pixel classification error is first amplified or reduced pixel by pixel, and then these results are summarized into the Focus loss. The Dice loss component does not directly multiply by the spatial weighting coefficients, but continues to constrain the prediction results from the perspective of overall region overlap of the entire image slice sample. Hard-to-classify pixel enhancement is mainly achieved through the Focus loss component. Overall region consistency is still maintained by the Dice loss component.

[0050] This spatial weighted map can be obtained by superimposing baseline weights on the difficulty coefficients corresponding to each pixel, as will be discussed later. Alternatively, it can be generated by broadcasting the overall difficulty of the samples into a spatial constant weighted map.

[0051] S33. Based on the recognition results of each pixel in each sample within the nth round of training sample set, the true labels, and the loss function of the nth round of training, obtain the total loss of the nth round of training to optimize the semantic segmentation task model.

[0052] Optionally, after the number of training rounds n reaches the spatial weight adjustment threshold, the spatial weighting coefficient of each pixel in the spatial weight map of each sample in the training sample set gradually decays according to the number of training rounds n, until the spatial weighting coefficient of each pixel in the spatial weight map degenerates to 1.

[0053] To mitigate the risk of noise amplification caused by continuously strengthening difficult pixels in the later stages of training, this invention further incorporates a weight decay mechanism. In the early stages of training, spatial weights remain fully effective; after the training progress reaches the weight decay threshold, the influence of spatial weights is gradually reduced; and after the training progress reaches the weight decay endpoint, the spatial weights degenerate into uniform weights of all 1s. In a preferred implementation, the weight decay threshold can be set to 50% of the total training progress (i.e., the spatial weight adjustment threshold), and the weight decay endpoint can be set to 90% of the total training progress. Effective weights can be calculated using the formula "1 + decay coefficient × (original weight - 1)," allowing the original spatial weights to smoothly revert to uniform weights.

[0054] By employing a spatial weight decay mechanism in the later stages of training, the system balances reinforcement learning in challenging regions in the early stages with stable convergence in the later stages, thereby reducing the risk of overfitting caused by continuous noise amplification.

[0055] Regarding the difficulty level of accurately obtaining each sample, this embodiment of the invention also provides an optional implementation method, please refer to... Figure 3 A training method for remote sensing photovoltaic image segmentation based on difficulty coordination.

[0056] Alternatively, the remote sensing photovoltaic image segmentation training methods with collaborative difficulty also include: S51, S52, S53, and S54, which are described in detail below.

[0057] S51, obtain the sample-level difficulty coefficient for each sample; S52, obtain the pixel-level difficulty coefficient for each sample; S53, the sample-level difficulty coefficient and pixel-level difficulty coefficient of each sample are fused to obtain the comprehensive difficulty of each sample.

[0058] This paper addresses the problem of separate modeling of sample-level difficulty coefficients (Patch level) and pixel-level difficulty coefficients (Pixel level), which makes it difficult to use them uniformly. It constructs both sample-level and pixel-level difficulty coefficients and allows pixel-level difficulty to be fed back to Patch-level scoring, thereby achieving integrated difficulty modeling with dual granularity.

[0059] In one alternative implementation, the sample-level difficulty coefficient of each sample can be directly used as its corresponding overall difficulty.

[0060] S54. Based on the overall difficulty of the samples and the preset difficulty level classification threshold, determine the difficulty level of each sample.

[0061] Based on the final overall difficulty, the data is stratified into three levels: Easy, Medium, and Hard, according to quantile thresholds. As an optional implementation, 33% and 67% quantiles can be used as stratification thresholds. The stratification results can be used for subsequent batch-controlled sampling or for constructing continuous sampling weights.

[0062] Optionally, in S51, the sample-level difficulty coefficient for each sample is obtained, including: S511, based on the image spatial source identifier of each sample, divides samples that belong to the same zoning area into the same spatial group.

[0063] Image spatial source identifiers include the original image name, map sheet number, region number, or other information that can characterize the spatial source. These are used for subsequent cross-validation grouping to prevent cross-leakage of spatially adjacent samples between the training and validation sets.

[0064] S512, take the k-th spatial group as the k-th validation subset, and take the other spatial groups other than the k-th spatial group as the k-th training subset (equivalent to spatially aware K-fold cross-validation partitioning of the training samples (Patch)), 1≤k≤K, where K represents the total number of spatial groups.

[0065] S513, train the k-th difficulty estimation model using the k-th training subset.

[0066] There are a total of K difficulty estimation models. All K difficulty estimation models are of the same type. The preferred difficulty estimation model is a supervised learning model that can output class probabilities, such as a gradient boosting tree model.

[0067] S514. The k-th difficulty estimation model after training is used to process the k-th validation subset to output the probability of each sample in the k-th validation subset containing the foreground (the probability of containing photovoltaic devices).

[0068] It should be understood that the k-th difficulty estimation model has not processed any samples in the k-th validation subset before, thus addressing the bias problem caused by backtracking from seen samples in the training set difficulty estimation. Generating training difficulty through a model without seen samples can reduce the difficulty bias caused by backtracking from the training set, making the difficulty representation closer to the true generalization difficulty.

[0069] Optionally, the k-th verification subset includes {sample i, sample j}, and the corresponding prediction result is [0.9, 0.2], that is, the probability of sample i containing the foreground is 0.9, the probability of sample j containing the foreground is 0.2, and thus the probability of all samples containing the foreground can be obtained.

[0070] After obtaining the foreground inclusion probability of all samples, the foreground inclusion probability of each sample is calibrated using sigmoid calibration, isotonic calibration, or other probability calibration methods to improve probability quality.

[0071] S515, the first coefficient is used as the primary coefficient of difficulty for positive samples, and the second coefficient is used as the primary coefficient of difficulty for negative samples, where the first coefficient + the second coefficient = 1, positive samples are those containing foreground probabilities greater than or equal to a set threshold, and negative samples are those containing foreground probabilities less than a set threshold. S516, Determine the uncertainty coefficient of each sample based on the probability of including the prospect of each sample; The normalized entropy can be calculated based on the probability of the sample containing the foreground, and the normalized entropy can be used as its corresponding uncertainty coefficient. Alternatively, the probability of the sample containing the foreground can be substituted into the uncertainty formula to obtain its corresponding uncertainty coefficient.

[0072] The formula for normalized entropy is:

[0073] The formula for the uncertainty coefficient is:

[0074] in, This represents the uncertainty coefficient for the i-th sample. Let represent the probability that the i-th sample contains the foreground.

[0075] It should be understood that when the probability of a sample including the prospect is close to 0 or 1, the corresponding uncertainty coefficient is low; when the probability of a sample including the prospect is close to 0.5, the corresponding uncertainty coefficient is high. In other words, the more hesitant the model, the greater the uncertainty.

[0076] S517, adjust the primary difficulty coefficients based on the uncertainty coefficients (or multiply the two) to obtain the joint difficulty coefficients for each sample.

[0077] S518, obtain the foreground proportion of positive samples and obtain their corresponding size difficulty coefficient.

[0078] Optionally, the formula for the dimensional difficulty coefficient is:

[0079] in: This represents the size difficulty coefficient of the i-th sample, in dimensionless form. This represents the proportion of the foreground area of ​​the i-th sample, in dimensionless form. This represents the minimum effective area threshold for positive samples, and can be, but is not limited to, taking a specific value. The unit is dimensionless. The shape adjustment parameter represents the difficulty of dimensional adjustment and can be, but is not limited to, taking the following values: The unit is dimensionless.

[0080] For pure background samples, this term remains 0. For foreground samples with a proportion less than 0, this term remains 0. For extremely small positive samples, this item is also treated as 0, without adding extra difficulty. For foreground components with a proportion not less than... The effective positive samples exhibit a U-shaped variation, meaning the area increases as it deviates from the mean and decreases as it approaches the mean.

[0081] The size difficulty coefficient adopts a U-shaped function. For samples where the foreground proportion reaches the preset noise threshold, both the very small and the very large targets exhibit high difficulty. For samples where the foreground proportion is lower than the preset noise threshold, the size difficulty can be suppressed or set to zero to reduce the interference of noisy samples on the overall difficulty.

[0082] S519, the joint difficulty coefficient of the negative samples is used as its corresponding sample-level difficulty coefficient, and the joint difficulty coefficient of the positive samples and the size difficulty coefficient are fused according to the preset weight to obtain its corresponding sample-level difficulty coefficient.

[0083] After obtaining the sample-level difficulty coefficients for all samples, robust normalization can be performed on these coefficients to obtain sample-level difficulty coefficients for fusion with the pixel-level difficulty coefficients. Robust normalization can, but is not limited to, using a normalization method based on interquartile range (IQR).

[0084] Optionally, in S52, the pixel-level difficulty coefficients for each sample are obtained, including: S521, take the k-th spatial group as the k-th validation subset, and take the other spatial groups outside the k-th spatial group as the k-th training subset (equivalent to performing spatially aware K-fold cross-validation partitioning on the training samples (Patch)), 1≤k≤K, where K represents the total number of spatial groups; S522, use the k-th training subset to train the k-th semantic segmentation warmup model.

[0085] S523, the k-th semantic segmentation warm-up model after training is used to process the k-th validation subset to output the prediction results of each pixel in each sample in the k-th validation subset. The prediction results represent the probability that a pixel is a foreground pixel.

[0086] S524 generates the difficulty coefficient corresponding to each pixel based on the prediction results of each pixel.

[0087] S525, take a preset number of pixels with the highest difficulty coefficients in the sample as reference pixels, and determine the pixel-level difficulty coefficient of the sample based on the difficulty coefficient of the reference pixels.

[0088] Optionally, S524, based on the prediction results of each pixel, generates a difficulty coefficient corresponding to the pixel, including: S524A calculates the normalized entropy based on the probability that a pixel is a foreground pixel, and uses the normalized entropy as the difficulty coefficient corresponding to the pixel. Alternatively, S524B calculates the difficulty coefficient corresponding to a pixel based on the probability that the pixel is a foreground pixel and the pixel's true value.

[0089] The true value of a pixel is either 0 or 1. When it is 0, it means that the pixel is a background pixel, and when it is 1, it means that the pixel is a foreground pixel.

[0090] The formula for the normalized entropy of a pixel in S524A is:

[0091] in: Represents pixels The normalized entropy, in dimensionless form, has a range of values ​​of . . This represents the probability that a pixel is a foreground pixel, and the unit is dimensionless.

[0092] when The uncertainty is greatest and the difficulty is higher when the value approaches 0.5. When the value is close to 0 or 1, the uncertainty is lower and the difficulty is lower.

[0093] The formula for calculating the difficulty coefficient of a pixel in S524B is:

[0094] in: Represents pixels The error-type difficulty, as a difficulty coefficient, is dimensionless and its value ranges from [value missing]. ; This represents the probability that a pixel is a foreground pixel, and the unit is dimensionless. This represents the pixel ground truth label, with 1 for the foreground and 0 for the background, and the unit is dimensionless.

[0095] After determining the difficulty coefficient corresponding to each pixel in the sample, the difficulty coefficients of each pixel in the sample can be multiplied by a preset coefficient for uniform scaling and then superimposed on the baseline weights to obtain a spatial weight map with a value greater than or equal to 1, so that difficult pixels receive higher weights during training. Each sample corresponds to a spatial weight map, with 1 as the baseline weight.

[0096] A spatial weight map with a value greater than or equal to 1 ensures that the weighting mechanism uses standard supervision as a baseline, only providing additional enhancement to difficult pixels without weakening the normal supervision of other pixels, thereby highlighting the learning of difficult regions while maintaining the stability of the training process and the overall region modeling capability.

[0097] Firstly, the weight coefficient corresponding to the standard training intensity is set to 1. When the weight coefficient is 1, the pixel participates in model training in the standard way; when the weight coefficient is greater than 1, the loss corresponding to the pixel is proportionally amplified during training, achieving reinforcement learning for difficult sample regions. This definition ensures that the spatial weight map only expresses the degree of enhancement, guaranteeing semantic consistency and physical interpretation stability of the weights, and avoiding the simultaneous existence of both enhancement and weakening effects within the same weight system. Secondly, the difficulty perception mechanism adopts a weighted enhancement strategy, where the weight coefficient is only used for loss amplification and does not introduce a reduction mechanism less than 1, thus avoiding active suppression of the original supervision signal. This method is used to maintain the consistency of basic supervision for different pixel categories, avoid unnecessary weakening of the learning signal for relatively easy samples or background regions, and ensure the stability and structural integrity of the training process. Thirdly, the lower limit of the weight coefficient is constrained to be no less than 1 to ensure that each pixel participates in at least standard intensity supervision learning, avoiding the compression of local loss contributions to an extremely low level due to excessively small weights. This constraint is used to prevent insufficient learning signals in local regions and to prevent the model from over-focusing on high-weight difficult regions, thereby improving the overall balance and robustness of feature learning.

[0098] In one optional implementation, the original remote sensing image and its labeled mask are sliced ​​(windowed according to a preset pixel area, with adjacent windows not overlapping during the slicing process) to obtain multiple samples (Patches). Each sample (Patch) includes a remote sensing image slice and its corresponding labeled mask slice. For each sample, the foreground proportion (determined based on the proportion of mask pixels in the labeled mask slice), image source identifier, label category (the label category of each pixel in the training sample, 0 for background, 1 for foreground photovoltaic), and basic features required for subsequent difficulty modeling (spectral features, texture features, shape features, and terrain features, which can be calculated based on the spectral index of the pixel) are statistically analyzed.

[0099] Please refer to Figure 4 , Figure 4 This diagram illustrates the effectiveness of sample segmentation in an embodiment of the present invention. It visually demonstrates that the main technical advantage of the invention is a significant improvement in the segmentation ability for difficult samples.

[0100] The experimental objective corresponding to this figure is: This figure illustrates whether, in the same remote sensing photovoltaic segmentation task, the model's ability to identify and segment difficult sample regions is significantly improved compared to the conventional baseline after the difficulty modeling and collaborative training mechanism proposed in this invention is introduced into the training process.

[0101] Can Figure 4 This diagram serves as a verification of the technical effectiveness. Its focus is not on the change in the network structure itself, but on the training data organization and training optimization methods proposed in this invention, which can significantly improve the segmentation results of difficult samples.

[0102] The experimental background corresponding to this figure: 1. The experimental task is semantic segmentation of photovoltaic regions in medium-resolution remote sensing images.

[0103] 2. All experimental groups used the same type of training data, the same type of segmentation backbone, and a unified evaluation process.

[0104] 3. The main experimental variables are whether to introduce Patch-level difficulty sampling, whether to introduce Pixel-level difficulty weighting, and whether to use a combined mechanism of both.

[0105] Therefore, this figure reflects the impact of different training strategies on the segmentation performance of difficult samples, rather than the differences caused by changes in the dataset or the model backbone.

[0106] Explanation of the meaning of difficult samples: The difficult samples in this figure are not randomly selected poor samples, but rather refer to samples or regions in remote sensing scenes that are more likely to be missed, falsely detected, or have unstable boundary segmentation. Such samples typically have one of the following characteristics: 1. Photovoltaic targets are small in size or have fragmented shapes.

[0107] 2. The boundaries of the photovoltaic array are not clear, making it easy to be confused with the background such as roads, bare ground, and factory roofs.

[0108] 3. The foreground proportion in the sample is extremely low or the distribution is extremely uneven.

[0109] 4. The model has high prediction uncertainty on this type of sample.

[0110] In the experimental analysis, the test samples are divided into three layers—Hard, Medium, and Easy—according to pre-defined statistical rules of difficulty. The Hard layer can be understood as the group of samples that is the most difficult to identify and best reflects the value of the invention. Therefore... Figure 4 Focus on HardIoU, rather than just the overall average.

[0111] Explanation of the evaluation indicators in the diagram: 1. IoU is the intersection-union ratio between the predicted region and the ground truth labeled region, used to measure segmentation accuracy.

[0112] 2. HardIoU is the IoU obtained only on a subset of difficult samples and is used to measure the model's ability to handle difficult examples.

[0113] 3. If indicators such as OverallIoU, MediumIoU, and EasyIoU appear simultaneously in the figure, their main function is to help illustrate that the present invention does not sacrifice overall performance for improved performance.

[0114] 4. For this invention, the most critical object of graph reading is HardIoU or the stratification index directly corresponding to difficult samples.

[0115] 5. In the figure, DASS refers to the difficulty mechanism driven sampling proposed in the embodiments of the present invention.

[0116] like Figure 4 As shown, the analysis content is as follows: Number Name Sampling End Loss End EXP1Baseline_Random random sampling standard loss, unweighted; EXP2StrongBaseline_ClassBalanced class-balanced sampling standard loss, unweighted; EXP3DASS_XGB_NoLossDASS uses hierarchical sampling, employing the patch-level difficulty standard loss provided by XGBoost, without weighting. EXP4DASS_XGB_XGBLossDASS uses layered sampling and employs XGBoost patch-level difficulty to broadcast the patch scalar difficulty as a uniform weight to each pixel before participating in the weighting. EXP5DASS_XGB_PixelLossDASS uses layered sampling, loads pixel weight maps with XGBoost patch-level difficulty, and weights them according to pixel difficulty. EXP6DASS_Pixel_NoLossDASS uses hierarchical sampling, employing pixel-level difficulty standard loss, without weighting. EXP7DASS_Pixel_FullSystemDASS uses hierarchical sampling, employing a pixel-level difficulty pixel weight map that weights pixels by their difficulty. EXP8LossOnly_Random_PixelLoss is a weighted graph of randomly sampled pixels, weighted by pixel difficulty. EXP9LossOnly_ClassBalanced_PixelLoss is a class-balanced sampling pixel weight map that is weighted by pixel difficulty. Public training settings: UNet++ segmentation model, 7-band channel input from remote sensing imagery, 256×256 pixel sample patch size, 128 batch sizes, 100 epochs, 3 random seeds (42 / 999 / 2025), with all other hyperparameters kept consistent across all experiments. Each bar in the figure represents the mean and standard deviation of the experiment with the same number in the table (error bars represent the standard deviation for 3 seeds).

[0117] The meaning of each comparative experimental group in the figure: Each experimental group can be understood as a set of controlled experiments that gradually transition from a "no difficulty mechanism" to "introducing difficulty sampling and loss synergy": 1. Baseline_Random: Random sampling baseline scheme, which does not use the difficulty guidance mechanism of this invention, and is used as the base control for all experiments.

[0118] 2. StrongBaseline_ClassBalanced: This option uses a strong baseline for class-balanced sampling. It does not use difficulty guidance and only alleviates the imbalance between the number of foreground and background samples. It is used to illustrate that simple class balancing cannot fully solve the problem of learning difficult samples.

[0119] 3. DASS_XGB_NoLoss: Introduces patch-level difficulty based on XGBoost for DASS hierarchical sampling, but the loss still uses standard loss without weighting. This is used to verify whether introducing difficulty information only in the sampling stage can bring performance improvement.

[0120] 4. DASS_XGB_XGBLoss: Based on the stratified sampling of patch-level difficulty, the scalar difficulty of the patch is used as a unified weight for the entire patch in the loss weighting, which is used to verify the effect of the synergistic effect of sample-level difficulty on the sampling end and the loss end.

[0121] 5. DASS_XGB_PixelLoss: Based on patch-level difficulty hierarchical sampling, a pixel-level weight map is introduced for pixel-by-pixel loss weighting. This is used to verify whether the cross-granularity collaborative mechanism of using coarse-grained sample-level difficulty at the sampling end and fine-grained pixel-level difficulty at the loss end is effective.

[0122] 6. DASS_Pixel_NoLoss: Employs DASS hierarchical sampling based on pixel-level difficulty, but still uses standard loss without weighting on the loss side. This is used to verify the effectiveness of using only fine-grained pixel-level difficulty to guide sampling.

[0123] 7. DASS_Pixel_FullSystem: A complete solution that uses a pixel-level difficulty map, introduces a pixel weight map at the loss side for pixel-by-pixel weighting, and related collaborative mechanisms to verify the effectiveness of fine-grained difficulty modeling paths.

[0124] 8. LossOnly_Random_PixelLoss: The sampling end maintains random sampling, and only the loss end introduces a pixel-level weight map for pixel-by-pixel weighting. It is used to verify the effect of only weighting the loss and not sampling the difficulty.

[0125] 9. LossOnly_ClassBalanced_PixelLoss: The sampling end adopts class-balanced sampling, and the loss end introduces a pixel-level weight map for pixel-by-pixel weighting. This is used to verify the effect under the conditions of class balance and pixel-level loss weighting, and to distinguish it from the difficulty sampling mechanism of this invention.

[0126] This diagram is primarily used to prove the following conclusion: 1. Compared with the random sampling baseline, as long as the difficulty sampling link of this invention is introduced during the training process, the difficulty sample index will show a significant leap.

[0127] 2. Compared with strong baselines that only perform class balancing, the improvement of this invention does not come from conventional resampling techniques, but from explicit modeling and targeted reinforcement of difficult samples.

[0128] 3. By simply introducing difficulty information at the sampling end, the overall indicators and the indicators of medium and high difficulty samples can be improved, which shows that the difficulty-driven hierarchical sampling mechanism in this invention can effectively improve the distribution of training samples and enhance the model learning effect.

[0129] 4. Based on the introduction of difficulty information at the sampling end, the introduction of a weighting mechanism related to difficulty at the loss end can further improve the overall performance of the model and the performance of difficult samples, indicating that there is a synergistic gain between the strengthening at the sampling end and the strengthening at the loss end.

[0130] 5. When sample-level difficulty is used at the sampling end and pixel-level difficulty is used at the loss end, the model can still achieve better results, indicating that the difficulty modeling mechanism of the present invention has cross-granularity collaborative capability and can adapt to different needs of different training stages for the expression of difficulty information.

[0131] 6. When pixel-level difficulty information is used in both the sampling end and the loss end, the model shows a strong comprehensive advantage in terms of overall accuracy, boundary accuracy, and medium-difficulty sample metrics. This indicates that fine-grained difficulty information can be applied more accurately to hard-to-segment regions and improve the overall stability of the model.

[0132] 7. When pixel-level weighting is introduced only at the loss end without introducing a difficulty sampling link at the sampling end, the improvement in the performance of difficult samples is significantly limited, indicating that weighting at the loss end alone is insufficient to replace the priority coverage of difficult samples at the sampling end.

[0133] 8. Although class-balanced sampling combined with pixel-level loss weighting has a certain effect on local metrics, its performance is not balanced across different difficulty levels, indicating that this type of conventional improvement method is difficult to simultaneously take into account both overall performance and performance on difficult samples.

[0134] 9. The advantages of this invention are mainly reflected in the levels of medium-difficulty and hard samples, indicating that this invention does not simply improve the segmentation results of simple samples, but effectively improves the problem of insufficient learning of hard samples in traditional schemes.

[0135] 10. By introducing difficulty information in collaboration between the sampling end and the loss end, and combining sample-level and pixel-level difficulty representation, this invention can achieve a better balance between overall accuracy, boundary quality, and sample performance at different difficulty levels, thus demonstrating a comprehensive technical effect superior to existing conventional training schemes.

[0136] Please see Figure 5 , Figure 5 The present invention provides a difficulty-coordinated remote sensing photovoltaic image segmentation training device, which is optionally applied to the electronic device described above.

[0137] The remote sensing photovoltaic image segmentation training device with difficulty coordination includes: a first processing unit 701 and a second processing unit 702.

[0138] The first processing unit 701 is used to determine the corresponding sample ratio according to the number of training rounds n. The sample ratio includes the proportion of samples of difficult type, normal type and easy type. The difficult type, normal type and easy type represent the pixel segmentation and recognition difficulty in the sample. The first processing unit 701 is also used to randomly draw samples from the total sample set according to the sample ratio to obtain the training sample set for the nth round. The second processing unit 702 is used to train the semantic segmentation task model using the nth round training sample set and the spatial weight map of each sample in the nth round training sample set, wherein the spatial weight map includes the spatial weighting coefficient of each pixel in the sample.

[0139] The second processing unit 702 can execute the above-described S30, and the first processing unit 701 can execute other steps in the above-described method embodiments.

[0140] It should be noted that the remote sensing photovoltaic image segmentation training device with difficulty coordination provided in this embodiment can execute the method flow shown in the above-described method flow embodiment to achieve the corresponding technical effects. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above-described embodiments.

[0141] This invention also provides a storage medium storing computer instructions and programs. These instructions and programs, when read and executed, perform the difficulty-coordinated remote sensing photovoltaic image segmentation training method described above. The storage medium may include memory, flash memory, registers, or a combination thereof.

[0142] The following provides an electronic device, which may be a server device, a computer device, or a mobile phone device, etc. This electronic device, for example... Figure 1 As shown, the aforementioned difficulty-coordinated remote sensing photovoltaic image segmentation training method can be implemented. Specifically, the electronic device includes: a processor 10, a memory 11, and a bus 12. The processor 10 may be a CPU. The memory 11 is used to store one or more programs, which, when executed by the processor 10, execute the difficulty-coordinated remote sensing photovoltaic image segmentation training method of the above embodiment.

[0143] In summary, the remote sensing photovoltaic image segmentation training method and related apparatus provided by this invention, based on the difficulty-coordinated training round number n, determine the corresponding sample ratio, which includes the proportion of samples of difficult, normal, and easy types. Samples are randomly drawn from the total sample set according to the sample ratio to obtain the nth round training sample set. The semantic segmentation task model is trained using spatial weighted loss using the nth round training sample set and the spatial weight map of each sample within the nth round training sample set. The spatial weight map includes the spatial weighting coefficients of each pixel in the sample. Randomly drawing samples from the total sample set according to the sample ratio to train the semantic segmentation task model can stably control the exposure ratio of difficult samples, improve the targeting of difficult region learning, and improve the segmentation performance of difficult regions, difficult boundaries, and difficult samples as a whole without changing the main structure of the basic segmentation network.

[0144] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0145] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A remote sensing photovoltaic image segmentation training method with collaborative difficulty, characterized in that, The method includes: The corresponding sample ratio is determined based on the number of training rounds n, wherein the sample ratio includes the proportion of samples of difficult, normal and easy types, and the difficult, normal and easy types represent the pixel segmentation and recognition difficulty in the samples. Samples are randomly drawn from the total sample set according to the stated sample ratio to obtain the training sample set for the nth round. The semantic segmentation task model is trained using a spatial weighted loss method using the nth round training sample set and the spatial weight map of each sample in the nth round training sample set. The spatial weight map includes the spatial weighting coefficients of each pixel in the sample.

2. The remote sensing photovoltaic image segmentation training method with collaborative difficulty as described in claim 1, characterized in that, The step of training the semantic segmentation task model using the nth round training sample set and the spatial weight graph of each sample within the nth round training sample set, including: The semantic segmentation task model is trained using the nth round training sample set, and the recognition results of each pixel in each sample in the nth round training sample set output by the semantic segmentation task model are obtained. Based on the spatial weight graph of each sample in the nth round of training sample set, construct the loss function for the nth round of training; Based on the recognition results of each pixel in each sample within the nth round of training sample set, the true labels, and the loss function of the nth round of training, the total loss of the nth round of training is obtained to optimize the training of the semantic segmentation task model.

3. The remote sensing photovoltaic image segmentation training method with collaborative difficulty as described in claim 2, characterized in that, After the number of training rounds n reaches the spatial weight adjustment threshold, the spatial weighting coefficient of each pixel in the spatial weight map of each sample in the training sample set gradually decays according to the number of training rounds n, until the spatial weighting coefficient of each pixel in the spatial weight map degenerates to 1.

4. The remote sensing photovoltaic image segmentation training method with collaborative difficulty as described in claim 1, characterized in that, The method further includes: Obtain the sample-level difficulty coefficient for each sample; Obtain the pixel-level difficulty coefficient for each sample; The sample-level difficulty coefficient and pixel-level difficulty coefficient of each sample are fused to obtain the comprehensive difficulty of each sample. The difficulty level of each sample is determined based on the overall difficulty of the sample and the preset difficulty level classification threshold.

5. The remote sensing photovoltaic image segmentation training method with collaborative difficulty as described in claim 4, characterized in that, The process of obtaining the sample-level difficulty coefficient for each sample includes: Based on the image spatial source identifier of each sample, samples that belong to the same sectional area in space are grouped into the same spatial group; The k-th spatial group is used as the k-th validation subset, and the other spatial groups outside the k-th spatial group are used as the k-th training subset, 1≤k≤K, where K represents the total number of spatial groups; Train the k-th difficulty estimation model using the k-th training subset; The k-th difficulty estimation model after training is used to process the k-th validation subset to output the probability of each sample in the k-th validation subset containing the foreground. The first coefficient is used as the primary coefficient of difficulty for positive samples, and the second coefficient is used as the primary coefficient of difficulty for negative samples. The first coefficient + the second coefficient = 1. Positive samples are those containing foreground probabilities greater than or equal to a set threshold, and negative samples are those containing foreground probabilities less than a set threshold. The uncertainty coefficient of each sample is determined based on the probability of including the prospect in each sample. The initial difficulty coefficient is adjusted based on the uncertainty coefficient to obtain the joint difficulty coefficient for each sample; Obtain the foreground proportion of positive samples and their corresponding size difficulty coefficient; The joint difficulty coefficient of negative samples is used as its corresponding sample-level difficulty coefficient. The joint difficulty coefficient of positive samples and the size difficulty coefficient are fused according to a preset weight to obtain their corresponding sample-level difficulty coefficient.

6. The remote sensing photovoltaic image segmentation training method with collaborative difficulty as described in claim 4, characterized in that, The process of obtaining the pixel-level difficulty coefficient for each sample includes: The k-th spatial group is used as the k-th validation subset, and the other spatial groups outside the k-th spatial group are used as the k-th training subset, 1≤k≤K, where K represents the total number of spatial groups; Train the k-th semantic segmentation warm-up model using the k-th training subset; The k-th semantic segmentation warm-up model after training is used to process the k-th validation subset to output the prediction result of each pixel in each sample in the k-th validation subset. The prediction result represents the probability that the pixel is a foreground pixel. Based on the prediction results of each pixel, generate the difficulty coefficient corresponding to the pixel; A predetermined number of pixels with the highest difficulty coefficients in the sample are selected as reference pixels, and the pixel-level difficulty coefficient of the sample is determined based on the difficulty coefficient of the reference pixels.

7. The remote sensing photovoltaic image segmentation training method with collaborative difficulty as described in claim 6, characterized in that, The step of generating the difficulty coefficient corresponding to each pixel based on the prediction results of each pixel includes: The normalized entropy is calculated based on the probability that a pixel is a foreground pixel, and the normalized entropy is used as the difficulty coefficient corresponding to the pixel. Alternatively, the difficulty coefficient corresponding to a pixel can be calculated based on the probability that the pixel is a foreground pixel and the pixel's true value.

8. A remote sensing photovoltaic image segmentation training device with collaborative difficulty, characterized in that, The device includes: The first processing unit is used to determine the corresponding sample ratio according to the number of training rounds n, wherein the sample ratio includes the proportion of difficult, normal and easy types of samples, and the difficult, normal and easy types represent the pixel segmentation and recognition difficulty in the samples. The first processing unit is further configured to randomly select samples from the total sample set according to the sample ratio to obtain the training sample set for the nth round; The second processing unit is used to train the semantic segmentation task model using the nth round training sample set and the spatial weight map of each sample in the nth round training sample set, wherein the spatial weight map includes the spatial weighting coefficient of each pixel in the sample.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method as described in any one of claims 1-7.

10. An electronic device, characterized in that, include: Processor and memory, the memory being used to store one or more programs; When the one or more programs are executed by the processor, the method as described in any one of claims 1-7 is implemented.

Images