Method for generating sample image, image prediction method, device, equipment and medium

Abstract

The present disclosure relates to a method for generating a sample image, an image prediction method, an apparatus, a device, and a medium. The method includes: determining attribute information of each sample object in an image to be processed, determining a segmentation region corresponding to each sample object according to the attribute information, and performing segmentation processing on the image to be processed according to each segmentation region to obtain a sample image corresponding to each sample object. The present disclosure can determine different segmentation regions for sample objects with different attributes, and thus obtain sample images of different sizes. By associating the attribute information of the sample object with the segmentation region corresponding to the sample object, in this way, the segmentation region can be adaptively adjusted according to the attributes of the sample object, and the integrity of each sample image can be ensured as much as possible.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, and in particular, to a method for generating a sample image, an image prediction method, an apparatus, a device, and a medium. Background Art

[0002] In object detection tasks or large-scale modeling processes, aerial images or satellite images and other large-scale images are often used. These images are characterized by containing large-scale ground information, but they also face problems such as a large map size and high requirements for the memory of electronic devices during the loading process. A common processing method is to uniformly slice the large-scale image and then input it into a training model for model learning. However, in the actual training process, the samples may be truncated, which affects the ability of the model to learn the overall performance of the samples, as well as problems such as too long training time. Summary of the Invention

[0003] To overcome the problems existing in the related art, the present disclosure provides a method for generating a sample image, an image prediction method, an apparatus, a device, and a medium.

[0004] According to a first aspect of an embodiment of the present disclosure, a method for generating a sample image is provided, including:

[0005] Determining attribute information of each sample object in the image to be processed;

[0006] Determining a segmentation region corresponding to each sample object according to the attribute information;

[0007] Performing segmentation processing on the image to be processed according to each segmentation region to obtain a sample image corresponding to each sample object.

[0008] In some embodiments, the attribute information includes: annotation information of the sample object;

[0009] The determining a segmentation region corresponding to each sample object according to the attribute information includes:

[0010] Determining a segmentation granularity of the sample object according to the annotation information and a preset relationship; wherein the preset relationship is used to represent a mapping relationship between the annotation information and the segmentation granularity;

[0011] Determining a segmentation region corresponding to each sample object according to the segmentation granularity.

[0012] In some embodiments, the determining a segmentation region corresponding to each sample object according to the segmentation granularity includes:

[0013] In a case where the segmentation granularity is greater than a first size threshold, determining the segmentation region according to the segmentation granularity of the sample object.

[0014] In some embodiments, determining the segmentation regions corresponding to the respective sample objects according to the segmentation granularity includes:

[0015] When the segmentation granularity is less than or equal to the first size threshold, determining the affiliated objects of the sample object;

[0016] When the segmentation granularity of the affiliated object is greater than the first size threshold, determining the segmentation region of the sample object according to the segmentation granularity of the affiliated object.

[0017] In some embodiments, before determining the segmentation regions corresponding to the respective sample objects according to the attribute information, the method further includes:

[0018] Performing a scaling process on the segmentation region according to a scaling threshold to obtain a scaled segmentation region;

[0019] Performing a segmentation process on the image to be processed according to the scaled segmentation region to obtain sample images corresponding to the respective sample objects.

[0020] According to a second aspect of the embodiments of the present disclosure, there is provided an image prediction method, including:

[0021] Performing a segmentation process on the image to be predicted to obtain a segmented image;

[0022] Identifying each of the segmented images according to a target detection model to determine a target prediction result corresponding to the image to be predicted;

[0023] Wherein, the target detection model is obtained by training an initial detection model according to sample images segmented from the image to be processed and attribute information of the sample objects, and the sample images are obtained by segmenting the image to be processed based on the attribute information of the sample objects.

[0024] In some embodiments, there is an overlapping region between two adjacent segmented images; identifying each of the segmented images according to a target detection model to determine a target prediction result corresponding to the image to be predicted includes:

[0025] Identifying each of the segmented images according to the target detection model to determine a first prediction result corresponding to each of the segmented images;

[0026] Obtaining the target prediction result corresponding to the image to be predicted at least according to each of the first prediction results.

[0027] In some embodiments, obtaining the target prediction result corresponding to the image to be predicted at least according to each of the first prediction results includes:

[0028] When the size of the image to be predicted is greater than or equal to the second size threshold, the first prediction results are stitched to obtain the target prediction result.

[0029] In some embodiments, obtaining the target prediction result corresponding to the image to be predicted at least according to the first prediction results includes:

[0030] When the size of the image to be predicted is less than the second size threshold, the image to be predicted is input into the target prediction model, and the target detection model is used to identify the image to be predicted to obtain a second prediction result corresponding to the image to be predicted;

[0031] The target prediction result is determined according to the first prediction results and the second prediction result.

[0032] In some embodiments, the image to be predicted includes: aerial images and / or satellite images, and the target prediction result at least includes: object type and object area; performing segmentation processing on the image to be predicted to obtain a segmented image includes:

[0033] Performing segmentation processing on the aerial image and / or the satellite image to obtain the segmented image;

[0034] Determining the target prediction result corresponding to the image to be predicted according to the target detection model to identify each of the segmented images includes:

[0035] According to the target detection model, identifying each of the segmented images to determine the object type and the object area of each object in the aerial image and / or the satellite image.

[0036] According to the third aspect of the embodiments of the present disclosure, a sample image generation device is provided, including:

[0037] A first determination module configured to determine the attribute information of each sample object in the image to be processed;

[0038] A second determination module configured to determine the segmentation region corresponding to each sample object according to the attribute information;

[0039] A first segmentation module configured to perform segmentation processing on the image to be processed according to each segmentation region to obtain a sample image corresponding to each sample object.

[0040] In some embodiments, the attribute information includes: the annotation information of the sample object;

[0041] The second determination module is configured to:

[0042] Determine the segmentation granularity of the sample object according to the annotation information and the preset relationship, where the preset relationship is used to represent the mapping relationship between the annotation information and the segmentation granularity;

[0043] Determine the segmentation region corresponding to each sample object according to the segmentation granularity.

[0044] In some embodiments, the second determination module is configured to:

[0045] In the case where the segmentation granularity is greater than the first size threshold, determine the segmentation region according to the segmentation granularity of the sample object.

[0046] In some embodiments, the second determination module is configured to:

[0047] In the case where the segmentation granularity is less than or equal to the first size threshold, determine the affiliated object of the sample object;

[0048] In the case where the segmentation granularity of the affiliated object is greater than the first size threshold, determine the segmentation region of the sample object according to the segmentation granularity of the affiliated object.

[0049] In some embodiments, the apparatus further includes:

[0050] A scaling module configured to perform a scaling process on the segmentation region according to a scaling threshold to obtain a scaled segmentation region;

[0051] A processing module configured to perform a segmentation process on the image to be processed according to the scaled segmentation region to obtain a sample image corresponding to each sample object.

[0052] According to a fourth aspect of the embodiments of the present disclosure, there is provided an image prediction apparatus, including:

[0053] A second segmentation module configured to perform a segmentation process on the image to be predicted to obtain a segmented image;

[0054] An identification module configured to identify each segmented image according to a target detection model to determine a target prediction result corresponding to the image to be predicted;

[0055] Wherein, the target detection model is obtained by training an initial detection model according to sample images segmented from the image to be processed and attribute information of the sample object, and the sample images are obtained by segmenting the image to be processed based on the attribute information of the sample object.

[0056] In some embodiments, there is an overlapping region between two adjacent segmented images; the identification module is configured to:

[0057] Perform a segmentation process on the image to be predicted to obtain a segmented image; wherein, there is an overlapping area between two adjacent segmented images.

[0058] Identify each of the segmented images according to the object detection model to determine a first prediction result corresponding to each of the segmented images.

[0059] Obtain a target prediction result corresponding to the image to be predicted based on at least each of the first prediction results.

[0060] In some embodiments, the identification module is configured to:

[0061] When the size of the image to be predicted is greater than or equal to a second size threshold, perform a splicing process on each of the first prediction results to obtain the target prediction result.

[0062] In some embodiments, the identification module is configured to:

[0063] When the size of the image to be predicted is less than the second size threshold, input the image to be predicted into the target prediction model, and identify the image to be predicted according to the object detection model to obtain a second prediction result corresponding to the image to be predicted.

[0064] Determine the target prediction result according to each of the first prediction results and the second prediction result.

[0065] In some embodiments, the image to be predicted includes: aerial images and / or satellite images, and the target prediction result at least includes: object type and object area; the second segmentation module is configured to:

[0066] Perform a segmentation process on the aerial image and / or the satellite image to obtain the segmented image.

[0067] The identification module is configured to:

[0068] Identify each of the segmented images according to the object detection model to determine the object type and the object area of each object in the aerial image and / or the satellite image.

[0069] According to a fifth aspect of the embodiments of the present disclosure, there is provided an electronic device, including:

[0070] A processor;

[0071] A memory configured to store instructions executable by the processor;

[0072] Wherein, the processor is configured to: when executed, implement the steps in any of the sample image generation methods in the first aspect or any of the image prediction methods in the second aspect described above.

[0073] According to a sixth aspect of the embodiments of the present disclosure, there is provided a non-transitory computer-readable storage medium. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the steps in any of the sample image generation methods in the first aspect or any of the image prediction methods in the second aspect described above.

[0074] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects:

[0075] In the embodiments of the present disclosure, by determining the attribute information of each sample object in the image to be processed, the segmentation region corresponding to each sample object is determined according to the attribute information, and finally, the image to be processed is segmented according to each segmentation region to obtain the sample images corresponding to each sample object. The present disclosure can determine different segmentation regions for sample objects with different attributes, and then obtain sample images of different sizes. By associating the attribute information of the sample object with the segmentation region corresponding to the sample object, in this way, the segmentation region can be adaptively adjusted according to the attributes of the sample object, and the integrity of each sample image is ensured as much as possible.

[0076] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure and used together with the specification to explain the principles of the present disclosure.

[0078] Figure 1 is a flowchart of a method for generating a sample image shown according to an exemplary embodiment of the present disclosure.

[0079] Figure 2 is a flowchart of an image prediction method shown according to an exemplary embodiment of the present disclosure.

[0080] Figure 3 is a flowchart of an image processing method shown according to an exemplary embodiment of the present disclosure.

[0081] Figure 4 is a block diagram of a device for generating a sample image shown according to an exemplary embodiment of the present disclosure.

[0082] Figure 5 is a block diagram of an image prediction device shown according to an exemplary embodiment of the present disclosure.

[0083] Figure 6 is a hardware structure block diagram of an electronic device shown according to an exemplary embodiment of the present disclosure Figure 1 .

[0084] Figure 7 is a hardware structure block diagram of an electronic device shown according to an exemplary embodiment of the present disclosure Figure 2 . Detailed implementation manners

[0085] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0086] Figure 1 is a flowchart of a method for generating a sample image shown according to an exemplary embodiment, as Figure 1 shown, the method for generating the sample image mainly includes the following steps:

[0087] In step 101, determine the attribute information of each sample object in the image to be processed;

[0088] In step 102, according to the attribute information, determine the segmentation region corresponding to each sample object;

[0089] In step 103, perform segmentation processing on the image to be processed according to each segmentation region to obtain a sample image corresponding to each sample object.

[0090] In some embodiments, the image processing method of the present disclosure can be applied to an electronic device, and the electronic device may include: a terminal device, for example, a mobile terminal or a fixed terminal. Among them, the mobile terminal may include: a mobile phone, a tablet computer, a laptop computer, or a wearable device, etc., and may also include a smart home device, for example, a smart speaker, etc. The fixed terminal may include: a desktop computer or a smart TV, etc.

[0091] Here, the image to be processed may include images with dimensions larger than a preset standard dimension. For example, an image with an actual dimension larger than 5000*5000 can be referred to as an image to be processed. The sample object may include: an object with attribute information located in the image to be processed. For example, the image to be processed may include images with large sizes such as aerial images, satellite images, and remote sensing images. The sample object may include objects of different sizes such as rivers, lakes, mountains, roads, buildings, airports, schools, walls, and windows in the image to be processed, which are not specifically limited in the present disclosure. The attribute information may include: various information used to characterize the characteristics of the sample object itself. In some embodiments, the attribute information may include at least one of the following: shape information, such as circular, rectangular, triangular, etc.; type information, such as river, road, basketball court, etc.; position information, such as the river is below the image to be processed, the school is at the center of the image to be processed, the airport is above the image to be processed, etc.; scale level, such as the river belongs to the first scale level range, the basketball court belongs to the second scale level range, etc.; size information, such as the airport is 100*100, the basketball court is 1*1, etc.; association information, such as the road is on the left side of the school, the basketball court is inside the school, etc.

[0092] In some embodiments, the dimension may be a numerical representation of the scale of an image or region such as the image to be processed, the sample image, and the segmentation region. The dimension value corresponds to the scale level. The larger the dimension value, the larger the scale and the higher the level.

[0093] In the implementation process, the various sample objects and the attribute information of each sample object in the image to be processed can be determined by manual annotation. The various sample objects and the attribute information of each sample object in the image to be processed can also be determined by an image processing algorithm.

[0094] In some other embodiments, the attribute information of each sample object in the image to be processed can be determined by manual annotation, or the attribute information of each sample object in the image to be processed can be determined by a preset mapping relationship.

[0095] Taking the determination of the various sample objects and the attribute information of each sample object in the image to be processed by manual annotation as an example, the electronic device can directly read the various sample objects manually annotated in the image to be processed, as well as the attribute information of each sample object.

[0096] In a possible embodiment, the electronic device can also preset the correspondence between sample objects of each type and attribute information. For example, the first sample object corresponds to the first attribute information, the second sample object corresponds to the second attribute information, and so on. Then, the electronic device can identify the object types of each sample object in the image to be processed through a type recognition algorithm or a trained type recognition model, and then obtain the attribute information corresponding to each sample object according to the object type and the preset correspondence between the sample object and the attribute information. In the embodiments of the present disclosure, there is no specific limitation on the method for determining the attribute information of each sample object in the image to be processed, and it can be custom-set according to the actual usage scenario.

[0097] The segmentation region can refer to the region used to segment each sample object. The sizes of the segmentation regions of each sample object can be different, and the sizes of the segmentation regions of each sample object are positively correlated with the actual size regions of each sample object. In some embodiments, the electronic device can determine the segmentation position of the segmentation region corresponding to the sample object according to the position information of the sample object included in the attribute information. For example, the segmentation position can be represented by coordinate points, and the segmentation position can be centered on the coordinate point (10, 10), etc. The segmentation region can include the segmentation size, etc. For example, when the sample object is a school, the segmentation size of the segmentation region can be 5*5, and the segmentation position corresponding to the segmentation region can be the coordinate point (10, 10), etc.

[0098] After the electronic device determines the attribute information of each sample object, it can determine the segmentation region corresponding to each sample object according to the attribute information. The electronic device can preset the correspondence between the attribute information and the segmentation region, and then determine the segmentation region according to the attribute information and the correspondence. For example, the electronic device presets that the attribute information a corresponds to the segmentation region A, the attribute information b corresponds to the segmentation region B, etc. When the electronic device determines that the attribute information of the sample object is a, it can determine that the segmentation region of the sample object is A, etc. The electronic device can also determine the segmentation region through a segmentation model. The segmentation model can include a pre-trained neural network model. The electronic device can input the attribute information of the sample object into the segmentation model to obtain the specific segmentation region of the sample object. Among them, for different sample objects, the types and quantities of the attribute information input into the segmentation model can be different, which helps to simply and quickly determine values such as the segmentation size in the segmentation region.

[0099] A sample image may refer to a small-sized image containing sample objects. One sample image may contain at least one sample object. For example, one sample image may contain one sample object, or one sample image may contain two sample objects, etc. The present disclosure does not make specific limitations and can be customized according to actual usage requirements. After the electronic device determines the segmentation regions corresponding to each sample object, it can segment the image to be processed according to each of the segmentation regions to obtain the sample images corresponding to each of the sample objects. For example: The electronic device determines that the size of the image to be processed is 50*50 and determines that there are two sample objects in the image to be processed. The segmentation region of the first sample object may be a region centered at the coordinate (10, 10) with a segmentation size of 7*7, and the segmentation region of the second sample object may be a region centered at the coordinate (30, 30) with a segmentation size of 5*5. The electronic device segments the image to be processed according to the segmentation regions, and can obtain a first sample image with a size of 7*7 and a second sample image with a size of 5*5, etc., and other regions of the image to be processed can be discarded.

[0100] In the case where the size of the image to be processed is large, if directly performing image processing such as target recognition and tracking, or for image applications such as model training and 3D reconstruction, a large amount of computing resources are required, and a long processing time will be consumed during the processing. Through the technical solution of the present disclosure, the sample images corresponding to each sample object can be accurately obtained, and the electronic device can directly perform image processing such as recognition and tracking on the sample images to improve the operation efficiency and accuracy of the electronic device.

[0101] In the embodiments of the present disclosure, by determining the attribute information of each sample object in the image to be processed, the segmentation regions corresponding to each sample object are determined according to the attribute information, and finally the image to be processed is segmented according to each of the segmentation regions to obtain the sample images corresponding to each of the sample objects. The present disclosure can determine different segmentation regions for sample objects with different attributes, and then obtain sample images of different sizes. By associating the attribute information of the sample object with the segmentation region corresponding to the sample object, in this way, the segmentation region can be adaptively adjusted according to the attributes of the sample object, and the integrity of each sample image can be ensured as much as possible.

[0102] In some embodiments, the attribute information includes: the annotation information of the sample object;

[0103] The determining the segmentation regions corresponding to each of the sample objects according to the attribute information includes:

[0104] Determining the segmentation granularity of the sample object according to the annotation information and a preset relationship; wherein the preset relationship is used to represent the mapping relationship between the annotation information and the segmentation granularity;

[0105] Determine the segmentation regions corresponding to each of the sample objects according to the segmentation granularity.

[0106] In the embodiments of the present disclosure, the attribute information may include the annotation information of the sample object. The annotation information may include type information (such as rivers, roads, basketball courts, etc.) and location information (such as a river is below the image to be processed, a school is at the center of the image to be processed, an airport is above the image to be processed, etc.). The present disclosure does not make specific limitations and can be customized according to actual usage requirements.

[0107] The segmentation granularity may refer to the size used for segmenting the sample object. The first sample object may correspond to the first segmentation granularity, the second sample object may correspond to the second segmentation granularity, etc. For example, the first sample object may be a school, and the corresponding segmentation granularity may be 10*10 in size. The second sample object may be a dormitory, and the corresponding segmentation granularity may be 5*5 in size, etc.

[0108] In some embodiments, the segmentation granularity is positively correlated with the actual size of the sample object, and different sample objects correspond to different segmentation granularities. During the process of determining the sample object, the smaller the granularity level, the higher the refinement degree. On the contrary, the larger the granularity level, the lower the refinement degree. That is, if the segmentation granularity is larger, the number of sample objects determined in the image to be processed is smaller. If the segmentation granularity is smaller, the number of sample objects determined in the image to be processed is larger. For example: the segmentation granularity corresponding to a school as the sample object is greater than the segmentation granularity of a window as the sample object. For the same image to be processed, if the segmentation granularity is 10*10 to determine the sample object, then the electronic device may determine 700 sample objects. If the segmentation granularity is 5*5 to determine the sample object, then the electronic device may determine 1200 sample objects, etc.

[0109] In some embodiments, the electronic device may preset the correspondence between the annotation information and the segmentation granularity. For example: annotation information a corresponds to segmentation granularity A, annotation information b corresponds to segmentation granularity B, etc. After the electronic device determines the annotation information, it may determine the segmentation granularity of the sample object according to the annotation information and the preset relationship.

[0110] After the electronic device determines the segmentation granularity of the sample object, it may determine the segmentation regions corresponding to each sample object according to the segmentation granularity. For example: the electronic device determines the location information carried in the first annotation information corresponding to the first sample object, and the location information may be represented by location coordinates as (125, 75). The electronic device may determine that the segmentation granularity corresponding to the first annotation information is 10*10 in size according to the first annotation information and the corresponding relationship. Then, it may determine that the segmentation size of the segmentation region corresponding to the first sample object is 10*10, etc.

[0111] In the embodiments of the present disclosure, the attribute information may include the annotation information of the sample object. By associating the annotation information with the segmentation granularity and then determining the segmentation region corresponding to each sample object according to the segmentation granularity, different segmentation granularities can be set for sample objects of different sizes, so as to accurately and quickly determine the segmentation region containing the complete sample object.

[0112] In some embodiments, determining the segmentation region corresponding to each sample object according to the segmentation granularity includes:

[0113] When the segmentation granularity is greater than the first size threshold, determining the segmentation region according to the segmentation granularity of the sample object.

[0114] In the embodiments of the present disclosure, after the electronic device determines the segmentation granularity corresponding to the sample object, it may determine whether the segmentation granularity meets the actual usage requirements of the user. For example: The electronic device may preset the first size threshold, and then determine the size relationship between the segmentation granularity and the first size threshold.

[0115] In some embodiments, the first size threshold may include at least one of the following: the first length threshold; the first width threshold; the first area threshold.

[0116] Since the segmentation granularity can be represented in the form of multiple values such as length, width, and area, in the comparison process, the length represented by the segmentation granularity can be directly compared with the first length threshold included in the first size threshold, and the width represented by the segmentation granularity can be compared with the first width threshold included in the first size threshold. When the length represented by the segmentation granularity is greater than the first length threshold and the width represented by the segmentation granularity is greater than the first length threshold, it can be determined that the segmentation granularity is greater than the first size threshold. For example: The electronic device determines that the segmentation granularity is 10*10, the first length threshold is 8, and the first width threshold is 11. Since the first length threshold is less than the length 10 represented by the segmentation granularity, it can be determined that the segmentation granularity is less than the first size threshold. By comparing the length and width, the sample object can be completely within the segmentation region, reducing the probability of incorrect cutting.

[0117] In some other embodiments, the area represented by the segmentation granularity may also be compared with the first area threshold. When the area represented by the segmentation granularity is greater than the first area threshold, it can be determined that the segmentation granularity is greater than the first size threshold. For example, the first area threshold is 5*5, and the area represented by the segmentation granularity is 8*8, then it can be determined that the segmentation granularity is greater than the first size threshold. By comparing the areas, the data processing efficiency is higher.

[0118] In some other embodiments, if the segmentation granularity and the first size threshold are represented by a single value, the magnitudes of the two values (or the meanings represented by the two values) can be directly compared. The present disclosure does not make specific limitations on the representation forms of the segmentation granularity and the first size threshold, and can be custom-set according to actual usage scenarios. For example, if the electronic device determines that the segmentation granularity is 1 and the first size threshold is 2, it can be determined that the segmentation granularity is less than the first size threshold, etc.

[0119] In the embodiments of the present disclosure, by determining the segmentation region according to the segmentation granularity of the sample object when the segmentation granularity is greater than the first size threshold, a more accurate segmentation region can be further determined, improving the accuracy of image processing. Before segmenting the image to be processed, comparing the segmentation granularity and the first size threshold to determine the segmentation granularity of the sample object, and then determining the segmentation region, can improve the accuracy of the segmentation region.

[0120] In some embodiments, the determining the segmentation regions corresponding to the respective sample objects according to the segmentation granularity includes:

[0121] When the segmentation granularity is less than or equal to the first size threshold, determining the affiliated object of the sample object;

[0122] When the segmentation granularity of the affiliated object is greater than the first size threshold, determining the segmentation region of the sample object according to the segmentation granularity of the affiliated object.

[0123] In the embodiments of the present disclosure, when the segmentation granularity is less than or equal to the first size threshold, the electronic device can determine the affiliated object of the sample object. The affiliated object can refer to an object having an affiliated relationship with the sample object, or an object on which the sample object can be affiliated, or an object with a segmentation granularity greater than that of the sample object, etc. For example, when the sample object is glass, the affiliated object can be a window; when the sample object is a window, the affiliated object can be a wall surface; when the sample object is a wall surface, the affiliated object can be a classroom; when the sample object is a classroom, the affiliated object can be a school, etc. For example, if the electronic device determines that the segmentation granularity of the first sample object is 5*5 and the first size threshold is 10*10, it can be determined that the segmentation granularity is less than the first size threshold, and then the electronic device can determine the affiliated object (such as a wall surface) of the sample object (such as a window).

[0124] In some embodiments, the electronic device can determine the affiliated object by comparing the distances between other sample objects within a preset distance of the first sample object and the first sample object. For example, the electronic device can determine that there are three sample objects, namely the second sample object, the third sample object, and the fourth sample object, within the preset distance (e.g., the preset distance can be 500 pixel distances, etc.) of the first sample object according to the attribute information corresponding to each sample object. If the distances between the three sample objects and the first sample object are 270, 150, and 460 respectively, then the electronic device can use the third sample object as the affiliated object of the first sample object.

[0125] In some embodiments, after the electronic device determines the affiliated object of the sample object, it can compare the segmentation granularity corresponding to the affiliated object with the first size threshold again. If the segmentation granularity corresponding to the affiliated object is greater than the first size threshold, the electronic device can determine the segmentation region of the sample object according to the segmentation granularity of the affiliated object. If the segmentation granularity corresponding to the affiliated object is less than or equal to the first size threshold, the electronic device needs to continue to find the affiliated object of the affiliated object until the segmentation granularity of the determined affiliated object is greater than the first size threshold. For example, if the electronic device determines that the segmentation granularity of the first sample object is 5*5, the first size threshold is 10*10, the segmentation granularity of the affiliated object (sample object B) of sample object A is 7*7, and the segmentation granularity of the affiliated object (sample object C) of sample object B is 11*11, then the electronic device can determine that the segmentation granularity corresponding to sample object C can be used to determine the segmentation region of sample object A.

[0126] In the embodiments of the present disclosure, by determining the affiliated object of the sample object when the segmentation granularity is less than or equal to the first size threshold, and determining the segmentation region of the sample object according to the segmentation granularity of the affiliated object when the segmentation granularity of the affiliated object is greater than the first size threshold, it is possible to adaptively set the segmentation regions of different sizes of the sample object, ensure the size of the segmentation regions corresponding to each sample object, prevent the characteristic information corresponding to the sample object from being ignored due to too small a segmentation region, and thus reduce the utilization significance of the sample image.

[0127] In some embodiments, before determining the segmentation region corresponding to each sample object according to the attribute information, the method further includes:

[0128] Performing a scaling process on the segmentation region according to a scaling threshold to obtain a scaled segmentation region;

[0129] Performing a segmentation process on the to-be-processed image according to the scaled segmentation region to obtain sample images corresponding to each sample object.

[0130] In the embodiments of the present disclosure, the electronic device may scale the segmented region according to a scaling threshold to obtain a scaled segmented region. For example, if the electronic device determines that the size of the segmented region is 10*10 and the scaling threshold is 1.1, then by magnifying the segmented region, the size of the magnified segmented region is 11*11. If the scaling threshold is 0.9, then by shrinking the segmented region, the size of the shrunk segmented region is 9*9, etc. Then, the electronic device may perform segmentation processing on the to-be-processed image according to the scaled segmented region to obtain sample images corresponding to each of the sample objects. The selection of the scaling threshold may be determined based on factors such as ensuring the integrity of the sample object while avoiding including neighboring sample objects adjacent to the sample object. For example, a fixed scaling threshold (such as 1.1, etc.) may be determined according to empirical values.

[0131] In a possible embodiment, after the electronic device obtains sample images corresponding to each of the sample objects, it may also screen out the sample images in which the sample objects are truncated and retain the remaining sample images, which helps to ensure the integrity of the sample objects in the sample images. The electronic device may determine whether the connected regions at the edges of the sample images are consistent by performing connected component detection on the sample images. If they are consistent, it may be determined that the sample objects in the sample images are complete. If they are inconsistent, it may be determined that the sample objects in the sample images are not complete. The present disclosure does not specifically limit the method for screening out the sample images in which the sample objects are truncated, and it may be custom-set according to actual usage requirements.

[0132] In the embodiments of the present disclosure, the electronic device may, according to actual usage requirements, further ensure that there are complete sample objects in each sample image after performing segmentation processing on the sample objects by magnifying the segmented region, and prevent the phenomenon that the sample objects are truncated. By shrinking the segmented region, it is possible to accurately and simply obtain specific details of a certain part of the sample object and obtain a segmented image for a certain part.

[0133] Through the technical solution of the present disclosure, it is possible to determine the attribute information of each sample object in the to-be-processed image, thereby determine the segmented region corresponding to each of the sample objects according to the attribute information, and finally perform segmentation processing on the to-be-processed image according to each of the segmented regions to obtain sample images corresponding to each of the sample objects. The present disclosure can determine different segmented regions for sample objects with different attributes, and then obtain sample images of different sizes. By associating the attribute information of the sample object with the segmented region corresponding to the sample object, in this way, the segmented region can be adaptively adjusted according to the attribute of the sample object, and the integrity of each sample image can be ensured as much as possible.

[0134] Figure 2is a flowchart of an image prediction method shown according to an exemplary embodiment. As Figure 2 shown, the image prediction method mainly includes the following steps:

[0135] In step 201, the image to be predicted is segmented to obtain a segmented image;

[0136] In step 202, each of the segmented images is recognized according to the target detection model to determine the target prediction result corresponding to the image to be predicted;

[0137] Wherein, the target detection model is obtained by training an initial detection model according to the sample images segmented from the image to be processed and the attribute information of the sample objects, and the sample images are obtained by segmenting the image to be processed based on the attribute information of the sample objects.

[0138] In the embodiments of the present disclosure, after the electronic device obtains the sample images, it can train the initial detection model according to the sample images and the attribute information to obtain a target detection model. The detection model may refer to a neural network model. During use, after the image to be predicted is input into the detection model, it can identify each target object in the image to be predicted and the attribute information corresponding to each target object, etc. The image to be predicted may refer to large-size images such as aerospace and satellite images that need to determine each target object and the attribute information corresponding to each target object. The target prediction result may include the target object and the attribute information corresponding to each target object, etc. The initial detection model may refer to an untrained detection model, and the target detection model may refer to a trained detection model.

[0139] In the embodiments of the present disclosure, since the size of the image to be processed may be large, if the entire image to be processed is directly input into the target detection model, the recognition time of the target detection model may be long, and the required computing power may be large. The electronic device can segment the image to be predicted to obtain multiple segmented images, and then input each segmented image into the target detection model respectively to obtain the prediction result of each segmented image, so as to obtain the target prediction result of the image to be processed. For example: The electronic device can segment an image to be processed with a size of 15*5 to obtain three segmented images with a size of 5*5.

[0140] In the embodiments of the present disclosure, during the training process, the sizes of the sample images input into the initial detection model may be different. For example: the size of the first sample image is 5*3, and the size of the second sample image is 9*9, etc. The types and quantities of the attribute information corresponding to the sample images input into the initial detection model may also be different. For example: the attribute information corresponding to the first sample image includes position information, and the attribute information corresponding to the second sample image includes position information and type information, etc.

[0141] After obtaining the target detection model, the electronic device can identify the image to be predicted according to the target detection model, and determine the target prediction result corresponding to the image to be predicted. For example: The electronic device determines through the target detection model that there are two target objects in the image to be predicted, the first target object is a river, the second target object is a school, and the actual size is 120*120 and other target prediction results.

[0142] In the embodiments of the present disclosure, by training the initial detection model according to the sample image and the attribute information, a target detection model is obtained. The image to be predicted is identified according to the target detection model, and the target prediction result corresponding to the image to be predicted is determined. By inputting sample images of different sizes, and each sample image contains a complete sample object, the training duration of the detection model can be reduced, and the recognition accuracy of the detection model can be improved, etc.

[0143] In some embodiments, there is an overlapping area between two adjacent segmentation images; the step of identifying each segmentation image according to the target detection model to determine the target prediction result corresponding to the image to be predicted includes:

[0144] Identify each segmentation image according to the target detection model to determine the first prediction result corresponding to each segmentation image;

[0145] At least based on each of the first prediction results, obtain the target prediction result corresponding to the image to be predicted.

[0146] In the embodiments of the present disclosure, the electronic device can perform segmentation processing on the image to be predicted to obtain segmentation images; wherein, there is an overlapping area between two adjacent segmentation images. The segmentation image may refer to a partial image in the image to be predicted, and all the segmentation images can be spliced into the complete image to be predicted. The segmentation processing of the present disclosure is not non-overlapping segmentation processing. For example: If the size of the image to be predicted is 10*5, then the image to be predicted can be non-overlappingly segmented from the middle to obtain two 5*5 segmentation images. The segmentation processing of the present disclosure is overlapping segmentation processing. For example: If the size of the image to be predicted is 10*5, then the image to be predicted can be segmented from the middle to obtain three 5*5 segmentation images, and the horizontal axis intervals of the segmentation images can be 0-5, 2.5-7.5, and 5-10, etc. In a possible embodiment, the electronic device can preset an overlap rate (e.g., 0.5), and perform segmentation processing on the image to be predicted according to the overlap rate to obtain multiple segmentation images with overlapping areas.

[0147] The first prediction result may refer to a part of the target prediction result, and the first prediction results corresponding to all the segmented images can be combined into the target prediction result. After obtaining the segmented images, the electronic device can identify each of the segmented images according to the target detection model to determine the first prediction result corresponding to each of the segmented images. For example, the electronic device determines that the first prediction result in the first segmented image includes that the first target object is a river, and the first prediction result in the second segmented image includes that the second target object is a school, with an actual size of 120*120, etc.

[0148] After the electronic device determines the first prediction result corresponding to each of the segmented images, it can obtain the target prediction result corresponding to the image to be predicted at least according to each of the first prediction results. For example, the electronic device combines (stitches) the first prediction results corresponding to all the segmented images to obtain the target prediction result corresponding to the image to be predicted. For example, the electronic device determines that the first prediction result in the first segmented image includes that the first target object is a river, and the first prediction result in the second segmented image includes that the second target object is a school, with an actual size of 120*120, etc. Then the electronic device can determine that the target prediction result corresponding to the image to be predicted may include that the first target object is a river, the second target object is a school, with an actual size of 120*120, etc.

[0149] In a possible embodiment, the stitching method for the first prediction result may be to determine adjacent segmented images. If there is the same target object in both the first segmented image and the second segmented image, but the target object in the first segmented image is complete and the target object in the second segmented image is not complete, then the electronic device can use the attribute information corresponding to the target object in the first segmented image as the attribute information corresponding to the target object in the target prediction result, which helps to obtain a more accurate target prediction result. The electronic device can detect the connected components of the segmented image to determine whether the connected components at the edge of the segmented image are consistent. If they are consistent, it can be determined that the target object in the segmented image is complete; if they are not consistent, it can be determined that the target object in the segmented image is not complete.

[0150] In the embodiments of the present disclosure, by performing segmentation processing on the image to be predicted to obtain segmented images, where there is an overlapping area between two adjacent segmented images, identifying each of the segmented images according to the target detection model to determine the first prediction result corresponding to each of the segmented images, and obtaining the target prediction result corresponding to the image to be predicted at least according to each of the first prediction results, the detection efficiency and detection accuracy of the target detection model can be effectively improved.

[0151] In some embodiments, the obtaining the target prediction result corresponding to the image to be predicted at least according to each of the first prediction results includes:

[0152] When the size of the image to be predicted is greater than or equal to a second size threshold, perform a splicing process on each of the first prediction results to obtain the target prediction result.

[0153] In the embodiments of the present disclosure, the electronic device may first determine whether the size of the image to be predicted is greater than or equal to a second size threshold. When the size of the image to be predicted is greater than or equal to the second size threshold, directly perform a splicing process on each of the first prediction results to obtain the target prediction result. The second size threshold may include at least one of the following: a second length threshold; a second width threshold; a second area threshold. Since the size of the image to be predicted and the second size threshold can be represented in the form of multiple values such as length, width, and area, during the comparison process, the length represented by the size of the image to be predicted can be directly compared with the second length threshold included in the second size threshold, and the width represented by the size of the image to be predicted can be compared with the second width threshold included in the second size threshold. When the length represented by the size of the image to be predicted is greater than the second length threshold and the width represented by the size of the image to be predicted is greater than the second length threshold, it can be determined that the size of the image to be predicted is greater than the second size threshold.

[0154] In some other embodiments, the area represented by the size of the image to be predicted may also be compared with the second area threshold. When the area represented by the size of the image to be predicted is greater than the second area threshold, it can be determined that the size of the image to be predicted is greater than the second size threshold. For example: The electronic device determines that the area represented by the size of the image to be predicted is 200*200, and the second size threshold is 100*100. The electronic device can determine that the size of the image to be predicted (such as a satellite image) is greater than the second size threshold. Then the electronic device can directly perform a splicing process on each of the first prediction results to obtain the target prediction result. The electronic device can also determine whether to use the method of directly performing a splicing process on each of the first prediction results to obtain the target prediction result according to the current memory, thread number, running state, etc. of the electronic device, or use other methods to obtain the target prediction result, etc.

[0155] In the embodiments of the present disclosure, by directly performing a splicing process on each of the first prediction results when the size of the image to be predicted is greater than or equal to the second size threshold to obtain the target prediction result, the target prediction result can be obtained quickly and accurately, improving the operation efficiency of the electronic device, etc.

[0156] In some embodiments, obtaining the target prediction result corresponding to the image to be predicted at least according to each of the first prediction results includes:

[0157] When the size of the image to be predicted is smaller than the second size threshold, input the image to be predicted into the target prediction model, and identify the image to be predicted according to the target detection model to obtain a second prediction result corresponding to the image to be predicted;

[0158] Determine the target prediction result according to each of the first prediction results and the second prediction result.

[0159] In the embodiments of the present disclosure, when the size of the image to be predicted is smaller than the second size threshold, the image to be predicted can be input into the target prediction model, and the image to be predicted can be identified according to the target detection model to obtain a second prediction result corresponding to the image to be predicted.

[0160] Since when the size of the image to be predicted is greater than or equal to the second size threshold, the image to be predicted cannot be directly input into the target detection model, and when the size of the image to be predicted is smaller than the second size threshold, the image to be predicted can be directly input into the target prediction model to obtain a second prediction result. For example, if the electronic device determines that the size of the image to be predicted is 50*50 and the second size threshold is 100*100, the electronic device can determine that the size of the image to be predicted (such as an aerial image) is smaller than the second size threshold. Then the electronic device can directly identify the entire image to be predicted to obtain a second prediction result corresponding to the image to be predicted. For example: the electronic device determines that the target prediction result corresponding to the image to be predicted may include that the first target object is a river, the second target object is a school, the actual size is 120*120, and the third target object is an airport, etc.

[0161] Then, the electronic device determines that the target prediction result includes that the first target object is a river, the second target object is a school, the actual size is 120*120, the third target object is an airport, etc. according to each of the first prediction results (for example, the electronic device determines that the first prediction result in the first segmented image includes that the first target object is a river, and determines that the first prediction result in the second segmented image includes that the second target object is a school, and the actual size is 120*120, etc.) and the second prediction result.

[0162] In a possible embodiment, when the size of the image to be predicted is smaller than the second size threshold, the electronic device can also directly identify the entire image to be predicted through the target detection model without performing segmentation processing, and directly determine the target prediction result corresponding to the image to be predicted.

[0163] In the embodiments of the present disclosure, when the size of the image to be predicted is smaller than the second size threshold, the image to be predicted is input into the target prediction model, and the target detection model is used to identify the image to be predicted to obtain a second prediction result corresponding to the image to be predicted. According to each of the first prediction results and the second prediction result, the target prediction result is determined. By adopting the above solution, it is possible to obtain both the second prediction result detected for the entire image and the first prediction result detected after segmentation at the same time, and then combine the two prediction results to obtain a more complete and accurate target prediction result, further improving the detection accuracy of the target detection model. Since the size of the image to be predicted has been determined to be small, it will not affect the detection efficiency of the target detection model.

[0164] In some embodiments, the image to be predicted includes: aerial images and / or satellite images, and the target prediction result at least includes: object type and object area; the process of segmenting the image to be predicted to obtain a segmented image includes:

[0165] Segmenting the aerial image and / or the satellite image to obtain the segmented image;

[0166] The step of using the target detection model to identify each of the segmented images to determine the target prediction result corresponding to the image to be predicted includes:

[0167] Using the target detection model to identify each of the segmented images to determine the object type and the object area of each object in the aerial image and / or the satellite image.

[0168] In the embodiments of the present disclosure, the image to be predicted may include large-size images such as aerial images and / or satellite images, and the target prediction result at least includes information such as object type and object area. The electronic device can first segment the large-size images such as aerial images and / or satellite images to obtain segmented images, and then use the trained target detection model to identify each of the segmented images to determine the object type and the object area of each object in the large-size images such as aerial images and / or satellite images. For example: the electronic device can determine that the rectangular area in the upper left corner of the aerial image is a school, the size of the rectangular area can be 100*70, and the circular area in the lower right corner of the aerial image is an airport, and the radius of the circular area can be 500, etc. as the prediction result.

[0169] In the embodiments of the present disclosure, by training an initial detection model according to the sample image and the attribute information, a target detection model is obtained. By using the target detection model to identify the image to be predicted, the target prediction result corresponding to the image to be predicted is determined. By inputting sample images of different sizes, and each sample image contains a complete sample object, the training time of the detection model can be reduced, and the recognition accuracy of the detection model can be improved, etc.

[0170] In a possible embodiment, the image processing method in the present disclosure can be applied to improve the efficiency of large-size target detection such as aviation and satellite images. Figure 3 It is a flowchart of an image processing method shown according to an exemplary embodiment, as Figure 3 shown, the image processing method mainly includes the following steps:

[0171] In step 301, an image is acquired.

[0172] Here, the image may at least include: images such as aviation and satellite images.

[0173] In step 302, according to the images acquired historically, the image to be processed is determined.

[0174] In step 303, the image to be processed is labeled to obtain labeling information.

[0175] Here, the labeling process may be to add labeling information to each sample object in the image to be processed, and the labeling information may include a labeled area (segmentation area).

[0176] In step 304, according to the labeling information, the labeled area is determined.

[0177] In step 305, according to the labeled area, the image to be processed is segmented to obtain a sample image.

[0178] Here, after determining the labeled area, the labeled area can also be enlarged. For example, if the determined labeled area is 10*10 and the magnification factor is 1.1, then the enlarged labeled area is 11*11.

[0179] In step 306, model training is performed according to the sample image and the labeling information.

[0180] Here, multiple sample images of different sizes and the corresponding labeling information of each sample can be input into the initial detection model for model training to obtain the configured parameters of the trained model.

[0181] In step 307, a target detection model is determined.

[0182] In step 308, a to-be-predicted image is determined according to the currently acquired image.

[0183] Here, the images acquired historically or the currently acquired image can also be subjected to region segmentation processing, with a part of the region being used as the to-be-processed image (or region), and another part of the region being used as the to-be-predicted image (or region), etc.

[0184] In step 309, the to-be-predicted image is segmented to obtain a segmented image.

[0185] Here, the to-be-predicted image can be subjected to overlapping image cutting processing to obtain a segmented image, where there is an overlapping region between two adjacent segmented images, and the sizes of the segmented images can be the same or different.

[0186] In step 310, the segmented image is detected by the target detection model to obtain a first prediction result.

[0187] Here, each segmented image corresponds to a first prediction result. The first prediction result can refer to the prediction result corresponding to a segmented image, and the prediction result can include annotation information.

[0188] In step 311, splicing processing is performed on each of the first prediction results.

[0189] Here, mainly because there is an overlapping region between adjacent segmented images, there is also overlapping content between the corresponding adjacent first prediction results. Therefore, it is necessary to perform splicing processing on the overlapping content between multiple first prediction results.

[0190] In step 312, a target prediction result is obtained.

[0191] Figure 4 It is a block diagram of a sample image generation device shown according to an exemplary embodiment. As Figure 4 shown, the sample image generation device 400 mainly includes:

[0192] A first determination module 401, configured to determine the attribute information of each sample object in the to-be-processed image;

[0193] A second determination module 402, configured to determine the segmentation region corresponding to each sample object according to the attribute information;

[0194] A first segmentation module 403, configured to segment the to-be-processed image according to each segmentation region to obtain a sample image corresponding to each sample object.

[0195] In some embodiments, the attribute information includes: the annotation information of the sample object;

[0196] The second determination module 402 is configured to:

[0197] Determine the segmentation granularity of the sample object according to the annotation information and a preset relationship, where the preset relationship is used to represent the mapping relationship between the annotation information and the segmentation granularity;

[0198] Determine the segmentation region corresponding to each sample object according to the segmentation granularity.

[0199] In some embodiments, the second determination module 402 is configured to:

[0200] When the segmentation granularity is greater than a first size threshold, determine the segmentation region according to the segmentation granularity of the sample object.

[0201] In some embodiments, the second determination module 402 is configured to:

[0202] When the segmentation granularity is less than or equal to the first size threshold, determine the affiliated object of the sample object;

[0203] When the segmentation granularity of the affiliated object is greater than the first size threshold, determine the segmentation region of the sample object according to the segmentation granularity of the affiliated object.

[0204] In some embodiments, the apparatus 400 further includes:

[0205] A scaling module, configured to perform a scaling process on the segmentation region according to a scaling threshold to obtain a scaled segmentation region;

[0206] A processing module, configured to perform a segmentation process on the image to be processed according to the scaled segmentation region to obtain a sample image corresponding to each sample object.

[0207] Figure 5 It is a block diagram of an image prediction apparatus shown according to an exemplary embodiment. As Figure 5 shown, the image prediction apparatus 500 mainly includes:

[0208] A second segmentation module 501, configured to perform a segmentation process on the image to be predicted to obtain a segmented image;

[0209] An identification module 502, configured to identify each segmented image according to a target detection model to determine a target prediction result corresponding to the image to be predicted;

[0210] Among them, the target detection model is obtained by training an initial detection model according to sample images segmented from the image to be processed and attribute information of the sample objects, and the sample images are obtained by segmenting the image to be processed based on the attribute information of the sample objects.

[0211] In some embodiments, there is an overlapping area between two adjacent segmentation images; the recognition module 502 is configured to:

[0212] Identify each of the segmentation images according to the target detection model to determine a first prediction result corresponding to each of the segmentation images;

[0213] Obtain a target prediction result corresponding to the image to be predicted based on at least each of the first prediction results.

[0214] In some embodiments, the recognition module 502 is configured to:

[0215] When the size of the image to be predicted is greater than or equal to a second size threshold, perform a splicing process on each of the first prediction results to obtain the target prediction result.

[0216] In some embodiments, the recognition module 502 is configured to:

[0217] When the size of the image to be predicted is less than the second size threshold, input the image to be predicted into the target prediction model, and identify the image to be predicted according to the target detection model to obtain a second prediction result corresponding to the image to be predicted;

[0218] Determine the target prediction result according to each of the first prediction results and the second prediction result.

[0219] In some embodiments, the image to be predicted includes: aerial images and / or satellite images, and the target prediction result at least includes: object type and object area; the second segmentation module 501 is configured to:

[0220] Perform a segmentation process on the aerial images and / or the satellite images to obtain the segmentation images;

[0221] The recognition module 502 is configured to:

[0222] Identify each of the segmentation images according to the target detection model to determine the object type and the object area of each object in the aerial images and / or the satellite images.

[0223] Regarding the device in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be elaborated here.

[0224] Figure 6 It is a hardware structure block diagram of an electronic device shown according to an exemplary embodiment. For example, device 600 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0225] Referring to Figure 6 , device 600 may include one or more of the following components: a processing component 602, a memory 604, a power supply component 606, a multimedia component 608, an audio component 610, an input / output (I / O) interface 612, a sensor component 614, and a communication component 616.

[0226] The processing component 602 generally controls the overall operation of device 600, such as operations associated with display, telephone calls, data communication, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above methods. In addition, the processing component 602 may include one or more modules to facilitate the interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.

[0227] The memory 604 is configured to store various types of data to support the operation of device 600. Examples of such data include instructions for any application or method operating on device 600, contact data, phone book data, messages, pictures, videos, etc. The memory 604 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk.

[0228] The power supply component 606 provides power to various components of device 600. The power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for device 600.

[0229] The multimedia component 608 includes a screen that provides an output interface between the device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of touch or swipe actions but also detect the duration and pressure associated with the touch or swipe operation. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.

[0230] The audio component 610 is configured to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC) that is configured to receive external audio signals when the device 600 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 604 or transmitted via the communication component 616. In some embodiments, the audio component 610 further includes a speaker for outputting audio signals.

[0231] The I / O interface 612 provides an interface between the processing component 602 and a peripheral interface module, and the peripheral interface module can be a keyboard, a click wheel, buttons, etc. These buttons can include but are not limited to: a home button, a volume button, a power button, and a lock button.

[0232] The sensor component 614 includes one or more sensors for providing a status assessment of various aspects of the device 600. For example, the sensor component 614 can detect the on / off state of the device 600, the relative positioning of components, such as the display and the keypad of the device 600. The sensor component 614 can also detect a change in the position of the device 600 or a component of the device 600, the presence or absence of user contact with the device 600, the orientation or acceleration / deceleration of the device 600, and the temperature change of the device 600. The sensor component 614 can include a proximity sensor that is configured to detect the presence of nearby objects without any physical contact. The sensor component 614 can also include a light sensor, such as a CMOS or a CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 614 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0233] The communication component 616 is configured to facilitate communication between the device 600 and other devices in a wired or wireless manner. The device 600 can access a communication standard-based wireless network, such as WI-FI, 4G, or 5G, or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0234] In an exemplary embodiment, the device 600 can be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above method.

[0235] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, and the above instructions can be executed by a processor 620 of the device 600 to complete the above method. For example, the non-transitory computer-readable storage medium can be a ROM, Random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0236] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute a method for generating a sample image, including:

[0237] Determine the attribute information of each sample object in the image to be processed;

[0238] According to the attribute information, determine the segmentation region corresponding to each sample object;

[0239] Perform segmentation processing on the image to be processed according to each segmentation region to obtain a sample image corresponding to each sample object.

[0240] Or enables the electronic device to execute an image prediction method, including:

[0241] Perform segmentation processing on the image to be predicted to obtain a segmented image;

[0242] Identify each segmented image according to a target detection model to determine the target prediction result corresponding to the image to be predicted;

[0243] Among them, the target detection model is obtained by training an initial detection model according to sample images segmented from the image to be processed and the attribute information of the sample objects, and the sample images are obtained by segmenting the image to be processed based on the attribute information of the sample objects.

[0244] Figure 7 It is a hardware structure block diagram of an electronic device 700 shown according to an exemplary embodiment. For example, the device 700 can be provided as a server. Referring to Figure 7 , the device 700 includes a processing component 722, which further includes one or more processors, and memory resources represented by a memory 732 for storing instructions executable by the processing component 722, such as application programs. The application programs stored in the memory 732 can include one or more modules each corresponding to a set of instructions. In addition, the processing component 722 is configured to execute instructions to perform a method for generating sample images, including:

[0245] Determine the attribute information of each sample object in the image to be processed;

[0246] According to the attribute information, determine the segmentation regions corresponding to each of the sample objects;

[0247] Segment the image to be processed according to each of the segmentation regions to obtain sample images corresponding to each of the sample objects.

[0248] Or to execute an image prediction method, including:

[0249] Perform segmentation processing on the image to be predicted to obtain a segmented image;

[0250] Identify each of the segmented images according to the target detection model to determine the target prediction result corresponding to the image to be predicted;

[0251] Among them, the target detection model is obtained by training an initial detection model according to sample images segmented from the image to be processed and the attribute information of the sample objects, and the sample images are obtained by segmenting the image to be processed based on the attribute information of the sample objects.

[0252] The device 700 may further include a power supply component 726 configured to perform power management of the device 700, a wired or wireless network interface 750 configured to connect the device 700 to a network, and an input / output (I / O) interface 758. The device 700 can operate based on an operating system stored in the memory 732, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM or the like.

[0253] Other embodiments of the present disclosure will be readily apparent to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common general knowledge or conventional technical means in the technical field not disclosed herein. The specification and examples are only to be considered as exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0254] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. A method for generating a sample image, characterized in that, the method includes: Determine the attribute information of each sample object in the image to be processed; wherein, the attribute information includes: the annotation information of the sample object; Determine the segmentation granularity of the sample object according to the annotation information and a preset relationship; wherein, the preset relationship is used to represent the mapping relationship between the annotation information and the segmentation granularity; Determine the segmentation region corresponding to each sample object according to the segmentation granularity; Perform segmentation processing on the image to be processed according to each segmentation region to obtain a sample image corresponding to each sample object; wherein, the step of determining the segmentation region corresponding to each sample object according to the segmentation granularity includes: When the segmentation granularity is less than or equal to a first size threshold, determine the affiliated object of the sample object; When the segmentation granularity of the affiliated object is greater than the first size threshold, determine the segmentation region of the sample object according to the segmentation granularity of the affiliated object; The image to be processed includes: aerial images and / or satellite images.

2. The method according to claim 1, characterized in that, the step of determining the segmentation region corresponding to each sample object according to the segmentation granularity includes: When the segmentation granularity is greater than the first size threshold, determine the segmentation region according to the segmentation granularity of the sample object.

3. The method according to claim 1, characterized in that, Before determining the segmentation region corresponding to each sample object according to the attribute information, the method further includes: Perform scaling processing on the segmentation region according to a scaling threshold to obtain a scaled segmentation region; Perform segmentation processing on the image to be processed according to the scaled segmentation region to obtain a sample image corresponding to each sample object.

4. An image prediction method, characterized in that, the method includes: Perform segmentation processing on the image to be predicted to obtain a segmented image; Identify each segmented image according to a target detection model to determine the target prediction result corresponding to the image to be predicted; wherein, the target detection model is: obtained by training an initial detection model according to the sample images segmented from the image to be processed and the attribute information of the sample objects, the sample images are obtained by performing segmentation on the image to be processed based on the segmentation regions of the sample objects, the segmentation regions are determined based on the segmentation granularity of the affiliated objects of the sample objects when the segmentation granularity of the affiliated objects is greater than the first size threshold; the affiliated objects are determined when the segmentation granularity of the sample objects is less than or equal to the first size threshold; the segmentation granularity of the sample objects is determined based on the annotation information included in the attribute information of the sample objects and a preset relationship; the preset relationship is used to represent the mapping relationship between the annotation information and the segmentation granularity; the image to be processed includes: aerial images and / or satellite images.

5. The method according to claim 4, characterized in that, There is an overlapping area between two adjacent ones of the segmented images; the identifying each of the segmented images according to the target detection model to determine a target prediction result corresponding to the image to be predicted includes: Identifying each of the segmented images according to the target detection model to determine a first prediction result corresponding to each of the segmented images; Obtaining a target prediction result corresponding to the image to be predicted at least according to each of the first prediction results.

6. The method according to claim 5, wherein, the obtaining a target prediction result corresponding to the image to be predicted at least according to each of the first prediction results includes: When the size of the image to be predicted is greater than or equal to a second size threshold, performing a splicing process on each of the first prediction results to obtain the target prediction result.

7. The method according to claim 5, wherein, the obtaining a target prediction result corresponding to the image to be predicted at least according to each of the first prediction results includes: When the size of the image to be predicted is less than the second size threshold, inputting the image to be predicted into the target prediction model, and identifying the image to be predicted according to the target detection model to obtain a second prediction result corresponding to the image to be predicted; Determining the target prediction result according to each of the first prediction results and the second prediction result.

8. The method according to claim 4, wherein, the image to be predicted includes: aerial image and / or satellite image, and the target prediction result at least includes: object type and object area; the performing a segmentation process on the image to be predicted to obtain segmented images includes: Performing a segmentation process on the aerial image and / or the satellite image to obtain the segmented images; the identifying each of the segmented images according to the target detection model to determine a target prediction result corresponding to the image to be predicted includes: Identifying each of the segmented images according to the target detection model to determine the object type and the object area of each object in the aerial image and / or the satellite image.

9. A sample image generation device, wherein, it includes: A first determination module configured to determine attribute information of each sample object in the image to be processed; wherein, the attribute information includes: annotation information of the sample object; A second determination module configured to: determine a segmentation granularity of the sample object according to the annotation information and a preset relationship; wherein, the preset relationship is used to represent a mapping relationship between the annotation information and the segmentation granularity; and determine a segmentation area corresponding to each sample object according to the segmentation granularity; A first segmentation module configured to perform a segmentation process on the image to be processed according to each segmentation area to obtain a sample image corresponding to each sample object; wherein, the second determination module is specifically configured to: When the segmentation granularity is less than or equal to the first size threshold, determine the affiliated object of the sample object; when the segmentation granularity of the affiliated object is greater than the first size threshold, determine the segmentation region of the sample object according to the segmentation granularity of the affiliated object. The image to be processed includes: aerial images and / or satellite images.

10. An image prediction device Characterized in that It includes: A second segmentation module configured to perform segmentation processing on the image to be predicted to obtain a segmented image; An identification module configured to identify each of the segmented images according to a target detection model to determine a target prediction result corresponding to the image to be predicted; Wherein, the target detection model is: obtained by training an initial detection model according to sample images and attribute information of sample objects segmented from the image to be processed, the sample images are obtained by segmenting the image to be processed based on the segmentation region of the sample object, the segmentation region is determined based on the segmentation granularity of the affiliated object when the segmentation granularity of the affiliated object of the sample object is greater than the first size threshold; the affiliated object is determined when the segmentation granularity of the sample object is less than or equal to the first size threshold; the segmentation granularity of the sample object is determined based on the annotation information included in the attribute information of the sample object and a preset relationship; the preset relationship is used to represent the mapping relationship between the annotation information and the segmentation granularity; the image to be processed includes: aerial images and / or satellite images.

11. An electronic device Characterized in that It includes: A processor; A memory configured to store instructions executable by the processor; Wherein, the processor is configured to: when executed, implement the steps in any one of the sample image generation methods in claims 1 to 3 above or any one of the image prediction methods in claims 4 to 8 above.

12. A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of an electronic device, enabling the electronic device to execute the steps in any one of the sample image generation methods in claims 1 to 3 above or any one of the image prediction methods in claims 4 to 8 above.

Images