Pixel instance rendering method and device, storage medium and electronic equipment

Abstract

The invention discloses a pixel instance rendering method and device, a storage medium and electronic equipment. The method comprises the following steps: acquiring a pre-shot target image; a target image is input into a target segmentation model, an instance segmentation result and a semantic segmentation result matched with at least one pixel instance are obtained, the instance segmentation result is used for representing whether the category of each pixel instance in the at least one pixel instance is a first-level category, and the semantic segmentation result is used for representing whether the category of each pixel instance in the at least one pixel instance is a second-level category. The output of a pixel decoder in the main body network is the input of a semantic segmentation decoder in the semantic segmentation network; based on the instance segmentation result, performing vectorization processing on the at least one pixel instance to obtain a vectorization result; and rendering the at least one pixel instance based on the vectorization result, the semantic segmentation result and the pixel coordinates of the at least one pixel instance. According to the method and the device, the technical problem of relatively low accuracy in the rendering process of the pixel instance in related technologies is solved.

Description

Technical Field

[0001] This application relates to the field of image processing, and more specifically, to a method and apparatus for rendering an instance of an image pixel, a storage medium, and an electronic device. Background Technology

[0002] In related technologies, to ensure that farmland instances in maps better represent real-world farmland, operators typically draw vector outlines of the farmland directly from satellite imagery. However, this manual method has a long production cycle and cannot be automatically updated, increasing labor and maintenance costs.

[0003] To address these issues, some related technologies have proposed automatically generating farmland map data by directly using open-source data (e.g., farmland data reported by various regions). However, in the real world, farmland varies greatly in form and location, including irregularly shaped plots and terraced fields. Therefore, the farmland data reported from open-source sources may differ significantly from the actual farmland data. For example, there may be data omissions and errors, resulting in the technical problem of low accuracy in automatically generated farmland map data using open-source data.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides a method and apparatus for rendering pixel instances, a storage medium, and an electronic device to at least solve the technical problem of low accuracy in identifying abnormal accounts.

[0006] According to one aspect of the embodiments of this application, a rendering method for a pixel instance is provided, comprising: acquiring a pre-captured target image, wherein the target image includes at least one pixel instance formed according to a preset shape, and one pixel instance in the at least one pixel instance represents a map pixel; inputting the target image into a target segmentation model to obtain an instance segmentation result and a semantic segmentation result matching the at least one pixel instance, wherein the instance segmentation result is used to indicate whether the category of each pixel instance in the at least one pixel instance is a first-level category, and the semantic segmentation result is used to indicate the second-level category to which each pixel instance belongs, the second-level category being a subcategory of the first-level category, the target segmentation model including a main network and a semantic segmentation network, the output of the pixel decoder in the main network being the input of the semantic segmentation decoder in the semantic segmentation network; vectorizing the at least one pixel instance based on the instance segmentation result to obtain a vectorized result; and rendering the at least one pixel instance based on the vectorized result, the semantic segmentation result, and the pixel coordinates of the at least one pixel instance.

[0007] Optionally, the above-described method of inputting the target image into a target segmentation model to obtain instance segmentation results and semantic segmentation results matching at least one pixel instance includes: sequentially obtaining each pixel instance from at least one pixel instance as the current pixel instance; inputting the target image into a pixel decoder to obtain a feature map with the same resolution as the target image; inputting the feature map into a target converter in the main network to obtain a current instance segmentation result matching the current pixel instance; and inputting the feature map into a semantic segmentation decoder to obtain a current semantic segmentation result matching the current pixel instance.

[0008] Optionally, the above-mentioned method of inputting the feature map into the semantic segmentation decoder to obtain the current semantic segmentation result matching the current pixel instance includes: inputting the feature map into the semantic segmentation decoder to obtain the secondary category to which each pixel belongs in the first set of pixels corresponding to the current pixel instance; and generating a first array to represent the current semantic segmentation result based on the secondary category to which each pixel belongs, wherein each element in the first array represents the secondary category to which each pixel belongs.

[0009] Optionally, the above method of obtaining a current instance segmentation result matching the current pixel instance by inputting the feature map into the target converter in the main network includes: inputting the feature map into the target converter, performing a multiplication operation between the target query vector and the feature map to obtain a probability distribution vector, wherein the probability distribution vector is used to describe the probability that each pixel in the first pixel set corresponding to the current pixel instance belongs to the first-level category, and the target query vector is used to describe the specific information that needs to be focused on from the feature map; and determining the current instance segmentation result based on the probability distribution vector.

[0010] Optionally, the above-mentioned vectorization process of at least one pixel instance based on the instance segmentation result to obtain the vectorization result includes: sequentially obtaining each pixel instance from the at least one pixel instance as the current pixel instance; determining the geometric contour of the current pixel instance based on an edge detection algorithm; extracting a set of boundary pixels from the set of second pixels at each position contained in the geometric contour based on a polygon approximation algorithm; converting the pixel coordinates of the set of boundary pixels into a set of contour coordinates; and determining the vectorization result of the current pixel instance based on the set of contour coordinates.

[0011] Optionally, the above method for determining the vectorization result of the current graphic element instance based on a set of contour coordinates includes: sequentially connecting the coordinate points corresponding to the set of contour coordinates to obtain a vector graphic that satisfies a preset geometric shape; and determining the vector graphic that satisfies the preset geometric shape as the vectorization result.

[0012] Optionally, the above-mentioned rendering of at least one pixel instance based on the vectorization result, semantic segmentation result, and pixel coordinates of at least one pixel instance includes: sequentially obtaining each pixel instance from the at least one pixel instance as the current pixel instance, and determining the current semantic segmentation result corresponding to the current pixel instance from the semantic segmentation result; determining the current instance segmentation result corresponding to the current pixel instance from the instance segmentation result; generating the current vectorization result based on the current instance segmentation result; and rendering the current pixel instance based on the current semantic segmentation result, the current vectorization result, and the pixel coordinates of each pixel in the first pixel set corresponding to the current pixel instance.

[0013] Optionally, the above-mentioned rendering of the current pixel instance based on the current semantic segmentation result, the current vectorization result, and the pixel coordinates of each pixel in the first pixel set corresponding to the current pixel instance includes: determining the target secondary category among the N secondary categories to which the current pixel instance belongs based on the current semantic segmentation result, where N is a positive integer greater than or equal to 2; converting the current pixel instance into a target pixel instance carrying the target secondary category based on the target secondary category; and rendering the current pixel instance based on the target pixel instance and the pixel coordinates of each pixel in the first pixel set.

[0014] Optionally, the above-mentioned determination of the target secondary category among the N secondary categories to which the current pixel instance belongs based on the current semantic segmentation result includes: determining the secondary category to which each pixel in the first pixel set corresponding to the current pixel instance belongs based on the first array corresponding to the current semantic segmentation result, wherein each element in the first array represents the secondary category to which each pixel belongs; dividing the first pixel set into N groups of pixels based on the secondary category to which each pixel in the first pixel set belongs, wherein one group of pixels in the N groups of pixels includes pixels belonging to one of the N secondary categories; determining the group of pixels with the largest number of pixels from the N groups of pixels, and determining the secondary category to which each pixel in the group with the largest number of pixels belongs as the target secondary category to which the current pixel instance belongs.

[0015] Optionally, the above-mentioned rendering of the current pixel instance based on the target pixel instance and the pixel coordinates of each pixel in the first pixel set includes: obtaining the target latitude and longitude of the target location when the target image was captured; converting the pixel coordinates of each pixel in the first pixel set into a set of latitude and longitude coordinates based on the target latitude and longitude coordinates; and rendering the current pixel instance based on the set of latitude and longitude coordinates and the target style indicated by the target secondary category carried by the target pixel instance.

[0016] Optionally, the above method further includes: when at least one pixel instance is a farmland instance, by inputting the target image into a target segmentation model, obtaining a farmland instance segmentation result and a farmland semantic segmentation result that match the farmland instance, wherein the farmland instance segmentation result is used to indicate whether the category of the farmland instance is farmland, and the farmland semantic segmentation result is used to indicate the sub-category to which the farmland instance belongs, farmland includes N sub-categories, each of the N sub-categories represents a different shape of farmland, and N is a positive integer greater than or equal to 2; based on the farmland instance segmentation result, at least one farmland instance is vectorized to obtain a vectorized result; based on the vectorized result, the farmland semantic segmentation result, and the pixel coordinates of the farmland instance, the farmland instance is rendered.

[0017] According to another aspect of the embodiments of this application, a rendering apparatus for a pixel instance is also provided, comprising: a first acquisition unit, configured to acquire a pre-captured target image, wherein the target image includes at least one pixel instance formed according to a preset shape, and one pixel instance in the at least one pixel instance represents a map pixel; a first processing unit, configured to input the target image into a target segmentation model to obtain an instance segmentation result and a semantic segmentation result matching the at least one pixel instance, wherein the instance segmentation result is used to indicate whether the category of each pixel instance in the at least one pixel instance is a first-level category, and the semantic segmentation result is used to indicate the second-level category to which each pixel instance belongs, the second-level category being a subcategory of the first-level category, the target segmentation model including a main network and a semantic segmentation network, the output of the pixel decoder in the main network being the input of the semantic segmentation decoder in the semantic segmentation network; a second processing unit, configured to perform vectorization processing on the at least one pixel instance based on the instance segmentation result to obtain a vectorization result; and a rendering unit, configured to render the at least one pixel instance based on the vectorization result, the semantic segmentation result, and the pixel coordinates of the at least one pixel instance.

[0018] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program for executing the rendering method of the above-described graphic element instance when the electronic device is run.

[0019] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0020] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to execute a rendering method for the above-described graphic element instance through the computer program.

[0021] The embodiments provided in this application utilize a dual-branch structure of a main network and a semantic segmentation network in the target segmentation model. The main network outputs an instance segmentation result representing the category probability distribution of each pixel in at least one pixel instance, while the semantic segmentation network outputs a semantic segmentation result representing the subdivided semantic category of each pixel in at least one pixel instance. Based on the instance segmentation result and the semantic segmentation result, the pixel information of the pixel instance is vector-processed, and rendered according to the pixel coordinates of the pixel instance and the vector processing result. In other words, by adopting the above-mentioned dual-branch model structure, the real-time global features and local details of at least one pixel in the target image can be captured simultaneously, improving the accuracy of the pixel instance category and its subcategories output by the model, and achieving the technical effect of improving the accuracy of the pixel instance rendering process. Attached Figure Description

[0022] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application.

[0023] Figure 1 This is a schematic diagram illustrating an application scenario of an optional pixel instance rendering method according to an embodiment of this application;

[0024] Figure 2 This is a flowchart of an optional pixel instance rendering method according to an embodiment of this application;

[0025] Figure 3 This is an overall schematic diagram of an optional target segmentation model according to an embodiment of this application;

[0026] Figure 4 This is an optional data-annotated image according to an embodiment of this application;

[0027] Figure 5 This is a schematic diagram of the structure of an optional Mask2Former model according to an embodiment of this application;

[0028] Figure 6 This is a binary image of an optional farmland mask according to an embodiment of this application;

[0029] Figure 7 It is a binary image of an optional farmland subdivision semantic mask according to an embodiment of this application;

[0030] Figure 8 This is a schematic diagram illustrating an optional voting method for determining farmland subcategories according to an embodiment of this application;

[0031] Figure 9This is a schematic diagram illustrating an optional process for obtaining farmland instances with subdivided semantic categories according to an embodiment of this application;

[0032] Figure 10 This is a schematic diagram of the input and output of an optional target segmentation model according to an embodiment of this application;

[0033] Figure 11 This is an overall flowchart of an optional pixel instance rendering method according to an embodiment of this application;

[0034] Figure 12 This is a schematic diagram illustrating the rendering effect of farmland on mobile phone maps and vehicle maps using embodiments of this application;

[0035] Figure 13 This is a schematic diagram of the structure of a rendering apparatus for an optional pixel instance according to an embodiment of this application;

[0036] Figure 14 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0037] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0038] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0039] The technical solutions in this application will comply with legal regulations during implementation. When performing operations according to the technical solutions in the embodiments, the data used will not involve user privacy, ensuring that the operation process is compliant and legal while guaranteeing data security.

[0040] In addition, when the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant regulations and standards of the relevant countries or regions.

[0041] According to one aspect of the embodiments of this application, a method for rendering a pixel instance is provided. As an optional implementation, the above-described method for rendering a pixel instance can be applied, but is not limited to, to applications such as... Figure 1 The application scenarios shown are as follows. In, for example... Figure 1 In the application scenario shown, the target terminal 102 can communicate with the server 106 via network 104, but is not limited to this. The server 106 can perform operations on the database 108, such as write or read data operations. The target terminal 102 may include, but is not limited to, a human-computer interaction screen, a processor, and a memory. The human-computer interaction screen may be used to display target images, rendered pixel instances, etc., on the target terminal 102. The processor may be used to respond to the human-computer interaction operations, execute corresponding operations, or generate corresponding instructions and send the generated instructions to the server 106. The memory is used to store relevant processing data, such as target images, at least one pixel instance, and semantic segmentation results.

[0042] Optionally, in this embodiment, the target terminal can be a terminal configured with a target client, which may include, but is not limited to, at least one of the following: mobile phone (such as Android phone, iOS phone, etc.), laptop computer, tablet computer, PDA, MID (Mobile Internet Devices), PAD, desktop computer, smart TV, etc. The target client may be a video client, instant messaging client, browser client, educational client, etc. The network may include, but is not limited to, wired network and wireless network, wherein the wired network includes: local area network, metropolitan area network and wide area network, and the wireless network includes: Bluetooth, WIFI and other networks that enable wireless communication. The server may be a single server, a server cluster composed of multiple servers, or a cloud server.

[0043] The technical solutions in this application embodiment can be applied, but are not limited to, to the automated production of map farmland data. By designing a multi-level, multi-branch instance segmentation deep learning model, farmland and its subcategories are identified, such as irregular block farmland, regular block farmland, terraced fields, and flat farmland. This achieves automated production of map farmland data, improves the accuracy of farmland geometric contours and subcategories, and enhances the rendering effect of map farmland.

[0044] To address the aforementioned issue of low accuracy in improving the rendering accuracy of pixel instances, this application proposes a method for rendering pixel instances. Figure 2 This is a flowchart of a method for rendering a pixel instance according to an embodiment of this application. The process includes the following steps S202 to S208.

[0045] It should be noted that the rendering method of the pixel instances shown in steps S202 to S208 can be, but is not limited to, executed by an electronic device. The electronic device can be, but is not limited to, a method such as... Figure 1 The target terminal or server shown.

[0046] Step S202: Obtain a pre-captured target image, wherein the target image includes at least one pixel instance formed according to a preset shape, and one pixel instance in the at least one pixel instance represents a map pixel.

[0047] Step S204: By inputting the target image into the target segmentation model, an instance segmentation result and a semantic segmentation result matching at least one pixel instance are obtained. The instance segmentation result is used to indicate whether the category of each pixel instance in the at least one pixel instance is a first-level category, and the semantic segmentation result is used to indicate the second-level category to which each pixel instance belongs. The second-level category is a subcategory of the first-level category. The target segmentation model includes a main network and a semantic segmentation network. The output of the pixel decoder in the main network is the input of the semantic segmentation decoder in the semantic segmentation network.

[0048] Step S206: Based on the instance segmentation result, at least one pixel instance is vectorized to obtain the vectorized result;

[0049] Step S208: Render at least one pixel instance based on the vectorization result, semantic segmentation result, and pixel coordinates of at least one pixel instance.

[0050] For ease of understanding, the embodiments of this application all use farmland instances in the map farmland as graphic element instances to explain and illustrate the rendering methods of the above graphic element embodiments.

[0051] In map applications, green spaces, water bodies, buildings, roads, etc. are usually rendered as elements of Beijing. However, in the real world, farmland, as a typical type of land cover, accounts for a very large proportion and is of great importance, and it is different from green spaces.

[0052] Adding farmland rendering to maps not only accurately depicts the physical world but also enhances the user's immersive experience. Farmland rendering relies on map farmland data, which typically refers to spatial data used in Geographic Information Systems (GIS) to represent the location, extent, type, and other relevant information of farmland. This data includes, but is not limited to, farmland boundaries, crop types, and planted areas. Farmland data has wide applications in agricultural planning, resource management, and environmental protection.

[0053] In practical applications, farmland takes many forms, including, for example, Figure 4 The data shows regular, irregular, terraced, and flat farmland. However, due to quality concerns, automated identification of farmland geometry and subcategories is very difficult. Among existing open-source datasets, the highest accuracy is achieved with the SinoLC-1 dataset, which provides a 1m resolution land cover dataset including cultivated land. However, its overall accuracy is only 73.61%, and it is only a semantic mask dataset, resulting in low precision and insufficient accuracy, failing to meet the quality standards for map farmland data.

[0054] To address the aforementioned issues, this application proposes a multi-level, multi-branch deep learning model. The target segmentation model (i.e., the multi-level, multi-branch deep learning model) used in this application will be briefly introduced below.

[0055] like Figure 3 As shown, the above target segmentation model is based on the existing deep learning model (Mask2Former) and adds a semantic segmentation branch, such as... Figure 3 The semantic segmentation network shown predicts the detailed semantics of farmland.

[0056] Mask2Former is a deep learning model based on the Transformer architecture, used to handle general image segmentation tasks, including semantic segmentation, instance segmentation, and panoptic segmentation. The core of this model lies in its masked attention mechanism, which extracts local features by restricting attention computation to the predicted masked region.

[0057] The main features of Mask2Former are as follows:

[0058] (1) Masked Attention: Through Masked Attention, the model restricts cross-attention to the foreground part of the feature map and uses the query feature to directly focus on useful local regions, instead of requiring the network to learn to specific attention from the global to the local. This Masked Attention can speed up convergence and improve accuracy.

[0059] (2) Multi-scale high-resolution features: In order to help the model segment small objects, Mask2Former uses multi-scale high-resolution features, which are input into the Transformer decoder through a pyramid-shaped decoder.

[0060] (3) Optimization and improvement: The model proposes some optimization methods, such as swapping the order of the self-attention mechanism and the cross-attention mechanism to make the query embedding learnable, and removing them, which all help to improve the performance of the model.

[0061] (4) Loss calculation at random sampling points: In order to reduce memory consumption, Mask2Former calculates the loss at random sampling points instead of all of them. This can reduce memory usage without affecting performance.

[0062] Using the existing Mask2Former model, farmland instances in the target image can be effectively segmented, that is, it can be determined whether the image contains farmland instances.

[0063] As described in the above embodiments, existing deep learning models (Mask2Former) can only segment farmland instances in the target image, but cannot accurately output the subdivision category of each farmland instance. Therefore, in situations such as Figure 3 Based on the Mask2Former model shown, a semantic segmentation network is added. The backbone in the main network is used for feature extraction, and the pixel decoder is used for processing to obtain high-resolution feature maps. By inputting the high-resolution feature maps into the semantic segmentation decoder and the target transformer decoder, respectively, the outputs represent the farmland sub-category of each pixel instance (which can be understood as semantic segmentation results) and the instance segmentation results of whether it is farmland or not.

[0064] The semantic segmentation decoder mainly converts the image features extracted by the encoder into pixel-level classification results, that is, it generates a category label for each pixel. Based on the category label of each pixel, it realizes the semantic understanding of each region in the image.

[0065] The backbone is the part of an instance segmentation model used for feature extraction. It is usually part of a convolutional neural network and is responsible for extracting useful features from the input image. It determines how much useful information the model can extract from the image.

[0066] The pixel decoder plays a role in instance segmentation models by converting low-level visual information into high-level semantic information. It provides the necessary high-resolution and multi-scale features for instance segmentation, and the feature maps output by the pixel decoder provide rich contextual information, which helps to distinguish different farmland instances and subcategories.

[0067] In this embodiment, the semantic category of each pixel in the target image output by the semantic segmentation decoder is used to determine the sub-category to which each farmland instance (which can also be understood as a pixel instance) in the target image belongs. For example, farmland instance 1 belongs to irregular block farmland in farmland (first-level category), farmland instance 2 belongs to terraced fields in farmland, etc.

[0068] It should be noted that both the instance segmentation results and the semantic segmentation results mentioned above are two-dimensional array structures. In the fields of computer vision and image processing, a mask is an image representation method used to represent a specific region. The farmland mask (which can also be understood as the instance segmentation result) is a binary image in which the value of each pixel indicates whether the pixel belongs to the farmland region. The value of each pixel in the binary image corresponding to the farmland semantic mask is used to indicate whether the pixel belongs to one of the subcategories of the farmland category.

[0069] After using the above target segmentation model to output the instance segmentation result and semantic segmentation result of at least one pixel instance in the target image, based on the instance segmentation result, the pixel information of at least one farmland instance is converted into vector graphics. For example, based on the farmland instance, the geometric outline of a farmland is determined. Finally, based on the vector graphics, the semantic segmentation result and the pixel coordinates of at least one pixel instance, at least one pixel instance is rendered.

[0070] Furthermore, if the image input to the target segmentation model does not contain farmland, the elements in the two-dimensional arrays corresponding to the instance segmentation results and semantic segmentation results output by the model are all 0.

[0071] The above approach utilizes a dual-branch structure of a main network and a semantic segmentation network in the target segmentation model. The main network outputs an instance segmentation result representing the category probability distribution of each pixel in at least one pixel instance, while the semantic segmentation network outputs a semantic segmentation result representing the subdivided semantic category of each pixel in at least one pixel instance. Based on the instance segmentation result and the semantic segmentation result, the pixel information of the pixel instance is vector-processed, and rendered according to the pixel coordinates of the pixel instance and the vector processing result. In other words, by adopting the above dual-branch model structure, it is possible to simultaneously capture the real-time global features and local details of at least one pixel in the target image, improving the accuracy of the pixel instance category and its subcategories output by the model, and achieving the technical effect of improving the accuracy of the pixel instance rendering process.

[0072] As an optional example, the above method, by inputting the target image into a target segmentation model, obtains instance segmentation results and semantic segmentation results that match at least one pixel instance, including:

[0073] Each graph element instance is sequentially obtained from at least one graph element instance as the current graph element instance;

[0074] By inputting the target image into the pixel decoder, a feature map with the same resolution as the target image is obtained;

[0075] By inputting the feature map into the target transformer in the main network, a current instance segmentation result matching the current pixel instance is obtained;

[0076] By inputting the feature map into the semantic segmentation decoder, the current semantic segmentation result that matches the current pixel instance is obtained.

[0077] As can be seen from the description in the above embodiments, the input of the above target segmentation model is a pre-acquired satellite image, and the output is a farmland mask (instance segmentation result) and a farmland subdivision semantic mask (semantic segmentation result) corresponding to each pixel instance in the image.

[0078] Before using the above target segmentation model to output the farmland mask and farmland subdivision semantic mask in the satellite image, it is also necessary to further process, such as Figure 3 The pre-built network model shown is used for training and testing, and the labeling process of the data used in the training process is particularly important.

[0079] Specifically, in order to support the training of the recognition model, this application embodiment designs a labeling scheme for farmland in satellite imagery. The original image to be labeled can be, but is not limited to, an optical satellite image of size 1024×1024 with a resolution of 0.5. The labelers need to label the farmland in it and label its sub-categories, including irregular block farmland, regular block farmland, terraced fields and flat farmland.

[0080] Examples of labeled data are as follows Figure 4 As shown, Image 1 is a regular block of farmland composed of irregular polygons, Image 2 is a regular block of farmland composed of regular polygons, Image 3 is terraced fields, and Image 4 is flat farmland.

[0081] By treating each farmland label as an example and performing instance-level recognition rather than pixel-level recognition, the integrity of the farmland can be more effectively identified. The Mask2Former model structure is as follows: Figure 5 As shown, it includes the Backbone layer, the pixel-level decoding layer (which can also be understood as a pixel decoder), and the Transformer decoding layer.

[0082] In other words, after the target image is input into the main network (i.e., the Mask2Former model structure) of the pre-trained target segmentation model, feature extraction is first performed through the Backbone layer in the main network, then pixel-level feature maps are obtained through the pixel-level decoding layer, and finally instance segmentation is achieved through the Transformer decoding layer (which can also be understood as the target transformer).

[0083] By inputting the high-resolution feature map from the pixel-level decoding layer as... Figure 3 The semantic segmentation decoder in the semantic segmentation network shown obtains the current example segmentation result that matches the current pixel instance.

[0084] As an optional example, the above method, by inputting the feature map into the semantic segmentation decoder, obtains the current semantic segmentation result that matches the current pixel instance, including:

[0085] By inputting the feature map into the semantic segmentation decoder, the secondary category to which each pixel belongs in the first set of pixels corresponding to the current pixel instance is obtained;

[0086] Based on the secondary category to which each pixel belongs, a first array is generated to represent the current semantic segmentation result, where each element in the first array represents the secondary category to which each pixel belongs.

[0087] Assuming the target image contains two farmlands, each farmland is treated as a farmland instance. The high-resolution feature map output from the pixel-level decoder in the Mask2Former model described above is then input into the target image. Figure 3 The semantic segmentation decoder shown obtains a farmland subdivision semantic mask corresponding to each farmland instance.

[0088] like Figure 6As shown, assuming there are two farmlands, namely Farmland 1 and Farmland 2, in the process of predicting each farmland using the Mask2Former model structure, the value of each pixel in each farmland is used to indicate whether the pixel belongs to the farmland area. For example, a pixel with a value of 1 indicates that the pixel belongs to the farmland area; otherwise, the pixel does not belong to the farmland area.

[0089] Following the above method, pixels with a value of 1 are highlighted, while pixels with a value of 0 are not displayed. This process is used to obtain the farmland masks for farmland 1 and farmland 2. For details, please refer to [reference needed]. Figure 6 .

[0090] Each displayed circle represents a pixel within a farmland area, and the value of that pixel indicates its category as farmland.

[0091] Similarly, the high-resolution feature map output from the pixel-level decoder in the Mask2Former model is input as... Figure 3 Following the semantic segmentation decoder shown, the subdivided semantics of each pixel in each farmland can be further identified.

[0092] In other words, the semantic segmentation decoder can convert the image features extracted by the encoder into pixel-level classification results, that is, generate a category label for each pixel. When processing the various second-level categories in the first-level category, the semantic segmentation decoder refines the categories in the following way:

[0093] S11 performs feature map fusion on the multi-scale feature maps output by the pixel decoder (which can also be understood as the pixel-level decoding layer);

[0094] These multi-size or high-resolution feature maps contain information from different levels of the image.

[0095] S12, through the upsampling process, increases the resolution of the feature map to the same resolution as the original input image;

[0096] Since the feature map output by the encoder is usually low resolution, the encoder needs to improve the resolution of the feature map through an upsampling process.

[0097] S13 utilizes an attention mechanism to make the model focus on the region in the image that is most relevant to the current segmentation task;

[0098] During the decoding process, attention mechanisms can help the model focus on the regions in the image that are most relevant to the current task, thereby improving the accuracy of segmentation.

[0099] S14, through the semantic segmentation decoder, generates one or more mask predictions;

[0100] Each mask prediction corresponds to a specific category in the image, and these mask predictions are typically generated through a series of convolutional layers and activation functions.

[0101] S15, further post-processing is performed on the mask prediction;

[0102] For example, nonmaximum suppression or thresholding can be used to remove redundant regions and optimize the segmentation results.

[0103] S16, learns specific feature representations for different subcategories;

[0104] This allows for the differentiation of different subcategories even within a primary category (major category, such as farmland).

[0105] S17, Processing feature maps at different scales;

[0106] Its purpose is to capture objects of different sizes and details, which helps to distinguish the various subcategories within a larger category.

[0107] S18 utilizes a cross-attention mechanism to refine the categories.

[0108] For example, in the Mask2Former model architecture, the decoder uses a cross-attention mechanism to refine the categories, which allows the model to consider global contextual information when generating masks.

[0109] After the processing in steps S11 to S18 above, the following is obtained: Figure 7 The image shown is a binary image corresponding to a farmland subdivision semantic mask. Each farmland subdivision semantic mask corresponds to a two-dimensional array. Each element in the two-dimensional array identifies the subdivision semantic category to which a pixel in a farmland region belongs, that is, one of the subcategories in the primary category.

[0110] from Figure 7 It can be seen intuitively that the area where farmland 1 is located contains multiple squares and a small number of stars. Each square is used to indicate that the value of the pixel indicates that it belongs to one of the subcategories of farmland. For example, a square indicates that the pixel belongs to regular block farmland; each star indicates that the pixel belongs to irregular block farmland.

[0111] Using the above method, the semantic segmentation decoder can effectively process and distinguish multiple subcategories within the same major category, achieving high-precision pixel-level classification.

[0112] As an optional example, the above method, by inputting the feature map into the target transformer in the main network, obtains a current instance segmentation result that matches the current pixel instance, including:

[0113] By inputting the feature map into the target transformer, the target query vector is multiplied with the feature map to obtain the probability distribution vector. The probability distribution vector is used to describe the probability that each pixel in the first set of pixels corresponding to the current pixel instance belongs to the first-level category. The target query vector is used to describe the specific information that needs to be focused on from the feature map.

[0114] The segmentation result of the current instance is determined based on the probability distribution vector.

[0115] Similar to the processing in the existing Mask2Former model, in the target segmentation model of this application embodiment, the main network, for each query (query vector), after passing through the Transformer decoding layer, multiplies it with the last feature map in the pixel-level decoding layer to obtain the corresponding class probability distribution and mask semantic map, thereby achieving graph element instance segmentation.

[0116] In deep learning models, query vectors interact with feature maps to generate class probability distributions. Query vectors are typically used in attention mechanisms or transformer architectures, while the last feature map in the pixel-level decoding layer refers to the deepest feature map generated during the decoding process in a convolutional neural network.

[0117] In the attention mechanism, query vectors are used to retrieve or focus on specific information from the feature map. Each query vector is multiplied by the feature map generated during the decoding process, which may involve dot product operations to calculate the similarity or matching degree between the query vector and each position in the feature map.

[0118] The multiplication operation described above yields a vector representing the probability distribution of each category at each location. This vector is then used to calculate the final category probability distribution, typically using the softmax function or other normalization functions.

[0119] In the process of image classification, assigning a category label to each pixel in the image, such as person, car, building, etc., can generate the category probability distribution of each pixel, thereby achieving accurate pixel-level classification.

[0120] As described in the above embodiments, by modifying the existing deep learning model Mask2Former, a semantic segmentation branch is added to predict the subdivided semantics of farmland instances. This not only ensures that the model can predict a good set of farmland instances, but also predicts more accurate subdivided semantics, achieving high quality in both geometry and semantics.

[0121] As an optional example, based on the instance segmentation results, at least one pixel instance is vectorized to obtain the vectorized result, including:

[0122] Each graph element instance is sequentially obtained from at least one graph element instance as the current graph element instance;

[0123] Based on the edge detection algorithm, the geometric contour of the current graphic element instance is determined;

[0124] Based on the polygon approximation algorithm, a set of boundary pixels is extracted from the set of second pixels at various locations contained in the geometric contour.

[0125] Convert the pixel coordinates of a set of boundary pixels into a set of contour coordinates;

[0126] Based on a set of contour coordinates, determine the vectorization result of the current graphic element instance.

[0127] Based on each pixel in each pixel instance, an edge detection algorithm is performed to determine the geometric contour of the current pixel instance, for example, ... Figure 9 As shown, an elliptical geometric contour is obtained using an edge detection algorithm, thus determining that the vector graphic of a farmland area is elliptical.

[0128] The basic steps of the edge detection algorithm are as follows:

[0129] S21, Image preprocessing;

[0130] This mainly includes noise reduction and contrast enhancement to improve the accuracy of edge detection.

[0131] S22, Calculate the gradient of the image;

[0132] This refers to the rate of change of image brightness. Commonly used gradient operators include the Sobel operator, Prewitt operator, and Canny operator.

[0133] S23, in the gradient direction, suppress the gradient value to ensure that the edge is a single pixel wide.

[0134] S24, uses a dual threshold detection method to determine strong edges and weak edges;

[0135] Two thresholds (a high threshold and a low threshold) are used to determine strong and weak edges.

[0136] S25, connect the detected edge segments to obtain continuous edges.

[0137] It should be noted that the Canny edge detection algorithm is a very popular edge detection method. It generates edge strength and inverse information through steps such as Gaussian filtering, gradient calculation, nonmaximum suppression, and double threshold detection. This information can be used to determine the precise boundaries of objects (e.g., farmland) in an image.

[0138] In this embodiment, the edge detection algorithm described above is used to identify the edges of farmland in the image and extract the geometric contours of the farmland. In order to reduce the number of points on consecutive edges, a polygon approximation algorithm (e.g., selecting the four vertices of a square) can be used to simplify the geometric contours of the farmland. This can maintain the basic shape of the geometric contours and reduce computation.

[0139] Additionally, it should be noted that, based on the aforementioned geometric contours, the points generated to represent the boundaries of farmland are typically not pixels, but rather more abstract mathematical points. These points define the outline of the farmland, but they are not individual pixels in the image; rather, they are coordinate points calculated by the algorithm.

[0140] As an optional implementation, the above method determines the vectorization result of the current graphic element instance based on a set of contour coordinates, including:

[0141] By sequentially connecting the coordinate points corresponding to a set of contour coordinates, a vector graphic that satisfies the preset geometric shape is obtained;

[0142] Vector graphics that meet the preset geometric shape are identified as vectorized results.

[0143] The boundary information of the farmland instance generated in the above manner is used to determine the N polygons corresponding to the farmland instance, where N is the number of farmland instances.

[0144] The above-mentioned contour extraction method, using an algorithm based on connected component analysis, ensures the continuity and accuracy of farmland instance boundary information. Connected component analysis can effectively handle the problem of discontinuous or ambiguous farmland boundaries, ensuring the accuracy of farmland boundary information and improving data processing efficiency. It is especially suitable for farmland monitoring and agricultural resource assessment projects.

[0145] As an optional example, the above rendering of at least one pixel instance based on the vectorization result, semantic segmentation result, and pixel coordinates of at least one pixel instance includes:

[0146] Each graph element instance is sequentially obtained from at least one graph element instance as the current graph element instance, and the current semantic segmentation result corresponding to the current graph element instance is determined from the semantic segmentation result.

[0147] Determine the current instance segmentation result corresponding to the current graph element instance from the instance segmentation results;

[0148] Generate the current vectorized result based on the current instance segmentation result;

[0149] The current pixel instance is rendered based on the current semantic segmentation result, the current vectorization result, and the pixel coordinates of each pixel in the first pixel set corresponding to the current pixel instance.

[0150] To facilitate understanding, the rendering process of the above-mentioned pixel instance will be explained using one pixel instance from at least one pixel instance as an example.

[0151] For example, such as Figure 9 As shown, assuming that the geometric contour of the extracted farmland instance is an approximately square region using the edge detection algorithm in the above embodiment, the region is vectorized to obtain a polygon corresponding to each farmland instance.

[0152] The vectorization process is as follows:

[0153] (1) Scanning: Converting paper maps or other graphic materials into digital raster images using a scanner;

[0154] (2) Image preprocessing: The scanned raster image is processed, including noise reduction, geometric correction, projection transformation, etc., to improve image quality;

[0155] (3) Georeferencing: Geographically referencing the image to ensure that the image matches the actual geographical location. This may involve the selection of control points and coordinate transformation.

[0156] (4) Vectorization: Using vectorization software tools, such as MapInfo and ArcGIS, raster images are manually or automatically converted into vector data. This process may involve tracing image outlines and creating vector features such as points, lines, and polygons;

[0157] (5) Editing and cleaning: Edit the vectorized data, remove redundant nodes, smooth lines, correct errors, and add or delete necessary elements;

[0158] (6) Attribute data input: Add attribute information to vector features, such as feature name, type, area, etc. This information is usually managed through a database or a dedicated attribute table;

[0159] (7) Quality control: Perform quality checks on the vectorization results to ensure the accuracy and completeness of the data;

[0160] (8) Saving and outputting: Saving vectorized data in a specific vector format for use in GIS (Geographic Information System) software or for further analysis and processing.

[0161] The purpose of vectorization is to convert raster data into vector data to facilitate more efficient spatial analysis, data management, and visualization. This process is extremely useful in Geographic Information Systems (GIS), computer-aided design, and many other fields.

[0162] As an optional implementation, the above-mentioned rendering of the current pixel instance based on the current semantic segmentation result, the current vectorization result, and the pixel coordinates of each pixel in the first pixel set corresponding to the current pixel instance includes:

[0163] Based on the current semantic segmentation results, determine the target secondary category among the N secondary categories to which the current graph element instance belongs, where N is a positive integer greater than or equal to 2;

[0164] Based on the target secondary category, the current graph element instance is converted into a target graph element instance carrying the target secondary category;

[0165] Render the current pixel instance based on the target pixel instance and the pixel coordinates of each pixel in the first pixel set.

[0166] After vectorizing the instance segmentation results corresponding to each pixel instance, a polygon corresponding to each pixel instance is obtained. Then, based on the farmland subdivision semantic mask, the subdivision semantic mask of the mask position of each farmland instance is output, and the subdivision semantic category of each farmland instance is output according to the number of pixels.

[0167] For example, such as Figure 9 As shown, after vectorization, assuming the following is used Figure 3 The binary image corresponding to the farmland subdivision semantic mask output by the semantic segmentation model shown is as follows: Figure 9 The rectangle shown represents the value of each pixel in the binary image corresponding to the farmland subdivision semantic mask, which indicates whether the pixel belongs to one of the subcategories of farmland.

[0168] Combination Figure 7 As can be seen, due to performance issues of the model, the elements in the farmland subdivision semantic mask output by the semantic segmentation network may indicate multiple subcategories to which each pixel belongs, for example, Figure 7 The binary image corresponding to the subdivided semantic mask of farmland 1 shown contains a first subcategory indicated by squares and a second subcategory indicated by stars.

[0169] The subdivision semantic category of each farmland instance is determined based on the number of pixels belonging to different subcategories in the farmland subdivision semantic mask.

[0170] It should be noted that, assuming the target image has dimensions of h×w, that is, length h and width w, where h and w are positive integers greater than or equal to 1, then the dimensions of the two-dimensional data corresponding to the farmland subdivision semantic mask are m×h×w, where m indicates that there are a total of 4 subclasses.

[0171] After determining the subcategories of farmland instances in the manner described above, it is possible, but not limited to, through... Figure 9 The overlay process shown generates farmland instances carrying subdivided semantic categories, and renders them based on these subdivided semantic categories.

[0172] As an optional implementation, the above method determines the target secondary category among the N secondary categories to which the current graph element instance belongs, based on the current semantic segmentation result, including:

[0173] Based on the first array corresponding to the current semantic segmentation result, determine the secondary category to which each pixel in the first set of pixels corresponding to the current pixel instance belongs. Each element in the first array represents the secondary category to which each pixel belongs.

[0174] Based on the secondary category to which each pixel in the first pixel set belongs, the first pixel set is divided into N groups of pixels. Among them, one group of pixels in the N groups includes pixels belonging to one of the N secondary categories.

[0175] From N groups of pixels, identify the group with the largest number of pixels, and determine the secondary category to which each pixel in the group with the largest number of pixels belongs as the target secondary category to which the current image element instance belongs.

[0176] Suppose that the value of each element in the first array corresponding to the current semantic segmentation result is used to represent that a pixel belongs to one of the subcategories of farmland, such as Figure 8 As shown, a square indicates that the pixel belongs to regular block farmland, and a star indicates that the pixel belongs to irregular block farmland. The number of squares and stars are counted. Then, by comparing the number of squares and stars, the regular block farmland corresponding to the larger number of squares is determined as the subdivision semantic category of the current farmland instance.

[0177] In other words, the value of each element in the two-dimensional array corresponding to the instance segmentation result (farmland mask) is used to represent the ID of the instance of that pixel. For example, 0 represents non-farmland and 1 represents farmland. The value of each element in the two-dimensional array corresponding to the semantic segmentation result (farmland subdivision semantic mask) is used to represent the semantic category ID to which the pixel belongs. For example, 1 represents regular block farmland, 2 represents irregular block farmland, 3 represents terraced fields, and 4 represents flat farmland, etc.

[0178] Therefore, by counting the number of each semantic category ID based on the number of pixels and conducting a vote, the final subdivision semantic category of the farmland instance can be determined.

[0179] The voting mechanism described above, based on the neighborhood information of farmland instances, can improve the accuracy and reliability of the subdivided semantic categories of farmland instances.

[0180] As an optional example, the above rendering of the current pixel instance based on the target pixel instance and the pixel coordinates of each pixel in the first pixel set includes:

[0181] Obtain the target's latitude and longitude at the time the target image was captured;

[0182] Based on the target latitude and longitude, convert the pixel coordinates of each pixel in the first pixel set into a set of latitude and longitude coordinates;

[0183] Render the current pixel instance based on a set of latitude and longitude coordinates and the target style indicated by the target secondary category carried by the target pixel instance.

[0184] like Figure 9 As shown, by overlaying the current instance segmentation result and the current semantic segmentation result, a farmland instance carrying a subdivided semantic category is obtained.

[0185] Assuming the farmland is a regular block of farmland, the latitude and longitude coordinates of the target location are obtained when the target image is captured. Then, based on the target latitude and longitude coordinates, the latitude and longitude coordinates of the four corner points of the farmland instance are determined. Using the latitude and longitude coordinates of the four corner points as geographic reference information, the pixel coordinates of the other pixels in the first pixel set, excluding the four corner points, are transformed to obtain the latitude and longitude coordinates of the other pixels.

[0186] Based on the latitude and longitude coordinates of the farmland instances, render each farmland instance in the target image onto the target map.

[0187] It should be noted that the latitude and longitude coordinates of farmland instances can be, but are not limited to, determined as map farmland data. Typically, the map farmland data of one farmland instance corresponds to one JSON file.

[0188] In summary, from data annotation and model recognition to data post-processing, the entire process of producing map farmland data has been automated, such as... Figure 10 As shown in (a), this is the original image after annotation, as follows: Figure 10 As shown in (b), this is the farmland prediction result that includes subdivided semantic categories.

[0189] To more clearly understand the implementation process of the above-mentioned automated production map farmland data, the following will combine... Figure 11The overall flowchart shown below provides further explanation.

[0190] S1102, acquire image;

[0191] Assume the original image is a 1024×1024 optical satellite image with a resolution of 0.5 meters.

[0192] S1104, perform data annotation on the image;

[0193] The labelers need to label the farmland and its subcategories, including irregularly shaped farmland, regularly shaped farmland, terraced fields, and flat farmland. See the attached reference for details. Figure 4 .

[0194] S1106, features are extracted from the image by inputting the labeled image into the backbone of the main network in the target segmentation model;

[0195] The main network in the target segmentation model can be, but is not limited to, the existing Mask2Former model. For details, please refer to the description in the above embodiments, which will not be repeated here.

[0196] S1108: Input the extracted features into the pixel decoder in the main network to obtain a high-resolution feature map;

[0197] S1110: By inputting the high-resolution feature map into the semantic segmentation decoder, a farmland subdivision semantic mask of the farmland instance is obtained;

[0198] The farmland subdivision semantic mask output by the semantic segmentation decoder is a two-dimensional array. The value of each element in the two-dimensional array is used to represent the ID of the semantic category to which the pixel belongs. For example, 1 represents regular block farmland, 2 represents irregular block farmland, 3 represents terraced fields, and 4 represents flat farmland.

[0199] S1112, by inputting the high-resolution feature map into the target transformer in the main network, the farmland mask for each farmland instance is obtained;

[0200] Each instance's farmland mask is a two-dimensional array, where the value of each element represents the ID of the instance for that pixel; for example, 0 represents non-farmland and 1 represents farmland.

[0201] S1114, The farmland instance's farmland subdivision semantic mask and farmland mask are superimposed to obtain a farmland instance with subdivision semantic category;

[0202] For details, please refer to... Figure 9 The implementation process shown will not be elaborated here.

[0203] S1116, obtain the latitude and longitude of the image and the pixel coordinates of the farmland instance;

[0204] S1118, Perform coordinate transformation on each pixel in the farmland instance with subdivided semantic categories to obtain the corresponding latitude and longitude coordinates;

[0205] Specifically, based on the latitude and longitude of the location when the image was captured, the pixel coordinates of the four corner points of the farmland instance are first converted into latitude and longitude coordinates. Then, using the latitude and longitude coordinates of the four corner points as reference information, the pixel coordinates of the remaining pixels are converted into latitude and longitude coordinates.

[0206] S1120 generates corresponding map farmland data based on the latitude and longitude coordinates of the farmland instance.

[0207] The farmland data on the map may, but is not limited to, exist in the form of a JSON file.

[0208] After obtaining the farmland mask and farmland subdivision semantic mask of the farmland instance, farmland instances with subdivision semantic categories can be generated through, but not limited to, the following steps.

[0209] S1122, Determine the geometric outline of the farmland instance based on the farmland mask;

[0210] Specifically, edge detection algorithms can be used to determine the geometric outline of farmland instances, as described in the above embodiments, and will not be repeated here.

[0211] S1124, the geometric contour of the farmland instance is vectorized to obtain the vectorized result;

[0212] The vectorization result can be, but is not limited to, represented as N polygons, where N is the number of farmland instances.

[0213] S1126, Vote on the subdivided semantic mask at the location of each farmland instance to determine the subdivided semantic category of each farmland instance;

[0214] S1128, The farmland subdivision semantic mask and the farmland mask of the farmland instance are superimposed to obtain the farmland instance with subdivision semantic category;

[0215] S1130, Generate farmland instances with detailed semantic categories.

[0216] In this embodiment, an innovative multi-level, multi-branch architecture is obtained by modifying the Mmask2Former model structure, significantly improving the accuracy and efficiency of automated farmland data production. Specifically, the dual-branch structure of semantic segmentation and instance segmentation can simultaneously capture the global features and local details of farmland. The feature fusion module ensures the accuracy of farmland and its sub-categories identification, while contour extraction and attribute voting in the data post-processing steps further improve the reliability of farmland instance boundaries and category information.

[0217] Furthermore, the flexibility of the coordinate transformation algorithm enables the generated farmland vector data file (JSON file) to adapt to various map data formats, enhancing the practicality of the solution. Overall, the technical solution of this application effectively addresses the complexity and diversity challenges in the automated production of farmland data, providing strong technical support for fields such as precision agriculture, land planning, and environmental monitoring.

[0218] As an optional example, the above method also includes:

[0219] When at least one pixel instance is a farmland instance, by inputting the target image into the target segmentation model, a farmland instance segmentation result and a farmland semantic segmentation result matching the farmland instance are obtained. The farmland instance segmentation result is used to indicate whether the category of the farmland instance is farmland, and the farmland semantic segmentation result is used to indicate the sub-category to which the farmland instance belongs. Farmland includes N sub-categories, and each sub-category in the N sub-categories represents a different shape of farmland. N is a positive integer greater than or equal to 2.

[0220] Based on the farmland instance segmentation results, at least one farmland instance is vectorized to obtain the vectorized result;

[0221] The farmland instance is rendered based on the vectorization results, farmland semantic segmentation results, and the pixel coordinates of the farmland instance.

[0222] In the case where at least one pixel instance is a farmland instance, by inputting the target image Figure 3 The target segmentation model shown extracts features from the backbone, and then inputs the extracted image features into the pixel decoder to obtain a high-resolution feature map.

[0223] By inputting high-resolution feature maps into a target converter, farmland instance segmentation results are obtained; by inputting high-resolution feature maps into a semantic segmentation decoder, farmland semantic segmentation results are obtained. In other words, based on satellite imagery, a multi-level, multi-branch deep learning model for instance segmentation is designed. Different branches identify farmland and its subcategories, ensuring the precision and accuracy of farmland geometry and subcategories.

[0224] By vectorizing the farmland instance segmentation results and farmland semantic segmentation results output by the aforementioned target segmentation model, vectorized farmland map data is output with an accuracy of up to 0.5m. This ultimately achieves automated production of farmland map data, saving manpower, reducing costs and increasing efficiency, while also supporting farmland map rendering and enhancing the user's immersive experience.

[0225] The technical solution adopted in this application embodiment is based on satellite imagery and a multi-level, multi-branch deep learning model is designed to achieve end-to-end identification of farmland geometry and subdivision categories, ensuring the accuracy of farmland geometry and category.

[0226] Furthermore, using automatically generated farmland map data for rendering can effectively recreate the real world, further enhancing the user's immersive experience. At the same time, while ensuring data quality, it significantly saves manpower and shortens the production cycle.

[0227] like Figure 12 As shown in (a), this is the effect of rendering farmland map data (multiple irregular squares) on a mobile map using the technical solution in the embodiments of this application. Figure 12 As shown in (b), this is the effect of rendering the farmland map data produced using the technical solution in the embodiment of this application in the vehicle map.

[0228] Obviously, the automated production schemes for map farmland data mentioned in the above embodiments are merely examples of rendering methods for map element instances and are not intended to limit them. For example, they are also applicable to the production of map data for other map elements, such as the automated production of map data for buildings containing multiple subcategories like residential, commercial, industrial, and agricultural buildings, and the automated production of map data for roads containing multiple subcategories like national highways, provincial highways, county roads, and township roads.

[0229] Specifically, after acquiring the target image, the image is labeled to obtain a tagged image, and the tagged image is input into a database such as... Figure 3 The backbone of the main network shown is used to extract features, and then the extracted image features are input into the pixel decoder to obtain a high-resolution feature map.

[0230] By inputting high-resolution feature maps into the semantic segmentation decoder and the target converter respectively, semantic segmentation results and building instance segmentation results are output respectively. Based on the building instance segmentation results, vectorization processing is performed to obtain N geometric contours. Based on the semantic segmentation results, the subdivided semantic categories of each building instance are obtained, and then the building instances with subdivided semantic categories are obtained through overlay processing. Finally, the building instances with subdivided semantic categories are rendered according to the latitude and longitude of the image.

[0231] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0232] According to another aspect of the embodiments of this application, as follows is also provided Figure 13 A rendering apparatus for a pixel instance is shown, the apparatus comprising:

[0233] The first acquisition unit 1302 is used to acquire a target image captured in advance, wherein the target image includes at least one graphic element instance formed according to a preset shape, and one graphic element instance in the at least one graphic element instance represents a map graphic element.

[0234] The first processing unit 1304 is used to input the target image into the target segmentation model to obtain an instance segmentation result and a semantic segmentation result that match at least one pixel instance. The instance segmentation result is used to indicate whether the category of each pixel instance in the at least one pixel instance is a first-level category, and the semantic segmentation result is used to indicate the second-level category to which each pixel instance belongs. The second-level category is a subcategory of the first-level category. The target segmentation model includes a main network and a semantic segmentation network. The output of the pixel decoder in the main network is the input of the semantic segmentation decoder in the semantic segmentation network.

[0235] The second processing unit 1306 is used to perform vectorization processing on at least one pixel instance based on the instance segmentation result to obtain a vectorized result;

[0236] Rendering unit 1308 is used to render at least one pixel instance based on vectorization results, semantic segmentation results, and pixel coordinates of at least one pixel instance.

[0237] Optionally, the first processing unit 1304 includes:

[0238] The first acquisition module is used to sequentially acquire each graphic element instance as the current graphic element instance from at least one graphic element instance;

[0239] The first processing module is used to obtain a feature map with the same resolution as the target image by inputting the target image into the pixel decoder.

[0240] The second processing module is used to obtain the current instance segmentation result that matches the current pixel instance by inputting the feature map into the target converter in the main network;

[0241] The third processing module is used to obtain the current semantic segmentation result that matches the current pixel instance by inputting the feature map into the semantic segmentation decoder.

[0242] Optionally, the third processing module mentioned above includes:

[0243] The first processing submodule is used to obtain the secondary category of each pixel in the first set of pixels corresponding to the current pixel instance by inputting the feature map into the semantic segmentation decoder.

[0244] The second processing submodule is used to generate a first array representing the current semantic segmentation result based on the secondary category to which each pixel belongs, wherein each element in the first array represents the secondary category to which each pixel belongs.

[0245] Optionally, the second processing module mentioned above includes:

[0246] The third processing submodule is used to input the feature map into the target converter, perform a multiplication operation between the target query vector and the feature map to obtain a probability distribution vector. The probability distribution vector is used to describe the probability that each pixel in the first set of pixels corresponding to the current pixel instance belongs to the first-level category. The target query vector is used to describe the specific information that needs to be focused on from the feature map.

[0247] The fourth processing submodule is used to determine the segmentation result of the current instance based on the probability distribution vector.

[0248] Optionally, the second processing unit 1306 includes:

[0249] The second acquisition unit is used to sequentially acquire each graphic element instance as the current graphic element instance from at least one graphic element instance;

[0250] The fourth processing module is used to determine the geometric contour of the current graphic element instance based on the edge detection algorithm;

[0251] The extraction module is used to extract a set of boundary pixels from the set of second pixels at various locations contained in the geometric contour, based on a polygon approximation algorithm.

[0252] The conversion module is used to convert the pixel coordinates of a set of boundary pixels into a set of contour coordinates.

[0253] The fifth processing module is used to determine the vectorization result of the current graphic element instance based on a set of contour coordinates.

[0254] Optionally, the fifth processing module mentioned above includes:

[0255] The fifth processing submodule is used to connect the coordinate points corresponding to a set of contour coordinates in sequence to obtain a vector graphic that satisfies the preset geometric shape.

[0256] The sixth processing submodule is used to determine the vector graphics that meet the preset geometric shape as the vectorized result.

[0257] Optionally, the rendering unit 1308 includes:

[0258] The third acquisition module is used to sequentially acquire each graph element instance from at least one graph element instance as the current graph element instance, and determine the current semantic segmentation result corresponding to the current graph element instance from the semantic segmentation result;

[0259] The sixth processing module is used to determine the current instance segmentation result corresponding to the current pixel instance from the instance segmentation results;

[0260] The seventh processing module is used to generate the current vectorized result based on the current instance segmentation result;

[0261] The rendering module is used to render the current pixel instance based on the current semantic segmentation result, the current vectorization result, and the pixel coordinates of each pixel in the first pixel set corresponding to the current pixel instance.

[0262] Optionally, the above rendering module includes:

[0263] The seventh processing submodule is used to determine the target secondary category among the N secondary categories to which the current graph element instance belongs, based on the current semantic segmentation result, where N is a positive integer greater than or equal to 2;

[0264] The first transformation submodule is used to transform the current grapheme instance into a target grapheme instance carrying the target secondary category based on the target secondary category;

[0265] The first rendering submodule is used to render the current pixel instance based on the target pixel instance and the pixel coordinates of each pixel in the first pixel set.

[0266] Optionally, the above rendering module includes:

[0267] The eighth processing submodule is used to determine the secondary category of each pixel in the first set of pixels corresponding to the current image element instance based on the first array corresponding to the current semantic segmentation result. Each element in the first array represents the secondary category to which each pixel belongs.

[0268] The ninth processing submodule is used to divide the first pixel set into N groups of pixels based on the secondary category to which each pixel belongs in the first pixel set. Each group of pixels in the N groups includes pixels belonging to one of the N secondary categories.

[0269] The tenth processing submodule is used to determine the group of pixels with the largest number of pixels from the N groups of pixels, and to determine the secondary category to which each pixel in the group with the largest number of pixels belongs as the target secondary category to which the current image element instance belongs.

[0270] Optionally, the above rendering module includes:

[0271] The acquisition submodule is used to acquire the latitude and longitude of the target location when the target image is captured.

[0272] The second conversion submodule is used to convert the pixel coordinates of each pixel in the first pixel set into a set of latitude and longitude coordinates based on the target latitude and longitude.

[0273] The second rendering submodule is used to render the current pixel instance based on a set of latitude and longitude coordinates and the target style indicated by the target secondary category carried by the target pixel instance.

[0274] Optionally, the above-mentioned device further includes:

[0275] The third processing unit is used to obtain a farmland instance segmentation result and a farmland semantic segmentation result that match the farmland instance by inputting the target image into the target segmentation model when at least one pixel instance is a farmland instance. The farmland instance segmentation result is used to indicate whether the category of the farmland instance is farmland, and the farmland semantic segmentation result is used to indicate the sub-category to which the farmland instance belongs. Farmland includes N sub-categories, and each sub-category in the N sub-categories represents a different shape of farmland. N is a positive integer greater than or equal to 2.

[0276] The fourth processing unit is used to perform vectorization processing on at least one farmland instance based on the farmland instance segmentation result to obtain the vectorization result;

[0277] The fifth processing unit is used to render farmland instances based on vectorization results, farmland semantic segmentation results, and the pixel coordinates of farmland instances.

[0278] By applying the aforementioned apparatus to a dual-branch structure—comprising a main network and a semantic segmentation network—in a target segmentation model, the main network outputs an instance segmentation result representing the class probability distribution of each pixel in at least one pixel instance, while the semantic segmentation network outputs a semantic segmentation result representing the subdivided semantic categories of each pixel in at least one pixel instance. Based on the instance segmentation and semantic segmentation results, the pixel information of the pixel instance is vector-processed, and rendered according to the pixel coordinates and vector processing results of the pixel instance. In other words, by employing the aforementioned dual-branch model structure, the real-time global features and local details of at least one pixel in the target image can be captured simultaneously, improving the accuracy of the pixel instance category and its subcategories output by the model, and achieving the technical effect of improving the accuracy of the pixel instance rendering process.

[0279] It should be noted that the embodiments of the rendering device for the pixel instance here can refer to the embodiments of the rendering method for the pixel instance described above, and will not be repeated here.

[0280] According to another aspect of the embodiments of this application, an electronic device for implementing the rendering method of the above-described pixel instance is also provided, the electronic device being... Figure 1 The target terminal or server is shown. This embodiment uses the electronic device as an example to illustrate the concept. Figure 14 As shown, the electronic device includes a memory 1402 and a processor 1404. The memory 1402 stores a computer program, and the processor 1404 is configured to execute the steps of any of the above method embodiments via the computer program.

[0281] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0282] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0283] S1, acquire a pre-captured target image, wherein the target image includes at least one pixel instance formed according to a preset shape, and one pixel instance in the at least one pixel instance represents a map pixel;

[0284] S2, by inputting the target image into the target segmentation model, an instance segmentation result and a semantic segmentation result matching at least one pixel instance are obtained. The instance segmentation result is used to indicate whether the category of each pixel instance in at least one pixel instance is a first-level category, and the semantic segmentation result is used to indicate the second-level category to which each pixel instance belongs. The second-level category is a subcategory of the first-level category. The target segmentation model includes a main network and a semantic segmentation network. The output of the pixel decoder in the main network is the input of the semantic segmentation decoder in the semantic segmentation network.

[0285] S3, based on the instance segmentation result, vectorize at least one grapheme instance to obtain the vectorized result;

[0286] S4 renders at least one pixel instance based on the vectorization result, the semantic segmentation result, and the pixel coordinates of at least one pixel instance.

[0287] Alternatively, as those skilled in the art will understand, Figure 14 The structure shown is for illustrative purposes only. Figure 14 This does not limit the structure of the aforementioned electronic devices or electronic equipment. For example, electronic devices or electronic equipment may also include components that are more... Figure 14 The more or fewer components shown (such as network interfaces, etc.), or having the same Figure 14 The different configurations shown.

[0288] The memory 1402 can be used to store software programs and modules, such as the program instructions / modules corresponding to the rendering method and apparatus for pixel instances in this embodiment. The processor 1404 executes various functional applications and data processing by running the software programs and modules stored in the memory 1402, thereby implementing the aforementioned rendering method for pixel instances. The memory 1402 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1402 may further include memory remotely located relative to the processor 1404, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 1402 may be used, but is not limited to, to store target images, semantic segmentation results, and instance segmentation results. As an example, such as... Figure 14 As shown, the memory 1402 may include, but is not limited to, the first acquisition unit 1302, the first processing unit 1304, the second processing unit 1306, and the rendering unit 1308 in the rendering apparatus of the aforementioned pixel instance. Furthermore, it may include, but is not limited to, other module units in the rendering apparatus of the aforementioned pixel instance, which will not be elaborated upon in this example.

[0289] Optionally, the transmission device 1406 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 1406 includes a Network Interface Controller (NIC), which can be connected to other network devices and routers via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 1406 is a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0290] In addition, the aforementioned electronic device also includes: a display 1408 for displaying scene images of the aforementioned virtual three-dimensional scene and images of rendered virtual objects; and a connection bus 1410 for connecting various module components in the aforementioned electronic device.

[0291] In other embodiments, the target terminal or server can be a node in a distributed system, which can be a blockchain system. This blockchain system is formed by connecting multiple nodes through network communication. The nodes can form a point-to-point network, and any type of computing device, such as a server or target terminal, can become a node in the blockchain system by joining this point-to-point network.

[0292] According to another aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform a rendering method for a pixel instance provided in various optional implementations of the aforementioned server verification processing, wherein the computer program is configured to execute the steps in any of the above-described method embodiments at runtime.

[0293] Optionally, in this embodiment, the computer-readable storage medium described above may be configured to store a computer program for performing the following steps:

[0294] S1, acquire a pre-captured target image, wherein the target image includes at least one pixel instance formed according to a preset shape, and one pixel instance in the at least one pixel instance represents a map pixel;

[0295] S2, by inputting the target image into the target segmentation model, an instance segmentation result and a semantic segmentation result matching at least one pixel instance are obtained. The instance segmentation result is used to indicate whether the category of each pixel instance in at least one pixel instance is a first-level category, and the semantic segmentation result is used to indicate the second-level category to which each pixel instance belongs. The second-level category is a subcategory of the first-level category. The target segmentation model includes a main network and a semantic segmentation network. The output of the pixel decoder in the main network is the input of the semantic segmentation decoder in the semantic segmentation network.

[0296] S3, based on the instance segmentation result, vectorize at least one grapheme instance to obtain the vectorized result;

[0297] S4 renders at least one pixel instance based on the vectorization result, the semantic segmentation result, and the pixel coordinates of at least one pixel instance.

[0298] Optionally, in embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0299] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the target terminal. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0300] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0301] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods of the various embodiments of this application.

[0302] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0303] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or the indirect coupling or communication connection of units or modules may be electrical or other forms.

[0304] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0305] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0306] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make several improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for rendering a pixel instance, characterized in that, include: Acquire a pre-captured target image, wherein the target image includes at least one pixel instance formed according to a preset shape, and one pixel instance in the at least one pixel instance represents a map pixel; By inputting the target image into the target segmentation model, instance segmentation results and semantic segmentation results matching the at least one pixel instance are obtained. The instance segmentation results are used to indicate whether the category of each pixel instance in the at least one pixel instance is a primary category, and the semantic segmentation results are used to indicate the secondary category to which each pixel instance belongs. The secondary category is a subcategory of the primary category. The target segmentation model includes a main network and a semantic segmentation network. The output of the pixel decoder in the main network is the input of the semantic segmentation decoder in the semantic segmentation network. Based on the instance segmentation result, the at least one graphic element instance is vectorized to obtain a vectorized result; Based on the vectorization result, the semantic segmentation result, and the pixel coordinates of the at least one pixel instance, the at least one pixel instance is rendered.

2. The method according to claim 1, characterized in that, The step of inputting the target image into a target segmentation model to obtain instance segmentation results and semantic segmentation results that match the at least one pixel instance includes: Each graphic element instance is sequentially obtained as the current graphic element instance from the at least one graphic element instance; By inputting the target image into the pixel decoder, a feature map with the same resolution as the target image is obtained; By inputting the feature map into the target converter in the main network, a current instance segmentation result matching the current pixel instance is obtained; By inputting the feature map into the semantic segmentation decoder, a current semantic segmentation result matching the current pixel instance is obtained.

3. The method according to claim 2, characterized in that, The step of inputting the feature map into the semantic segmentation decoder to obtain the current semantic segmentation result matching the current pixel instance includes: By inputting the feature map into the semantic segmentation decoder, the secondary category to which each pixel belongs in the first set of pixels corresponding to the current pixel instance is obtained; Based on the secondary category to which each pixel belongs, a first array is generated to represent the current semantic segmentation result, wherein each element in the first array represents the secondary category to which each pixel belongs.

4. The method according to claim 2, characterized in that, The step of inputting the feature map into the target converter in the main network to obtain a current instance segmentation result that matches the current pixel instance includes: By inputting the feature map into the target converter, a multiplication operation is performed on the target query vector and the feature map to obtain a probability distribution vector. The probability distribution vector is used to describe the probability that each pixel in the first set of pixels corresponding to the current image element instance belongs to the first-level category. The target query vector is used to describe the specific information that needs to be focused on from the feature map. Based on the probability distribution vector, the current instance segmentation result is determined.

5. The method according to claim 1, characterized in that, The step of vectorizing the at least one pixel instance based on the instance segmentation result to obtain a vectorized result includes: Each graphic element instance is sequentially obtained as the current graphic element instance from the at least one graphic element instance; The geometric contour of the current graphic element instance is determined based on the edge detection algorithm; Based on the polygon approximation algorithm, a set of boundary pixels is extracted from the set of second pixels at each position contained in the geometric contour. Convert the pixel coordinates of the set of boundary pixels into a set of contour coordinates; Based on the set of contour coordinates, the vectorization result of the current graphic element instance is determined.

6. The method according to claim 5, characterized in that, Determining the vectorization result of the current pixel instance based on the set of contour coordinates includes: By sequentially connecting the coordinate points corresponding to the set of contour coordinates, a vector graphic that satisfies the preset geometric shape is obtained; The vector graphics that satisfy the preset geometry are determined as the vectorization result.

7. The method according to claim 1, characterized in that, The rendering of the at least one pixel instance based on the vectorization result, the semantic segmentation result, and the pixel coordinates of the at least one pixel instance includes: Each graph element instance is sequentially obtained from the at least one graph element instance as the current graph element instance, and the current semantic segmentation result corresponding to the current graph element instance is determined from the semantic segmentation result; The current instance segmentation result corresponding to the current pixel instance is determined from the instance segmentation result; Based on the current instance segmentation result, generate the current vectorization result; The current pixel instance is rendered based on the current semantic segmentation result, the current vectorization result, and the pixel coordinates of each pixel in the first pixel set corresponding to the current pixel instance.

8. The method according to claim 7, characterized in that, The step of rendering the current pixel instance based on the current semantic segmentation result, the current vectorization result, and the pixel coordinates of each pixel in the first pixel set corresponding to the current pixel instance includes: Based on the current semantic segmentation result, determine the target secondary category among the N secondary categories to which the current graph element instance belongs, where N is a positive integer greater than or equal to 2; Based on the target secondary category, the current grapheme instance is converted into a target grapheme instance carrying the target secondary category; The current pixel instance is rendered based on the target pixel instance and the pixel coordinates of each pixel in the first pixel set.

9. The method according to claim 8, characterized in that, The step of determining the target secondary category among the N secondary categories to which the current graph element instance belongs based on the current semantic segmentation result includes: Based on the first array corresponding to the current semantic segmentation result, the secondary category to which each pixel in the first set of pixels corresponding to the current image instance belongs is determined, wherein each element in the first array represents the secondary category to which each pixel belongs; Based on the secondary category to which each pixel in the first pixel set belongs, the first pixel set is divided into N groups of pixels, wherein one group of pixels in the N groups of pixels includes pixels belonging to one of the N secondary categories; From the N groups of pixels, determine the group of pixels with the largest number of pixels, and determine the secondary category to which each pixel in the group with the largest number of pixels belongs as the target secondary category to which the current image element instance belongs.

10. The method according to claim 8, characterized in that, The step of rendering the current pixel instance based on the target pixel instance and the pixel coordinates of each pixel in the first pixel set includes: Obtain the target's latitude and longitude at the target location when the target image was captured; Based on the target latitude and longitude, the pixel coordinates of each pixel in the first pixel set are converted into a set of latitude and longitude coordinates; Based on the set of latitude and longitude coordinates and the target style indicated by the target secondary category carried by the target pixel instance, the current pixel instance is rendered.

11. The method according to any one of claims 1 to 10, characterized in that, The method further includes: When at least one pixel instance is a farmland instance, by inputting the target image into the target segmentation model, a farmland instance segmentation result and a farmland semantic segmentation result matching the farmland instance are obtained. The farmland instance segmentation result is used to indicate whether the category of the farmland instance is farmland, and the farmland semantic segmentation result is used to indicate the sub-category to which the farmland instance belongs. The farmland includes N sub-categories, and each of the N sub-categories represents a different shape of farmland, where N is a positive integer greater than or equal to 2. Based on the farmland instance segmentation results, the at least one farmland instance is vectorized to obtain the vectorized result; The farmland instance is rendered based on the vectorization result, the farmland semantic segmentation result, and the pixel coordinates of the farmland instance.

12. A rendering apparatus for a pixel instance, characterized in that, include: The first acquisition unit is used to acquire a pre-captured target image, wherein the target image includes at least one graphic element instance formed according to a preset shape, and one graphic element instance in the at least one graphic element instance represents a map graphic element; The first processing unit is configured to input the target image into a target segmentation model to obtain an instance segmentation result and a semantic segmentation result that match the at least one pixel instance. The instance segmentation result is used to indicate whether the category of each pixel instance in the at least one pixel instance is a primary category, and the semantic segmentation result is used to indicate the secondary category to which each pixel instance belongs. The secondary category is a subcategory of the primary category. The target segmentation model includes a main network and a semantic segmentation network. The output of the pixel decoder in the main network is the input of the semantic segmentation decoder in the semantic segmentation network. The second processing unit is used to perform vectorization processing on the at least one graphic element instance based on the instance segmentation result to obtain a vectorized result; A rendering unit is used to render the at least one pixel instance based on the vectorization result, the semantic segmentation result, and the pixel coordinates of the at least one pixel instance.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program can be executed by a terminal device or computer at runtime as described in any one of claims 1 to 11.

14. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1 to 11.

15. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method described in any one of claims 1 to 11 through the computer program.

Images