Color rendering method and device, electronic equipment and storage medium

Abstract

The invention provides a color rendering method and device, electronic equipment and a storage medium. The color rendering method and device can be applied to various scenes such as maps and smart traffic. The method comprises the following steps: acquiring a street view image presenting a target building; obtaining an outer wall pixel area where an outer wall of the target building is located in the street view image; based on the color of the outer wall pixel area, determining the color of the outer wall body of the target building; and according to the color of the outer wall body of the target building, carrying out color rendering on a building model of the target building in an electronic map. By adopting the method, the obtained color of the outer wall of the target building can be more accurate, the building model presented after the color rendering is performed on the building model of the target building in the electronic map by using the color is closer to the target building, and the scene in the real world can be truly restored; the problem that the color of the building model presented in the electronic map is single is avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the image processing technology field, and particularly relates to a color rendering method and device, an electronic device and a storage medium. BACKGROUND

[0002] In an electronic map, a building is usually presented in a corresponding building model. The color of the building model corresponding to the building in the electronic map is rendered according to a color preset for each type of building. For example, if the color corresponding to a shopping mall type building is preset as purple and the color corresponding to a school type building is preset as yellow, then the color of the building model corresponding to the shopping mall building in the electronic map is purple and the color of the building model corresponding to the school building in the electronic map is yellow. Since the color of the building model is rendered according to the color preset for each type of building, the color of the building model in the electronic map is relatively monotonous. SUMMARY

[0003] The present application provides a color rendering method and device, an electronic device and a storage medium, which can extract a real building color, and present a building model that is closer to a target building after rendering the color of the building model corresponding to the target building in an electronic map using the color, so that the electronic map can more realistically restore the scene in the real world.

[0004] The technical solution of the present application is implemented as follows:

[0005] The present application provides a color rendering method, which comprises: acquiring a street view image presenting a target building; acquiring an outer wall pixel region where an outer wall of the target building is located in the street view image; determining an outer wall body color of the target building based on the color of the outer wall pixel region; and rendering the color of a building model corresponding to the target building in an electronic map according to the outer wall body color of the target building.

[0006] The present application provides a color rendering device, which comprises: an image acquisition module configured to acquire a street view image presenting a target building; a pixel region acquisition module configured to acquire an outer wall pixel region where an outer wall of the target building is located in the street view image; a color determination module configured to determine an outer wall body color of the target building based on the color of the outer wall pixel region; and a color rendering module configured to render the color of a building model corresponding to the target building in an electronic map according to the outer wall body color of the target building.

[0007] In an implementation, the pixel region obtaining module comprises a building surface identifying unit, a first region determining unit and a second region determining unit. The building surface identifying unit is configured to identify building surfaces in the street view image based on a building surface identifying model to obtain a building surface identifying result, the building surface identifying result comprising position information of building surface pixel regions in the street view image and building surface types to which the building surface pixel regions belong, the building surface pixel region being a pixel region in which a building surface of the target building is located; the first region determining unit is configured to determine at least one target building surface pixel region from building surface pixel regions with a front building surface type based on the building surface identifying result; and the second region determining unit is configured to determine the outer wall pixel region in the at least one target building surface pixel region.

[0008] In an implementation, the second region determining unit is further configured to identify windows in each of the target building surface pixel regions to determine window pixel regions in the target building surface pixel regions; perform mask processing on the window pixel regions in the target building surface pixel regions; and take pixel regions in the target building surface pixel regions that are not masked as the outer wall pixel region.

[0009] In an implementation, the first region determining unit is further configured to determine N building surface pixel regions with a front building surface type from building surface pixel regions in a plurality of street view images based on building surface identifying results of the plurality of street view images, N being a positive integer; determine pixel areas of each of the building surface pixel regions with a front building surface type; select K building surface pixel regions with the largest pixel areas from the N building surface pixel regions with a front building surface type as the target building surface pixel regions based on the pixel areas, K being a positive integer and K not being greater than N.

[0010] In an implementation, the color rendering device further comprises a data acquisition module, a building surface identification module, an identification loss determination module, and an identification model training module. The data acquisition module is configured to acquire a plurality of building images and label information of each building image, the label information being used to indicate label position information of each building surface pixel region in the building image and a building surface type to which each building surface pixel region belongs. The building surface identification module is configured to perform building surface identification on the building image by using a building surface identification model to obtain a reference building surface identification result, the reference building surface identification result indicating predicted position information of each building surface pixel region in the building image and a predicted building surface type to which each building surface pixel region belongs. The identification loss determination module is configured to determine a building surface identification loss according to the label position information of each building surface pixel region in the building image and the building surface type to which each building surface pixel region belongs, and the predicted position information of each building surface pixel region in the building image and the predicted building surface type to which each building surface pixel region belongs. The identification model training module is configured to adjust parameters of the building surface identification model according to the building surface identification loss.

[0011] In an implementation, the plurality of outer wall pixel regions are provided, and the color determination module comprises a clustering unit, a statistical unit, a category determination unit, and a color determination unit. The clustering unit is configured to cluster colors of the plurality of outer wall pixel regions to determine a clustering result, the clustering result indicating a category to which each outer wall pixel region belongs. The statistical unit is configured to count a number of colors belonging to each category based on the clustering result. The category determination unit is configured to determine a target category as a category with the largest number of colors. The color determination unit is configured to determine an outer wall color of the target building according to colors belonging to the target category in the clustering result.

[0012] In an implementation, the device further comprises a segmentation module and a proportion determination module. The segmentation module is configured to perform super-pixel segmentation on the outer wall pixel region to determine at least one super-pixel region in the outer wall pixel region. The color determination module is further configured to perform color extraction on each super-pixel region to determine a color of each super-pixel region. The proportion determination module is configured to determine a color proportion of each color in the outer wall pixel region based on the color of each super-pixel region and a pixel proportion of each super-pixel region in the outer wall pixel region. The color determination module is further configured to determine a color with the largest color proportion in the outer wall pixel region as the color of the outer wall pixel region.

[0013] In an implementation, the color rendering module is further configured to, when it is determined that the target building identifier corresponds to a superior building identifier based on a hierarchical relationship between the target building identifier and different building identifiers, render the building model of the reference building in the electronic map in the color of the outer wall of the target building, the reference building being a building other than the target building in a building group indicated by the superior building identifier corresponding to the target building identifier.

[0014] In an implementation, the image obtaining module includes an image obtaining unit, a main building identifying unit, and an image selecting unit. The image obtaining unit is configured to obtain a plurality of candidate street view images associated with a target building identifier of a target building. The main building identifying unit is configured to perform main building identification on each of the candidate street view images by using a main building identification model to obtain a main building identification result of each of the candidate street view images, the main building identification result being used to indicate whether the main building of the target building is present in the candidate street view image. The image selecting unit is configured to select, based on the main building identification result of each of the candidate street view images, a candidate street view image in which the main building of the target building is present from the plurality of candidate street view images as the street view image in which the target building is present.

[0015] The present application provides an electronic device, including: one or more processors; a memory; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to perform the color rendering method provided in the present application.

[0016] The present application provides a computer readable storage medium, which stores program codes, and the program codes can be invoked by a processor to execute the color rendering method provided in the present application.

[0017] The present application provides a computer program product, which includes computer programs / instructions, and the computer programs / instructions are executed by a processor to implement the color rendering method provided in the present application.

[0018] The present application has the following beneficial effects:

[0019] The application obtains a street view image presenting a target building, obtains an outer wall pixel region where an outer wall of the target building is located in the street view image, determines an outer wall color of the target building based on a color of the outer wall pixel region, and renders a building model of the target building in an electronic map according to the outer wall color of the target building. In the application, since the street view image provides a ground view real scene picture, the color of the pixel region where the outer wall of the target building is located in the street view image represents the outer wall color of the target building. By taking the outer wall color as the color of the building model of the target building in the electronic map, highly realistic color restoration can be realized on the building model in the electronic map, so that the electronic map can restore the scene in the real world more realistically, and the problem that the color of the building model in the electronic map is relatively single in the prior art is solved. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative effort.

[0021] Figure 1 is an architecture schematic diagram of a color rendering system provided by the embodiments of the application;

[0022] Figure 2 is a structure schematic diagram of a server provided by the embodiments of the application;

[0023] Figure 3 is a flow schematic diagram of a color rendering method provided by the embodiments of the application;

[0024] Figure 4 is another flow schematic diagram of a color rendering method provided by the embodiments of the application;

[0025] Figure 5 is a schematic diagram of subject building identification of a candidate street view image provided by the embodiments of the application;

[0026] Figure 6 is another flow schematic diagram of a color rendering method provided by the embodiments of the application;

[0027] Figure 7 is a schematic diagram of building surface identification of a street view image provided by the embodiments of the application;

[0028] Figure 8 is a schematic diagram of determining an outer wall pixel region from a target building surface pixel region provided by the embodiments of the application;

[0029] Figure 9 is another flowchart of a color rendering method provided by an embodiment of the present application;

[0030] Figure 10 is another flowchart of a color rendering method provided by an embodiment of the present application;

[0031] Figure 11 is a schematic diagram of color clustering on a pixel region of an external wall provided by an embodiment of the present application;

[0032] Figure 12 is still another flowchart of a color rendering method provided by an embodiment of the present application;

[0033] Figure 13 is a schematic diagram of an electronic map after color rendering provided by an embodiment of the present application;

[0034] Figure 14 is a flowchart of determining a color of a building in a color rendering method provided by an embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings, and the described embodiments should not be regarded as limiting the present application, and all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0036] In the following description, "some embodiments" are described, which describe a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict. In the following description, the term "a plurality of" refers to at least two.

[0037] In the following description, the term "first\second" is only to distinguish similar objects, and does not represent a specific order of the objects. It can be understood that "first\second" can be interchanged in a specific order or sequence as allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0038] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0039] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined target, and can be implemented in whole or in part by using software, hardware (such as a processing circuit or a memory), or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an integral module or unit that includes the functions of the module or unit.

[0040] Before the embodiments of the present application are further described in detail, the terms and phrases involved in the embodiments of the present application are explained, and the terms and phrases involved in the embodiments of the present application are applicable to the following explanations.

[0041] Computer vision technology: (Computer Vision, CV) Computer vision is a science that studies how to make machines "see", and more specifically, it refers to using cameras and computers to replace human eyes to identify, track and measure targets, and further perform image processing to make computer processing more suitable for human eye observation or image transmission to instrument detection. As a scientific discipline, computer vision researches related theories and technologies, and tries to establish artificial intelligence systems that can obtain information from images or multidimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR (Optical Character Recognition), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and other technologies. It also includes common face recognition, fingerprint recognition and other biometric identification technologies.

[0042] Point of interest: also known as POI (Point of Interest) or Point of Information. For a map product, a geographical location of interest or concern can be referred to as a POI, such as a store, a bar, a gas station. In the expression of an electronic map, a POI can represent a building, a shop, a scenic spot, or the location of a community, etc.

[0043] Street view image: usually refers to an image that presents a city street. In the present application, a street view image that presents a building refers to an image that presents the exterior or appearance of the building. The street view image can be collected by a street view collection vehicle, or taken by a smartphone, tablet, camera, drone, etc.

[0044] Reference Figure 1 , Figure 1is a schematic diagram of an application scenario of the color rendering method provided by the embodiments of the present application, in which the terminal device 400 connects the server 200 through the network 300, and the server 200 connects the database 500, wherein the network 300 can be a wide area network or a local area network, or a combination of the two.

[0045] In some embodiments, taking the server as an example, the color rendering method provided by the embodiments of the present application can be implemented by the server. For example, the server 200 can obtain a street view image presenting a target building from the database 500; obtain an outer wall pixel region where an outer wall of the target building is located in the street view image; determine an outer wall body color of the target building based on the color of the outer wall pixel region; and perform color rendering on a building model of the target building in an electronic map according to the outer wall body color of the target building. The storage location of the street view image presenting the target building is not limited, and is not limited to the database 500, for example, can also be stored in a distributed file system of the server 200, a blockchain, etc.

[0046] After the server 200 completes the color rendering on the building model of the target building in the electronic map, the server 200 can send the electronic map after color rendering to the terminal device 400 in response to the map acquisition instruction sent by the terminal device 400, so that the terminal device 400 displays the electronic map after color rendering after receiving the electronic map after color rendering.

[0047] It should be understood that the color rendering method provided by the embodiments of the present application can also be implemented by the terminal device 400, or by the terminal device 400 and the server 200 together.

[0048] In some embodiments, the terminal device 400 or the server 200 can implement the color rendering method provided by the embodiments of the present application by running a computer program, for example, the computer program can be a native program or a software module in the operating system; can be a native application (APP, Application), that is, a program that needs to be installed in the operating system to run; can also be a mini program, that is, a program that only needs to be downloaded into a browser environment to run; can also be a mini program that can be embedded into any APP, and the mini program can be controlled by the user to run or close. In summary, the above computer program can be any form of application program, module or plug-in.

[0049] In some embodiments, the server 200 can be a standalone physical server, a server cluster composed of multiple physical servers, or a distributed system, and can also be a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and basic cloud computing services such as big data and artificial intelligence platforms. The terminal device 400 can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart voice interaction device, a smart home appliance, a vehicle-mounted terminal, an aircraft, and the like, but is not limited thereto. The terminal device and the server can be connected directly or indirectly through wired or wireless communication, and the present embodiments are not limited in this regard.

[0050] The present embodiments can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, intelligent transportation, and assisted driving.

[0051] In some embodiments, various data (such as street view images) involved in the present embodiments can be stored in a blockchain, and the data can be ensured to be reliable based on the tamper-proof nature of the blockchain.

[0052] The electronic device provided by the present embodiments is taken as an example of a server, and referring to Figure 2 , Figure 2 is a structural schematic diagram of the server 200 provided by the present embodiments, Figure 2 The server 20 shown in FIG. 1 includes at least one processor 210, a memory 250, and at least one network interface 220. The various components in the server 200 are coupled together by a bus system 240. It can be understood that the bus system 240 is used to realize the connection and communication between the components. In addition to a data bus, the bus system 240 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, all the buses are marked as the bus system 240 in Figure 2 .

[0053] The processor 210 can be an integrated circuit chip having a signal processing capability, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0054] The memory 250 can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 250 can optionally include one or more storage devices that are physically located away from the processor 210.

[0055] The memory 250 includes volatile memory or nonvolatile memory, and can include both volatile and nonvolatile memory. The nonvolatile memory can be read only memory (ROM), and the volatile memory can be random access memory (RAM). The memory 250 described in the embodiments of the present application is intended to include any suitable type of memory.

[0056] In some embodiments, the memory 250 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, which are exemplarily illustrated below.

[0057] The operating system 251 includes system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0058] The network communication module 252 is used to reach other computing devices via one or more (wired or wireless) network interfaces 220, exemplary network interfaces 220 including Bluetooth, wireless fidelity (Wi-Fi), and universal serial bus (USB), etc.

[0059] In some embodiments, the color rendering device provided by the embodiments of the present application can be implemented in a software manner, Figure 2 A color rendering device 255 stored in the memory 250 is shown, which can be software in the form of programs and plug-ins, etc., including the following software modules: an image acquisition module 2551, a pixel region acquisition module 2552, a color determination module 2553, and a color rendering module 2554. These modules are logical, and thus can be combined or further split according to the implemented functions. The functions of each module will be described below.

[0060] The color rendering method provided by the embodiments of the present application will be described in conjunction with exemplary applications and implementations of the electronic device provided by the embodiments of the present application.

[0061] Referring to Figure 3 , Figure 3 is a flowchart of the color rendering method provided by the embodiments of the present application, which will be described below in conjunction with the steps shown in Figure 3

[0062] Step S110: Acquire a street view image showing a target building.

[0063] ​The target building refers to a building whose outer wall color is to be extracted, which can be a building, a building in a scenic spot, one or more buildings in a community, or one or more buildings in a building complex, etc.

[0064] The street view image presenting the target building refers to an image presenting a part or the whole appearance of the target building. It can be understood that the street view image presenting the target building is at least taken from the outside of the target building, and thus the street view image presenting the target building can present the outer wall of the target building. The street view image can be a series of continuous and high-quality photos taken on an actual street by a street view car, a drone or other special equipment, which usually has a high resolution; and is usually taken under natural lighting conditions, and can reflect the true color of the building under different time and weather conditions. In other embodiments, the street view image can also include an image taken and uploaded by a user and published.

[0065] The street view image presenting the target building can be one or more. For example, when the street view image is more than one, the multiple street view images can be images taken at different angles, or images taken under different weather conditions, or images taken at different positions, or a combination thereof, which is not limited herein.

[0066] In some embodiments, since part of the building has the building name of the building marked on the outer wall, a street view image presenting the building name of the target building can be obtained from the street view image as the street view image presenting the target building.

[0067] For example, if the building name of the target building is Building 1 in A Community, a street view image presenting "Building 1 in A Community" can be obtained as the street view image presenting the target building; and if the building name of the target building is YY Mansion, a street view image presenting the word "YY Mansion" can be obtained as the street view image presenting the target building.

[0068] If the street view image presenting the target building is more than one, and the target building is multiple buildings in the same community, such as multiple buildings in "A Community", the multiple street view images can include a street view image presenting the word "Building 1 in A Community", a street view image presenting the word "Building 3 in A Community", etc. If the street view image presenting the target building is more than one, and the target building is multiple buildings in the same building complex, such as multiple buildings in "XX Mansion", the multiple street view images can include a street view image presenting the word "Building C in XX Mansion", a street view image presenting the word "Building F in XX Mansion", etc.

[0069] In an implementation, the step S110 can be searching a street view image associated with the identification information of the target building from a database. The database stores a plurality of images associated with identification information of each building, and the identification information of the building includes one or more information capable of identifying the building, such as the name of the building, the location (e.g. the latitude and longitude coordinates), etc. For example, if the database is a database of a map application, a user can take an image at a location and associate the taken image with a location point shown in the map application at the location, and publish the image in the map application. Thus, the database of the map application can store images associated with each location point. On this basis, the image associated with the location information of the target building can be obtained from the database of the map application to obtain the street view image of the target building.

[0070] For another example, in a content sharing interactive application such as a travel application, the sharing content (e.g. life notes) published by a user includes a topic label and an image. The topic label can be the name of an object to which the published sharing content is directed, such as a building name. The corresponding published image can be an image of the building indicated by the building name. A database of the content sharing interactive application stores a large amount of sharing content published by users. In this case, the sharing content including the building name of the target building in the topic label can be obtained from the database of the content sharing interactive application. Then, the image presenting the building can be obtained from the sharing content to filter the street view image presenting the target building from the images presenting the building.

[0071] For another example, in a life service application, a user can publish review content for an object, such as a store, a scenic spot, a building, etc. The published review content is displayed in association with the object in a page of the life service application. The published review content includes an image related to the object. For example, if the object is a building, the published review content includes an image presenting the appearance of the building, an image presenting the interior decoration of the building, an image presenting the surrounding environment of the building, etc. The review content published by users for each object is stored in a database of the life service application. On this basis, the review content published for the target building can be obtained from the database of the life service application. Then, the image can be obtained from the obtained review content to filter the street view image presenting the target building from the images.

[0072] In some embodiments, the images associated with the identification information of the target building obtained in the database of the above-mentioned travel application, map application, life service application and other content sharing interactive applications (for the convenience of description, the images associated with the identification information of the target building are referred to as candidate images) may be images that mainly present the interior of the target building, or images that do not present the target building. In the solution of the present application, the street view image presenting the target building is obtained mainly to determine the exterior wall color of the target building. Therefore, it is necessary to filter out the street view image presenting the exterior wall of the target building from the candidate images. On this basis, the candidate images can be further filtered to obtain an image presenting the building name of the target building as the street view image presenting the target building. In this way, by first filtering out the associated images based on the identification information (location information or name, etc.) of the target building, and then filtering out the image presenting the building name of the target building from the images filtered in the previous step as the street view image presenting the target building, it is ensured that the street view image obtained by filtering presents the exterior wall of the target building, thereby providing a data basis for the subsequent exterior wall color extraction, thereby ensuring the accuracy of the exterior wall color extraction.

[0073] In some embodiments, based on the multiple candidate images obtained from the database of travel applications, map applications, life service applications and other content sharing interactive applications as described above, the candidate images can be used to identify building exterior wall elements to obtain candidate images presenting building exterior wall elements as street view images.

[0074] It should be understood that the collection, use, processing and storage of the above street view images must comply with the regulations of the region. Step S120: Obtain the outer wall pixel area where the outer wall of the target building is located in the street view image.

[0075] The exterior wall of a target building generally refers to the visible portion of the wall that constitutes the exterior of the building's main structure. Alternatively, the exterior wall of a target building refers to the wall corresponding to the exterior facade of the target building. In this application, the pixel region containing the exterior wall of a target building in a street view image is referred to as the exterior wall pixel region.

[0076] It should be noted that some buildings (such as buildings in residential areas and most office buildings) have facades that include at least one item, such as windows and balconies, in addition to walls. For such buildings, the exterior wall pixel area where the exterior wall is located refers to the wall portion of the building's facade, and does not include the area where windows and balconies are located. Some buildings have the visible portion of the exterior structure (exterior facade) consisting of panels (such as glass panels). Some areas of the facade are windows, and some areas are panels that cannot be opened. In this case, the panels on the building's facade can be regarded as the building's exterior wall.

[0077] In some embodiments, in the case that the facade of the building is panels, and some of the panels are windows, due to the color difference between the pixels of the opened panels (i.e. the windows) and the pixels of the unopened panels (e.g. the panels that are not windows or other panels that cannot be opened) in the image captured from the outside of the building towards the building when the windows are opened, in this case, the pixel region of the facade of the building in the street view image that corresponds to the unopened panels can be determined as the outer wall pixel region.

[0078] In some embodiments, the building facade in the street view image that presents the target building is identified to identify the building facade pixel region in the street view image that presents the building facade, which is the pixel region in the street view image that presents the facade of the building; and then the building facade pixel region is semantically segmented to identify the outer wall pixel region in the building facade pixel region.

[0079] Step S130: determining the color of the outer wall of the target building based on the color of the outer wall pixel region.

[0080] In some embodiments, the color of the outer wall pixel region can be extracted to determine the color of the outer wall pixel region.

[0081] In some embodiments, considering that the reflection and absorption characteristics of different regions on the outer wall of the building are different, which results in the color difference of the pixels at different positions in the outer wall pixel region. And the texture and color of the material of the outer wall of the building may also be different, such as the color difference between the bricks of a brick wall, the natural texture and color variation of a concrete or stone surface, which also results in the color difference of the pixels at different positions in the outer wall pixel region. Or the white balance setting, exposure compensation, noise, etc. of the device (such as a drone, a camera on a street view car or a camera on a user terminal) used to capture the image may also result in the color difference of the pixels at different positions in the outer wall pixel region. Therefore, the color extracted from the outer wall pixel region may be multiple. In this case, the multiple colors involved in the outer wall pixel region can be determined first, and one of the multiple colors involved in the outer wall pixel region can be selected as the color of the outer wall pixel region.

[0082] In one implementation, the way to determine the color of the outer wall pixel region based on the color of the different pixels in the outer wall pixel region can be: to calculate the average value of the color values (such as RGB or HSV) of all the pixels in the outer wall pixel region, and to determine the color corresponding to the average value as the color of the outer wall pixel region.

[0083] In some embodiments, the color value (such as RGB or HSV) of each pixel point in the exterior wall pixel area is statistically analyzed to obtain the number of pixels corresponding to different colors, and a color with a number of pixels greater than a preset number is obtained. The color of the exterior wall pixel area is determined based on the color and the corresponding number of pixels. For example, the color mean is calculated for the color with a number of pixels greater than the preset number, and the color indicated by the calculated color mean is used as the color of the exterior wall pixel area; alternatively, the color with the largest number of pixels is used as the color of the exterior wall pixel area.

[0084] In some embodiments, principal component analysis is performed on the colors of all pixels in the exterior wall pixel area to determine the main eigenvalues ​​and eigenvectors of the color distribution in the exterior wall pixel area, and the colors corresponding to the main eigenvalues ​​and eigenvectors are used as the colors of the exterior wall pixel area.

[0085] When determining the exterior wall color of a target building based on the color of an exterior wall pixel region, if there is a single exterior wall pixel region, the color of that exterior wall pixel region can be determined as the exterior wall color of the target building. If there are multiple exterior wall pixel regions, a color reference value can be obtained for the colors of the multiple exterior wall pixel regions, and the color indicated by the color reference value can be used as the exterior wall color of the target building. The color reference value can be a color average, a color median, etc., which is not specifically limited here.

[0086] In some embodiments, the colors of multiple exterior wall pixel regions can be clustered to determine the category to which the color of each exterior wall pixel region belongs. Each category represents a color tendency. The exterior wall color of the target building is then determined based on the color represented by the category with the largest coverage area. For example, the color mean of the colors belonging to the category with the largest coverage area is calculated, and the color indicated by the calculated color mean is used as the exterior wall color of the target building.

[0087] It should be understood that the above-mentioned method of determining the color of the exterior wall of the target building based on the color of the exterior wall pixel area is only illustrative, and there are more confirmation methods, which will not be detailed here.

[0088] Step S140: performing color rendering on the building model of the target building in the electronic map according to the color of the outer wall of the target building.

[0089] The building model of the target building in the electronic map refers to a three-dimensional model constructed for the target building, and the building model can be a three-dimensional model with the same shape as the target building. In a specific embodiment, the building model of the target building in the electronic map can be rendered into the exterior wall color of the target building by a rendering engine.

[0090] The color of the target building in the electronic map is rendered according to the color of the outer wall of the target building, so that the color of the building model of the target building in the electronic map is the color of the outer wall of the target building, thereby realizing that the color of the building model corresponding to the target building in the electronic map is the same as the appearance color of the target building in the actual physical environment, that is, the color of the target building in the actual physical environment is restored in the electronic map, the restoration degree of the building model presented in the electronic map is improved, and the user experience is improved.

[0091] In some embodiments, the outer wall material can also be identified based on the outer wall pixel region to determine the outer wall building material of the target building, and then in the process of rendering the building model of the target building in the electronic map, the material of the building model is determined according to the outer wall building material of the target building, and the color of the building model is rendered according to the color of the outer wall of the target building. Therefore, after rendering, the texture of the building model of the target building in the electronic map is basically consistent with the texture of the outer wall of the target building, and the color is also basically consistent. In this way, the color and texture of the target building in the actual physical environment are restored in the electronic map, the appearance restoration degree of the actual building in the electronic map is further improved, and the user experience is improved.

[0092] Specifically, when identifying the outer wall material based on the outer wall pixel region, the feature of the outer wall pixel region can be extracted by using a material identification model, the prediction probability of each material corresponding to the outer wall pixel region is determined based on the feature of the outer wall pixel region, and the material corresponding to the outer wall pixel region is finally determined based on the prediction probability of each material. The material identification model can be obtained by training the outer wall image carrying the material label.

[0093] By using the above method of the present application, since the street view image provides a real scene picture from the ground view, the color of the pixel region where the outer wall of the target building in the street view image represents the outer wall color of the target building. By taking the outer wall color as the color of the building model of the target building in the electronic map, the problem of single color of the building model in the electronic map in the prior art is solved.

[0094] In addition, by taking the outer wall color as the color of the building model of the target building in the electronic map, highly realistic color restoration can be realized on the building model in the electronic map, so that the electronic map can more realistically restore the scene in the real world, and the user can quickly correspond the building to the building in the real world based on the color of the building model when using the electronic map.

[0095] Please refer to Figure 4As shown, considering that the street view images are usually stored in the database in association with the identification information of the target building, and there may be some street view images that do not present the main structure of the target building, in order to quickly and accurately obtain the street view images presenting the target building, in an implementable manner, step S110 can include:

[0096] Step S110a: Obtain a plurality of candidate street view images associated with the target building identification of the target building.

[0097] The target building identification can be the building name of the target building, or the identification string assigned to the target building in the database, and can also be the location information of the target building.

[0098] Among them, the plurality of candidate street view images are a plurality of candidate street view images stored in the database in association with the target building identification.

[0099] Step S110b: using the main building identification model to identify the main building of each candidate street view image, and obtaining the main building identification result of each candidate street view image.

[0100] The main building identification result is used to indicate whether the main building of the target building is presented in the candidate street view image, and the main building identification model is obtained by training a plurality of sample street view images and label information of each sample street view image. The label information is used to indicate whether the main building of the corresponding building is presented in the sample street view image.

[0101] The main building of the building refers to the main part of the building, which can also be understood as the main outer wall or most of the outer wall of the building. Correspondingly, if a candidate street view image presenting a building presents most of the outer wall of the building, it can be considered that the candidate street view image presents the main building of the building.

[0102] When using the main building identification model to analyze the street view image, the task of the main building identification model is to identify whether the main outer wall of the building is presented in the candidate street view image. This means that the main building identification model needs to distinguish whether the content in the image focuses on the outer wall of the building, rather than the interior decoration of the building, or the surrounding environment, or the local appearance, etc.

[0103] Specifically, the training process of the main building recognition model is as follows: using the feature extraction network in the building recognition model to extract features from the sample street view image to obtain image features; using the classification network in the building recognition model to perform classification based on the image features to obtain the predicted probability of the main building present in the sample street view image, obtaining a loss value based on the predicted probability and label information, and adjusting the model parameters of the main building recognition model based on the loss value. The neural network used in the building recognition model can be, but is not limited to, a ResNet (Residual Network) network, a VGG (Visual Geometry Group Network) network, an Inception (Inception Convolutional Neural Network) network, etc.

[0104] Step S110c: Based on the main building recognition results of each candidate street view image, a candidate street view image showing the main building of the target building is selected from the plurality of candidate street view images as the street view image showing the target building.

[0105] Since the candidate street view image of the main building of the target building is presented, the main outer wall of the target building is presented. Therefore, the outer wall color of the target building can be accurately extracted based on the candidate street view image of the main building of the target building.

[0106] like Figure 5 As shown, it is exemplified that the target building is "XX Building", and there are 4 candidate street view images (A, B, C and D) obtained from the database for "XX Building". The main building recognition model is used to perform main building recognition on multiple candidate images of the target building "XX Building". It can be determined that A and B present the main building of the target building "XX Building", so A and B are used as street view images presenting the target building, C and D do not present the main building of the target building "XX Building", and C and D are candidate street view images that need to be discarded so that the color of the target building can be determined later using the street view images presenting the target building.

[0107] By performing steps S110a to S110c above, it is ensured that the obtained street view image showing the target building shows the main structure of the target building, that is, the main exterior wall of the target building. By selecting a candidate street view image showing the main structure of the target building from multiple candidate street view images, it is facilitated to subsequently accurately extract the exterior wall color of the target building from the street view image showing the main structure of the target building.

[0108] In the same street view image, multiple building faces of a building are presented, and the colors of different building faces can be different, and the user is more concerned about the color of the outer wall of the front face of the building. Based on this, in an embodiment of the present application, please refer to Figure 6 The step S120 can include steps S120a-S120c as follows:

[0109] Step S120a: building face recognition is performed on the street view image by a building face recognition model to obtain a building face recognition result.

[0110] The building face recognition result includes position information of the building face pixel region in the street view image and the building face type to which each building face pixel region belongs. The building face pixel region refers to the pixel region where the building face of the target building is located.

[0111] The building face type to which the building face pixel region belongs represents the building face type of the building face represented by the building face pixel region. The building face type can include a front face and a side face. The front face usually refers to the most important and representative face of the building, which is usually the side facing the street or public open space, and the front face is usually the design focus of the building. The side face plays a more auxiliary role, taking into account the overallity, practicality of the building and the harmony and unity of the surrounding environment. Therefore, in practice, the user is more concerned about the color of the outer wall of the building as the front face, or the user's perception of the color of the building is mainly the color of the outer wall of the building as the front face.

[0112] The building face recognition model is used to identify the position information (i.e. pixel position information) of each building face presented in the image in the image and the building face type of each building face presented. The building face recognition model can be constructed by one or more neural networks, such as convolutional neural network, fully connected neural network, transformer network (Transformer), etc., which are not specifically limited here.

[0113] The building face recognition model can be obtained by training a neural network model using an image dataset containing labeled building wall region and type.

[0114] For example, please refer to Figure 7 If the building recognition model is used to recognize the three street view images (E and Figure 5If A and B in the street view image are respectively identified, the building surface identification result of the building wall surface area and the type of the building wall surface area can be obtained. Specifically, the identification result of A is A1, and the building wall surface area and the building surface type corresponding to the building wall surface area are marked as a front surface in A1. The thick black closed line in A1 is the boundary of the building wall surface area, and the pixel area surrounded by the thick black closed line in A1 is the building wall surface area. The identification result of B is B1, and the building wall surface area and the building surface type corresponding to the building wall surface area are marked as a front surface in B1. The thick black closed line in B1 is the boundary of the building wall surface area, and the pixel area surrounded by the thick black closed line in B1 is the building wall surface area. The identification result of E is E1, and two building wall surface areas are marked in E1. The pixel area surrounded by the thick black closed line on the left is one of the building wall surface areas, and the building surface type corresponding to the building wall surface area is a front surface. The pixel area surrounded by the thick black closed line on the right is the other building wall surface area, and the building surface type corresponding to the building wall surface area is a side surface.

[0115] Step S120b: determining at least one target building surface pixel area from the building surface pixel area with a front surface type based on the building surface identification result.

[0116] When the street view image is one, if the building surface pixel area with a front surface type included in the street view image is one, the building surface pixel area can be determined as the target building surface pixel area. If the building surface pixel area with a front surface type included in the street view image is multiple, the multiple building surface pixel areas with a front surface type can be sorted in descending order of the number of pixel points, and the building surface pixel area with a front surface type with a preset value in the sorting can be taken as the target building surface pixel area.

[0117] When the street view image is multiple, such as M, at least one target building surface pixel area can be selected from the M street view images. K target building surface pixel areas can also be selected from the M street view images, wherein the number of pixel points included in the target building surface pixel area is greater than the number of pixel points included in the pixel area that is not selected. It should be understood that in this implementation, there can be a street view image that does not include a target building surface pixel area.

[0118] In an implementation, if the street view images are multiple, each street view image corresponds to a building surface recognition result including at least one building surface pixel region with a front building surface type, the step S120b can be, for each street view image, if there is only one building surface pixel region with a front building surface type in the street view image, determining the building surface pixel region as a target building surface pixel region; if there are multiple building surface pixel regions with a front building surface type in the street view image, determining a target building surface pixel region from the multiple building surface pixel regions based on the number of pixel points corresponding to each building surface pixel region. In this implementation, each street view image includes at least one target building surface pixel region.

[0119] The way of determining a target building surface pixel region from the multiple building surface pixel regions based on the number of pixel points corresponding to each building surface pixel region can be: obtaining the number of pixel points in each building surface pixel region with a front building surface type, sorting the multiple building surface pixel regions with a front building surface type in descending order of the number of pixel points, and selecting a building surface pixel region with a front building surface type as a target building surface pixel region; or: obtaining the number of pixel points in the street view image and the number of pixel points in each building surface pixel region with a front building surface type, determining the pixel proportion of each building surface pixel region in the street view image according to the number of pixel points in the multiple building surface pixel regions with a front building surface type and the number of pixel points in the street view image, and selecting a building surface pixel region with a front building surface type as a target building surface pixel region.

[0120] In this way, when the street view images of the target building are M, and the target building surface pixel regions determined are K, K and M are positive integers, and K>M.

[0121] In another implementation, if the street view images are multiple, the step S120b can further include: determining N building surface pixel regions with a front building surface type from the multiple street view images based on the building surface recognition results of the multiple street view images, N being a positive integer; determining the pixel area of each building surface pixel region with a front building surface type; selecting K building surface pixel regions with a front building surface type from the N building surface pixel regions with a front building surface type based on the pixel area, as target building surface pixel regions, K being a positive integer, and K≤N.

[0122] It should be understood that the foregoing way of determining a target building surface pixel region is only illustrative, and there can be more ways of determining a target building surface pixel region, which are not described herein.

[0123] Step S120c: determining an outer wall pixel region in the at least one target building surface pixel region.

[0124] In the target building facade pixel region, in addition to the outer wall pixel region, there can be a window pixel region, an entrance passage pixel region, a decorative element pixel region, a billboard or sign pixel region, an air conditioner outdoor unit or air vent pixel region, and an illumination device pixel region, and the like. The outer wall pixel region represents all pixels of the outer wall part of the building; the window pixel region represents all pixels of the window part of the building; the decorative element pixel region is composed of pixels such as balconies, columns, eaves, and line decorations; the entrance passage pixel region includes pixels of building entrance auxiliary structures such as sunshades, rainshades, and porticos; the illumination device pixel region includes pixels of illumination devices installed on the outer facade of the building, such as wall lamps, neon lights, and spotlights; the billboard or sign pixel region includes pixels of advertising materials or business signs attached to the outer facade of the building; and the air conditioner outdoor unit or air vent pixel region includes pixels of areas where air conditioner outdoor units, air vent duct openings, and the like are installed on the outside of the building.

[0125] Correspondingly, in some embodiments, wall body recognition can be performed on the target building facade pixel region to determine the outer wall pixel region in the target building facade pixel region. In specific embodiments, semantic segmentation can be performed on the target building facade pixel region to obtain a semantic segmentation result, which is used to indicate the wall body pixel region and the non-wall body pixel region in the target building facade pixel region, so as to achieve wall body recognition. It is worth mentioning that, since the target building facade pixel region corresponds to an outer facade of the target building, the wall body pixel region in the target building facade pixel region is the outer wall body pixel region.

[0126] In some embodiments, the non-outer wall pixel region in the target building facade pixel region is identified by using the identification model, and the non-outer wall pixel region in the target building facade pixel region is subjected to mask processing, and the pixel region in the target building facade pixel region that is not masked is taken as the outer wall pixel region.

[0127] In an implementation manner of the present application, the above step S120c can be wall body recognition on each target building facade pixel region to determine the wall body pixel region in the target building facade pixel region. In this implementation manner, semantic segmentation can be performed on each pixel point in the target building facade pixel region to determine the semantic type of each pixel point, and the semantic type is used to indicate whether the pixel point is a wall body. The set of pixel points of the target building facade pixel region whose type is a wall body is taken as the outer wall pixel region.

[0128] It should be understood that when the wall includes walls, windows, entryways, decorative elements, billboards, air conditioner outdoor units, vents, lighting equipment, etc., the pixel points of non-wall type can be pixel points in the window pixel area, entryway pixel area, decorative element pixel area, billboard or signboard pixel area, air conditioner outdoor unit or vent pixel area and lighting equipment pixel area.

[0129] In another possible implementation of the present application, if the building surface includes walls and windows, the above step S120c can also be: performing window recognition on each target building surface pixel area to determine the window pixel area in the target building surface pixel area; performing mask processing on the window pixel area in the target building surface pixel area, and using the unmasked pixel area in the target building surface pixel area as the exterior wall pixel area. The mask processing on the window pixel area in the target building surface pixel area can be performed by setting the window pixel area in the target building surface pixel area to a mask of a specified color, and the mask of the specified color is different from the color of the pixel points representing the wall in the target building surface pixel area. In this way, in the subsequent process, if the masked target building surface pixel area is processed, only the unmasked pixel area in the masked target building surface pixel area can be processed, without paying attention to the masked window pixel area therein.

[0130] For example, Figure 8 As shown, it shows the Figure 7 After extracting the target building surface pixel area B2 from B1, the window recognition of the target building surface pixel area B2 obtains the recognition result shown in B3. The pixel areas surrounded by the small boxes in B3 are the window pixel areas. Masking B3 can obtain the mask area shown in B4. Afterwards, the unmasked pixel areas in the target building surface pixel area can be used as the exterior wall pixel area.

[0131] In the above manner of determining the wall pixel region, the image recognition model for wall recognition (for the sake of distinction, referred to as wall recognition model) can be pre-trained, and the specific training process can be: obtaining an image dataset for training the image recognition model, which can include a plurality of first sample building images and first semantic labels of pixels in the first sample building images, wherein the first semantic labels are used to indicate whether the pixels represent walls. The first sample building images included in the image dataset can be satellite images, aerial images, street view images, or building plans, etc. The image recognition model used can include a feature extraction layer and a classification layer, wherein the feature extraction layer is used to extract semantic features of each pixel, and the classification layer is used to perform classification recognition based on the extracted semantic features to obtain first predicted semantics of each pixel in the first sample building images, which are used to indicate whether the pixels are predicted to represent walls. Then, the image recognition loss is determined based on the first predicted semantics of each pixel in the first sample building images and the first semantic labels of each pixel in the first sample building images, and the parameters of the image recognition model are adjusted based on the image recognition loss until a first training end condition is reached.

[0132] In the above embodiment, the image recognition model for window recognition (for the sake of distinction, referred to as window recognition model) can be trained in a similar manner, and the training data for training the image recognition model for window recognition includes a plurality of second sample building images and second semantic labels of pixels in the second sample building images, wherein the second semantic labels are used to indicate whether the corresponding pixels represent windows; then, the second sample building images are input into the window recognition model, the semantic feature map of the second sample building images is extracted by the window recognition model, and classification is performed according to the semantic feature map to output second predicted semantics of each pixel in the second sample building images, which are used to indicate whether the corresponding pixels represent windows; then, the window recognition loss is determined according to the second predicted semantics of each pixel in the second sample building images and the second semantic labels of each pixel in the second sample building images, and the parameters of the window recognition model are adjusted according to the window recognition loss until a second training end condition is reached.

[0133] Referring to Figure 9 In an implementation manner, the building surface recognition model used in the above step S120a can be trained based on steps S210-S240:

[0134] Step S210: obtaining a plurality of building images and label information of each building image.

[0135] The label information is used to indicate the labeled position information of each building surface pixel region in the building image and the building surface type to which each building surface pixel region belongs.

[0136] The plurality of building images can cover various building styles, materials, colors, and environmental lighting conditions. After performing the above step S210, the plurality of building images can be normalized, cropped, enhanced, and the like.

[0137] Step S220: performing building surface recognition on the building image by using a building surface recognition model to obtain a reference building surface recognition result.

[0138] The reference building surface recognition result indicates the predicted location information of each building surface pixel region in the building image and the predicted building surface type to which each building surface pixel region belongs.

[0139] The building surface recognition model can be a deep learning model, such as Mask R-CNN (Mask Region Convolutional Neural Network), U-Net (U-shaped Convolutional Neural Network), and the like, which can perform target detection and semantic segmentation tasks, and can simultaneously predict the boundaries of the building surface and the building surface type. In an implementation, the building surface recognition model can include a feature extraction layer (usually based on a pre-trained convolutional neural network such as ResNet, VGG, etc.), a detection layer, a segmentation layer (RoI Pooling, FPN, Decoder, etc.), and an output layer. When the building surface recognition model is used to perform building surface recognition on the building image, the functions and processing procedures of each layer are as follows:

[0140] The feature extraction layer can extract rich multi-level features from the building image. The low-level features mainly include edge, texture and other basic information, and the high-level features can capture more abstract and complex shape and structure information. Through convolution, pooling and other operations, the building image is converted into a feature map with rich semantic information. The detection layer can use target detection algorithms such as Fast R-CNN, Faster R-CNN and YOLO (You Only Look Once) based on the features extracted by the feature extraction layer to detect the candidate regions (Region of Interest, RoI) of the building surface represented by the building image based on the features output by the feature extraction layer. For example, in Faster R-CNN, a series of anchor boxes (Anchor Boxes) surrounding the pixel regions of the building surface are generated using RPN (Region Proposal Network, a neural network structure specially used to generate candidate regions (i.e. region frames that may be target objects)), and then the coordinates of each anchor box are determined. The segmentation layer is used for pixel-level semantic classification of the features of the candidate regions. Specifically, RoI Pooling, Feature Pyramid Network (FPN) or decoder (Decoder) technology can be used. RoI Pooling maps candidate regions of different sizes to fixed-size feature vectors, FPN is used to obtain multi-scale features and fuse them, and Decoder is used for upsampling to restore low-resolution feature maps to the original image size, thereby realizing pixel-level prediction to obtain the building surface type to which each pixel belongs. The output layer outputs the final building surface recognition result, i.e. the predicted position information of each building surface pixel region in the building image and the predicted building surface type to which each building surface pixel region belongs.

[0141] Step S230: Determine the building surface recognition loss according to the labeled position information of the building surface pixel region in the building image and the building surface type to which each building surface pixel region belongs, and the predicted position information of each building surface pixel region in the building image and the predicted building surface type to which each building surface pixel region belongs.

[0142] In determining the building surface recognition loss, a loss function is used to determine the building surface recognition loss according to the labeled position information of the building surface pixel region in the building image and the building surface type to which each building surface pixel region belongs, and the predicted position information of each building surface pixel region in the building image and the predicted building surface type to which each building surface pixel region belongs. The loss function can be one of cross-entropy loss and Dice loss, or a combination of multiple loss functions.

[0143] In some embodiments, in step S230, the first loss can be determined according to the labeled position information of the building surface pixel region in the building image and the predicted position information of each building surface pixel region in the building image; the second loss can be determined according to the building surface type to which each building surface pixel region belongs and the predicted building surface type to which each building surface pixel region belongs; and then the first loss and the second loss are weighted to obtain the building surface identification loss. The first loss reflects the difference between the predicted position information of the predicted building surface pixel region and the labeled position information. The second loss reflects the difference between the predicted building surface type and the actual building surface type.

[0144] In some embodiments, the first loss and the second loss described above can be calculated by an absolute value loss function, a mean square error loss function, a cross-entropy loss function, etc., which are not limited here.

[0145] Step S240: Adjust the parameters of the building surface identification model according to the building surface identification loss until the training end condition is reached.

[0146] The training end condition can be that the number of training reaches a preset number, or the building surface identification loss is less than a preset loss threshold.

[0147] The building surface identification model obtained by the above steps S210-S240 can learn the characteristics of various building surfaces, and can accurately identify each building surface in a new building image, locate the specific position of the building surface in the image, and accurately identify the building surface type to which the building surface belongs. After training, the building surface identification can be quickly and accurately performed on any given building image in practical applications.

[0148] In one implementation manner, when the outer wall pixel region is multiple, please refer to FIG. 1B, the step S130 described above can include steps S130a-S130d: Figure 10

[0149] Step S130a: Clustering the colors of the multiple outer wall pixel regions to determine a clustering result, the clustering result indicating the category to which the color of each outer wall pixel region belongs.

[0150] In step S130a, clustering the colors of the multiple outer wall pixel regions can cluster one or more similar colors into one category. In specific embodiments, K-Means, density-based spatial clustering algorithm, spectral clustering, hierarchical clustering, etc. can be used.

[0151] ​For example, if K-Means clustering is used, first, the color of each pixel region of the outer wall (for example, the color can be represented by RGB values, or by HSV values, etc., which are not specifically limited here) is taken as a data point; then, P (P can be set according to actual needs, which is not specifically limited here, P is a positive integer greater than 1) cluster centers are initialized, and then the color of each pixel region is assigned to the category to which the nearest cluster center belongs, and the cluster center is updated to the average value or center point of the color of all categories. Repeat the above assignment and update steps until the cluster center no longer changes significantly or the maximum number of iterations is reached. Finally, the color of each pixel region of the outer wall is assigned to a category, and the clustering result is a list of categories and the color of the pixel region of the outer wall contained in each category.

[0152] If hierarchical clustering is used, the color of each pixel region of the outer wall is represented as a feature vector, and the value of a dimension in the feature vector is determined by the value of the color of the pixel region of the outer wall in a color channel. For example, the value of a dimension in the feature vector can be equal to the value of the color of the pixel region of the outer wall in a color channel, or equal to the value after normalizing the value of the color of the pixel region of the outer wall in a color channel. Then, for each feature vector corresponding to the color of each pixel region of the outer wall, the similarity (such as Euclidean distance, Manhattan distance, or cosine similarity) between the color of the pixel region of the outer wall and the color of other pixel regions of the outer wall is calculated, and these similarity values form a similarity matrix, where an element of the matrix represents the similarity between the colors of two different pixel regions of the outer wall. Then, a bottom-up (agglomerative) or top-down (divisive) method is used for clustering. For example, in agglomerative clustering, each pixel region of the outer wall is initially considered as an independent category, and then the two closest categories are merged layer by layer according to the similarity matrix until a certain termination condition (such as merging into a predetermined number of categories) is reached. Finally, a clustering tree is generated based on the hierarchical clustering process, which intuitively shows the process of merging the colors of the pixel regions during the clustering process. By pruning the clustering tree (selecting an appropriate threshold), the optimal number of categories can be determined and the colors of the pixel regions of the outer wall can be divided into different categories.

[0153] Step S130b: Based on the clustering result, the number of colors belonging to each category is counted.

[0154] Specifically, all the categories to which the pixel regions of the outer wall belong in the clustering result can be traversed, and the color of each category can be counted to obtain the number of colors belonging to each category. By counting the number of colors of each category, the proportion of the color of each category in the overall color distribution of the outer wall can be understood.

[0155] Step S130c: Determine the category with the largest number of colors as the target category.

[0156] By comparing the number of colors in each category, the category with the largest number of colors is selected as the target category. This is because a large number of colors means that this color is the most representative in the entire exterior wall. Therefore, this category is used as the target category. At this time, the color corresponding to the target category is likely to represent the color of the building's exterior wall.

[0157] Step S130d: Determine the exterior wall color of the target building based on the colors belonging to the target category in the clustering results.

[0158] There may be multiple colors belonging to the target category. The colors of all pixel areas included in the target category can be averaged, and the color indicated by the color average can be used as the exterior wall color of the target building. In the process of averaging, the color values ​​of multiple colors in the same color channel can be averaged to obtain the color average in each color channel. Then, the color indicated by the color average of the multiple channels is used as the color average.

[0159] In other embodiments, the color corresponding to the median of the multiple colors belonging to the target category can be used as the exterior wall color of the target building. The color obtained by using the above method can reflect the color characteristics of most exterior wall pixel areas.

[0160] like Figure 11 As shown, a street view image set consisting of multiple street view images showing a target building is shown. For each street view image in the street view image set, the color of the outer wall pixel area corresponding to the street view image is obtained to obtain a color set. Figure 11 The position of each street view image in the street view image set is the same as the position of the color of the exterior wall pixel area corresponding to the street view image in the color set. For example, if a street view image is located in the i-th row and j-th column in the street view image set, then the color of the exterior wall pixel area corresponding to the street view image is also located in the i-th row and j-th column in the color set. After obtaining the colors of the exterior wall pixel areas corresponding to multiple street view images, a clustering method can be used to obtain multiple categories (such as 3 categories). By counting the number of colors in each category, the target category with the largest number of colors is obtained. Based on the colors corresponding to the target category, the following is finally determined: Figure 11 The exterior wall color of the target building is shown.

[0161] Considering that the exterior wall pixel area includes multiple pixels and the colors of each pixel are different, in order to obtain the color of the exterior wall pixel area more accurately and to obtain the color of the target building more efficiently, please refer to Figure 12In this embodiment, before step S130 is performed, the color of the outer wall pixel region can be determined according to steps S150-S180 as follows:

[0162] Step S150: Superpixel segmentation is performed on the outer wall pixel region to determine at least one superpixel region in the outer wall pixel region.

[0163] The superpixel segmentation is a technology for dividing an image into multiple regions with high homogeneity, and the homogeneity mainly refers to the similarity of pixels in the region in terms of color, texture, etc. The specific segmentation process can be as follows: 1. Select L (L is a positive integer) initial seed points uniformly distributed in the outer wall pixel region as superpixel centers. The L superpixel centers can be uniformly spaced in the color space (such as Lab color space) to take into account the color and spatial distance, and L is an integer greater than 1. 2. For each pixel, calculate the distance of the pixel to each superpixel center, which is usually the joint distance of the pixel and the superpixel center in the color space and the spatial position. 3. Assign each pixel to the nearest superpixel center to form multiple superpixel regions. 4. Take the mean or weighted average of each pixel in the superpixel region in the color space and the position space as the updated superpixel center, and return to step 2 until the superpixel center no longer moves significantly or the maximum number of iterations is reached. The division of the superpixel region is completed.

[0164] Step S160: Color extraction is performed on each superpixel region to determine the color of each superpixel region.

[0165] For each superpixel region, the color values of all pixels inside it are counted, and the color of the superpixel region can usually be obtained by calculating the average of the pixel RGB values or other color quantization indicators (such as the average of the HSV, Lab, etc. color space).

[0166] Step S170: Based on the color of each superpixel region and the pixel proportion of each superpixel region in the outer wall pixel region, the color proportion of each color in the outer wall pixel region is determined.

[0167] Specifically, the pixel area or total number of pixels of each superpixel region is counted, and then divided by the total pixel area or total number of pixels of the outer wall pixel region to obtain the proportion of each superpixel region in the outer wall pixel region. Based on the proportion of each superpixel region in the entire outer wall pixel region and the color of each superpixel region, the color proportion of each color in the outer wall pixel region is determined.

[0168] Step S180: The color with the highest color proportion in the outer wall pixel region is taken as the color of the outer wall pixel region.

[0169] After obtaining the color of each superpixel region and the proportion of each color in the wall pixel region, the color with the largest proportion is found, which can be used as the color of the outer wall pixel region.

[0170] By using the above steps S150-S180, the outer wall pixel region can be divided into several connected regions with similar color and texture by superpixel segmentation. The originally large number of pixel groups can be simplified into a smaller number of superpixel regions with more semantic information. Since the color inside the superpixel is usually consistent, there is no random fluctuation of color. Therefore, by calculating the average color or representative color of each superpixel region, a more accurate color can be obtained.

[0171] Considering that the architectural style and color of multiple buildings in the same building group are usually uniform, in order to improve the rendering efficiency of the building models corresponding to the multiple buildings belonging to the same building group in the electronic map, in some embodiments, if the target building is one of the buildings in the building group, after step S140, the method further includes: if it is determined that the target building identifier corresponds to the upper level building identifier based on the hierarchical relationship between the target building identifier and the different building identifiers, performing color rendering on the building model of the reference building in the electronic map according to the color of the outer wall of the target building, and the reference building refers to the other buildings in the building group indicated by the upper level building identifier corresponding to the target building identifier except the target building.

[0172] Since the target building is one of the buildings in the building group, the upper level building identifier of the target building identifier corresponds to the building group identifier of the building group where the target building is located, and the building group where the target building is located can be a community, an office park, a building cluster, etc. For example, if the target building identifier is "A community 1 building", the upper level building identifier of the target building identifier is "A community", and the reference building is the other buildings in "A community" except "A community 1 building"; if the target building identifier is "XX building C", the upper level building identifier of the target building identifier is "XX building", and the reference building is the other buildings in "XX building" except "XX building C".

[0173] In constructing the hierarchical relationship between different building identifiers, the different building identifiers can be divided according to the relationship between the corresponding geographic spaces, or can be divided according to the building name information. For example, if a building identifier H1 corresponds to a region h1 in the geographic space, and another building identifier H2 corresponds to a region h2 in the geographic space, and the region h1 includes the region h2, it can be determined that the building identifier H1 is the upper level building identifier of the building identifier H2. When dividing according to the building name information, if a building identifier is "A community", and another building identifier is "A community 1# building", it can be determined that "A community" corresponds to more buildings than "A community 1# building". At this time, it can be determined that the building identifier corresponding to "A community" is the upper level building identifier of the building identifier corresponding to "A community 1# building".

[0174] In the above embodiment, after determining the outer wall color of a building in a building group, the outer wall color of the building is determined as the outer wall color of all buildings in the building group. In this way, it is not necessary to extract the outer wall color of each building in the building group, and the calculation amount can be reduced. The outer wall color of one building in the building group can be directly reused to perform color rendering on the building models corresponding to all buildings in the electronic map, and the efficiency of color rendering can be improved.

[0175] As shown in Figure 13 , a rendered electronic map obtained by respectively performing color rendering on the building models corresponding to each building in the electronic map using the above method of the present application is shown. In the electronic map, as shown in Figure 13 , the colors of the building models corresponding to different buildings in the same building group are the same.

[0176] By using the hierarchical relationship between building identifiers, the color rendering of different buildings in the same building group on the electronic map can be quickly and effectively performed, which not only conforms to the logic of the real world (the same building group generally has consistent style and color), but also improves the quality of the electronic map and the user experience when using the electronic map.

[0177] In the map scenario, the building identifier in the database is often associated with a POI, which can be a location with a specific meaning or value, such as a store, a restaurant, a park, a museum, or an office building. The building is one of the important components of the POI, and the street view image stored in the database is usually associated with the information of the POI. Therefore, in the map scenario, the building identifier can be regarded as the information of the POI, and the street view image associated with the POI can be obtained. At this time, each building or building group can be regarded as a POI. When the target building is a POI in a building group, by searching for its association with the building identifier of the upper level, other POIs (i.e., reference buildings) of the building group to which the POI belongs can be inferred. In this way, based on the color of the outer wall of the target building, the color of the other buildings in the building group can be rendered, which can improve the rendering efficiency while maintaining the overall consistency and reducing the workload of individual processing.

[0178] For example, again taking the information of the POI associated with the building identifier of the target building as “A community No. 1 building” as an example, the process of color rendering of the building model corresponding to “A community No. 1 building” in the electronic map by using the method of the present application can be divided into six stages: POI data pulling, effective data judgment, building surface identification, color extraction, color clustering, and color attribute assignment to the building model.

[0179] POI data pulling stage: obtaining multiple candidate street view images associated with the POI from the database.

[0180] It should be noted that for a building in a community (the information of the POI corresponding to the identifier of the building is “A community No. 1 building”), the street view image obtained based on the information of the POI is usually less. Considering that the buildings in the community are usually of the same color, the street view image based on the information of the main node POI of the POI (such as “A community”) can be obtained, and the street view image is used as the multiple candidate street view images corresponding to the information of the POI (“A community No. 1 building”). When the POI is the target building, the main node POI of the POI is the building group to which the target building belongs, and the information of the main node POI is the identifier of the building group.

[0181] Effective data judgment stage: in order to use the obtained street view image to render the color of the building model corresponding to "A community No. 1 building" in the electronic map, it is necessary to select the street view image including the main building (including the outer wall) from the plurality of candidate street view images. In this embodiment, the main building recognition model can be used to determine whether the main building is included in each candidate street view image. The candidate street view image including the main building is determined as effective data, that is, it can be used for subsequent building recognition and color extraction. The candidate street view image not including the main building is determined as a picture not including the main building, which cannot provide effective information and is filtered out in this link.

[0182] Building face recognition stage: considering that the front face of the building is the most important and representative face of the building, in order to make the finally obtained color more reflect the color of the building in the real scene, after obtaining M street view images showing "A community No. 1 building" as the target building, the building face recognition model is used to recognize the building face of the street view image, and the building face recognition result is obtained. The building face recognition result includes the position information of the building face pixel region in the street view image and the building face type to which each building face pixel region belongs. The building face pixel region refers to the pixel region where the building face of "A community No. 1 building" is located. The building face type includes front face or side face. Then, based on the building face recognition result, the N target building face pixel regions with the largest area are selected from the building face pixel regions with the front face type, wherein the area of the target building face pixel region is larger than that of the building face pixel region not selected.

[0183] Color extraction stage: based on the target building face pixel region selected in the building face recognition stage, the outer wall color of the target building is extracted. In order to make the color extraction closer to the color of the building, the window in the target building face pixel region can be identified to determine the window pixel region in the target building face pixel region. The window pixel region in the target building face pixel region is masked, and the pixel region in the target building face pixel region not masked is used as the outer wall pixel region. Then, the color of the outer wall pixel region is extracted to ensure that the color of the building is extracted instead of the color of the window. When the color of the outer wall pixel region is extracted, the superpixel segmentation method can be used to determine at least one superpixel region in the outer wall pixel region, the color of each superpixel region is extracted, the color proportion of each superpixel region in the outer wall pixel region is determined based on the color of each superpixel region and the pixel proportion of each superpixel region in the outer wall pixel region, and the color with the largest color proportion in the outer wall pixel region is used as the color of the outer wall pixel region.

[0184] The color clustering stage can be implemented by the color extraction stage described above. For the "A community No. 1 building", N valid data have been obtained, and an outer wall color is extracted from each data. That is, the "A community No. 1 building" corresponds to N outer wall colors. Therefore, the step needs to cluster the N outer wall colors as the color of the final "A community No. 1 building" POI, excluding some interference caused by data quality, etc. This stage can perform hierarchical clustering on the N colors (with HSV channel as input), and finally obtain multiple categories. The category with the most colors is determined as the target category, and the average value of each color in the target category is the final result of the color clustering stage, that is, the color of the "A community No. 1 building" POI.

[0185] The color attribute assignment stage of the building model can obtain the color of the "A community No. 1 building" POI in the electronic map. Since the "A community No. 1 building" POI corresponds to a master node, which represents the "A community", and the colors of the buildings in the community are usually the same, after obtaining the color of the "A community No. 1 building" POI, the color can be used as the color of the buildings corresponding to all the sub-nodes connected to the master node (A community).

[0186] As shown in Figure 14 , considering that the color attribute assignment stage of the building model and the previous stages (POI data pulling, valid data judgment, building surface recognition, color extraction, and color clustering stage) can be executed on different devices, the colors corresponding to the information of multiple POIs are obtained, and the information of the multiple POIs and the corresponding colors are associated and stored in the database. When rendering the building models corresponding to different buildings in the electronic map, the information of the POI corresponding to each building identifier can be obtained, and it is determined whether the obtained POI information corresponds to a color based on the information of the POI corresponding to each building identifier and the associated storage of the information of the POI and the color. If it corresponds to a color, the building identifier corresponding to the POI and the corresponding color are associated and stored, so that the building model corresponding to the building identifier in the electronic map is rendered based on the color corresponding to the building identifier in the subsequent stage. If it does not correspond to a color, it is determined whether the POI without the corresponding color has a master node POI. If the POI corresponds to a master node POI, it is determined whether the master node POI corresponds to a color based on the data stored in the database. If it does, the color corresponding to the master node POI is used as the color of the POI without the corresponding color.

[0187] Thus, the color corresponding to each building identifier is obtained, and the color of each building model in the electronic map is rendered based on the color corresponding to each building identifier.

[0188] The following continues to describe an exemplary structure of the color rendering device 255 provided in the embodiment of the present application implemented as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the color rendering device 255 of the memory 250 may include: an image acquisition module 2551 , a pixel area acquisition module 2552 , a color determination module 2553 and a color rendering module 2554 .

[0189] The image acquisition module 2551 is used to acquire a street view image showing a target building; the pixel area acquisition module 2552 is used to acquire the exterior wall pixel area of ​​the target building in the street view image; the color determination module 2553 is used to determine the exterior wall color of the target building using a color that is not based on the exterior wall pixel area; and the color rendering module 2554 is used to color-render the building model of the target building in the electronic map according to the exterior wall color of the target building.

[0190] In one embodiment, the image acquisition module 2551 includes an image acquisition unit, a main building recognition unit, and an image selection unit. The image acquisition unit is configured to acquire multiple candidate street view images associated with a target building identifier of a target building; the main building recognition unit is configured to use a main building recognition model to perform main building recognition on each candidate street view image to obtain a main building recognition result for each candidate street view image, the main building recognition result being used to indicate whether the main building of the target building is present in the candidate street view image. The main building recognition model is trained based on multiple sample street view images and label information of each sample street view image, the label information being used to indicate whether the sample street view image presents the main building of the corresponding building; and the image selection unit is configured to select, from the multiple candidate street view images, a candidate street view image that presents the main building of the target building, based on the main building recognition results of each candidate street view image, as the street view image presenting the target building.

[0191] In one embodiment, the pixel region acquisition module 2552 includes a building face recognition unit, a first region determination unit, and a second region determination unit. The building face recognition unit is configured to perform building face recognition on a street view image using a building face recognition model to obtain a building face recognition result, the building face recognition result including location information of building face pixel regions in the street view image and the building face type to which each building face pixel region belongs. The building face pixel region refers to the pixel region where the building face of the target building is located. The first region determination unit is configured to determine, based on the building face recognition result, at least one target building face pixel region from among the building face pixel regions with a front building face type. The second region determination unit is configured to determine an exterior wall pixel region within the at least one target building face pixel region.

[0192] In an implementation, the second region determining unit is further configured to perform window identification on each target building surface pixel region, determine a window pixel region in the target building surface pixel region; perform mask processing on the window pixel region in the target building surface pixel region, and take a pixel region in the target building surface pixel region that is not masked as an outer wall pixel region.

[0193] In an implementation, the first region determining unit is further configured to determine, based on the building surface identification results of the plurality of street view images, N building surface pixel regions with a front building surface type from the building surface pixel regions in the plurality of street view images, N being a positive integer; determine pixel areas of each building surface pixel region with the front building surface type; select, based on the pixel areas, K building surface pixel regions with the largest pixel areas from the N building surface pixel regions with the front building surface type as target building surface pixel regions, K being a positive integer and K not greater than N.

[0194] In an implementation, the color rendering device 255 further includes a data obtaining module, a building surface identification module, an identification loss determining module, and an identification model training module. The data obtaining module is configured to obtain a plurality of building images and label information of each building image, the label information being used to indicate label position information of each building surface pixel region in the building image and a building surface type to which each building surface pixel region belongs. The building surface identification module is configured to perform building surface identification on the building images by using a building surface identification model to obtain a reference building surface identification result, the reference building surface identification result indicating predicted position information of each building surface pixel region in the building image and a predicted building surface type to which each building surface pixel region belongs. The identification loss determining module is configured to determine a building surface identification loss based on the label position information of each building surface pixel region in the building image and the building surface type to which each building surface pixel region belongs, and the predicted position information of each building surface pixel region in the building image and the predicted building surface type to which each building surface pixel region belongs. The identification model training module is configured to adjust parameters of the building surface identification model based on the building surface identification loss.

[0195] In an implementation, the target building has a plurality of outer wall pixel regions, and the color determining module 2553 includes a clustering unit, a statistical unit, a category determining unit, and a color determining unit. The clustering unit is configured to cluster colors of the plurality of outer wall pixel regions to determine a clustering result, the clustering result indicating categories to which the colors of the outer wall pixel regions belong. The statistical unit is configured to count the number of colors belonging to each category based on the clustering result. The category determining unit is configured to determine a target category as a category with the largest number of colors. The color determining unit is configured to determine an outer wall color of the target building based on the colors belonging to the target category in the clustering result.

[0196] In an implementation, the apparatus 255 further includes a segmentation module and a proportion determination module. The segmentation module is configured to perform superpixel segmentation on the outer wall pixel region to determine at least one superpixel region in the outer wall pixel region. The color determination module is further configured to perform color extraction on each superpixel region to determine a color of each superpixel region. The proportion determination module is configured to determine a color proportion of each color in the outer wall pixel region based on the color of each superpixel region and a pixel proportion of each superpixel region in the outer wall pixel region. The color determination module is further configured to determine a color with the largest color proportion in the outer wall pixel region as the color of the outer wall pixel region.

[0197] In an implementation, the color rendering module 2554 is further configured to perform color rendering on a building model of a reference building in the electronic map according to the outer wall color of the target building when it is determined that the target building identification corresponds to a higher-level building identification based on the hierarchical relationship between the target building identification and different building identifications, and the reference building refers to a building other than the target building in a building group indicated by the higher-level building identification corresponding to the target building identification.

[0198] The embodiment of the present application provides a computer program product or a computer program, which includes executable instructions stored in a computer readable storage medium. The processor of the electronic device reads the executable instructions from the computer readable storage medium, and the processor executes the executable instructions, so that the electronic device executes the artificial intelligence-based text-to-image model training method and the artificial intelligence-based image generation method provided in the embodiment of the present application.

[0199] The embodiment of the present application provides a computer readable storage medium storing executable instructions, wherein the executable instructions are stored in the computer readable storage medium. When the executable instructions are executed by the processor, the processor will execute the foregoing method steps provided by the embodiment of the present application.

[0200] In some embodiments, the computer readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disc, or CD-ROM; or various devices including one or any combination of the above memories.

[0201] In some embodiments, the executable instructions can be in the form of programs, software, software modules, scripts or codes, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and can be deployed in any form, including being deployed as independent programs or being deployed as modules, components, subroutines or other units suitable for use in a computing environment.

[0202] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinating files (e.g., files storing one or more modules, subroutines, or code portions).

[0203] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0204] The above are merely examples of the present application and are not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are included in the scope of protection of the present application.

Claims

1. A color rendering method characterized by, The method comprises the following steps: obtaining a street view image showing a target building; obtaining an outer wall pixel region in the street view image, wherein the outer wall pixel region is located at an outer wall of the target building; determining an outer wall color of the target building based on a color of the outer wall pixel region; performing color rendering on a building model of the target building in an electronic map according to the outer wall color of the target building.

2. The method of claim 1, wherein, The step of obtaining the outer wall pixel region in the street view image comprises the following steps: performing building surface identification on the street view image by using a building surface identification model to obtain a building surface identification result, wherein the building surface identification result comprises position information of a building surface pixel region in the street view image and a building surface type to which each building surface pixel region belongs, and the building surface pixel region refers to a pixel region where a building surface of the target building is located; determining at least one target building surface pixel region from building surface pixel regions with a front building surface type based on the building surface identification result; determining the outer wall pixel region in the at least one target building surface pixel region.

3. The method of claim 2, wherein, The step of determining the outer wall pixel region in the at least one target building surface pixel region comprises the following steps: performing window identification on each target building surface pixel region to determine a window pixel region in the target building surface pixel region; performing mask processing on the window pixel region in the target building surface pixel region, and taking a pixel region in the target building surface pixel region that is not masked as the outer wall pixel region.

4. The method of claim 2, wherein, The street view image is a plurality of street view images, and the step of determining at least one target building surface pixel region from building surface pixel regions with a front building surface type based on the building surface identification result comprises the following steps: determining N building surface pixel regions with a front building surface type from building surface pixel regions in the plurality of street view images based on building surface identification results of the plurality of street view images, wherein N is a positive integer; determining a pixel area of each building surface pixel region with a front building surface type; selecting K building surface pixel regions with the largest pixel area from the N building surface pixel regions with a front building surface type as the target building surface pixel region based on the pixel area, wherein K is a positive integer, and K is not greater than N.

5. The method of claim 2, wherein, Before the step of performing building surface identification on the street view image by using a building surface identification model to obtain a building surface identification result, the method further comprises the following steps: obtaining a plurality of building images and label information of each building image, wherein the label information is used to indicate label position information of each building surface pixel region in the building image and a building surface type to which each building surface pixel region belongs; performing building surface identification on the building image by using a building surface identification model to obtain a reference building surface identification result, wherein the reference building surface identification result indicates predicted position information of each building surface pixel region in the building image and a predicted building surface type to which each building surface pixel region belongs; According to the label position information of the building surface pixel region in the building image and the building surface type to which each building surface pixel region belongs, and the predicted position information of each building surface pixel region in the building image and the predicted building surface type to which each building surface pixel region belongs, a building surface recognition loss is determined; According to the building surface recognition loss, the parameters of the building surface recognition model are adjusted until a training end condition is reached.

6. The method according to any one of claims 1 to 5, characterized in that, The outer wall pixel region is multiple; the outer wall body color of the target building is determined based on the color of the outer wall pixel region, including: The colors of the plurality of outer wall pixel regions are clustered to determine a clustering result, the clustering result indicating the categories to which the colors of the outer wall pixel regions belong; Based on the clustering result, the number of colors belonging to each category is counted; The category with the most number of colors is determined as the target category; According to the colors belonging to the target category in the clustering result, the outer wall body color of the target building is determined.

7. The method according to any one of claims 1 to 5, characterized in that, Before the outer wall body color of the target building is determined based on the color of the outer wall pixel region, the method further includes: The outer wall pixel region is super-pixel segmented to determine at least one super-pixel region in the outer wall pixel region; The color of each super-pixel region is extracted to determine the color of each super-pixel region; Based on the color of each super-pixel region and the pixel proportion of each super-pixel region in the outer wall pixel region, the color proportion of each color in the outer wall pixel region is determined; The color with the most color proportion in the outer wall pixel region is taken as the color of the outer wall pixel region.

8. The method according to any one of claims 1-5, characterized in that, After the outer wall body color of the target building is determined based on the color of the outer wall pixel region, the method further includes: If it is determined based on the hierarchical relationship between the target building identifier of the target building and different building identifiers that the target building identifier corresponds to a higher level building identifier, the building model of a reference building in the electronic map is rendered in color according to the outer wall body color of the target building, and the reference building refers to other buildings in the building group indicated by the higher level building identifier corresponding to the target building identifier, except the target building.

9. The method according to any one of claims 1-5, characterized in that, The method of obtaining the street view image presenting the target building includes: Obtaining a plurality of candidate street view images associated with the target building identifier of the target building; Using a main building recognition model to perform main building recognition on each candidate street view image to obtain a main building recognition result of each candidate street view image, the main building recognition result being used to indicate whether the main building of the target building is presented in the candidate street view image; Based on the main building recognition result of each candidate street view image, a candidate street view image presenting the main building of the target building is selected from the plurality of candidate street view images as the street view image presenting the target building.

10. A color rendering apparatus, characterized by comprising: The device includes: An image acquisition module for obtaining a street view image presenting a target building; A pixel region acquisition module for acquiring an outer wall pixel region where an outer wall of the target building is located in the street view image; The color determining module is configured to determine the color of the outer wall of the target building without using the color of the pixel region of the outer wall; The color rendering module is configured to perform color rendering on the building model of the target building in the electronic map according to the color of the outer wall of the target building.

11. An electronic device, comprising: The computer program product comprises: one or more processors; a memory; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to perform the method according to any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer readable storage medium stores program code, and the program code can be called and executed by the processor to perform the method according to any one of claims 1-9.

13. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instruction is executed by the processor to implement the steps of the method according to any one of claims 1-9.

Images