Model texture updating method, device and computer equipment

Abstract

The present application relates to a model texture update method, device, computer equipment and storage medium. The method comprises: obtaining a real scene image to be processed and real scene acquisition parameters corresponding to the real scene image to be processed; performing coordinate correction on the real scene image to be processed according to the real scene acquisition parameters to obtain the target camera posture; obtaining a texture update detection area according to the target camera posture, the historical real scene map model and the historical building texture map; when there is a texture update in the texture update detection area, extracting an updated texture image from the real scene image to be processed; performing texture remapping on the updated texture image to update the historical real scene map model. The use of this method can make the surface texture of the updated historical real scene map model clear and improve the quality of the real scene map model.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a model texture updating method, device and computer equipment. Background Art

[0002] With the development of computer technology, the technology of reconstructing three-dimensional models using oblique photogrammetry has emerged. The three-dimensional models reconstructed using oblique photogrammetry have played an important role in the construction of basic geographic information of smart cities, urban information models and real-life three-dimensional maps.

[0003] In traditional technology, the method of reconstructing three-dimensional models by oblique photogrammetry is to obtain high-resolution image information from different directions through multi-view cameras, and then combine technologies in the fields of photogrammetry, computational geometry, computer vision, etc. to automatically reconstruct the three-dimensional grid model of the shooting area.

[0004] However, when updating the real-life 3D map model, the method of reconstructing the 3D model by oblique photogrammetry cannot obtain high-precision images in some aerial survey areas due to angle, distance and other reasons, which will cause the surface texture of the reconstructed real-life 3D map model to be blurred and unrecognizable, resulting in the problem of poor quality of the real-life 3D map model. Summary of the invention

[0005] Based on this, it is necessary to provide a model texture updating method, device, computer equipment and storage medium that can improve the quality of real-life map models in response to the above technical problems.

[0006] A model texture updating method, the method comprising:

[0007] Acquire a real scene image to be processed and real scene acquisition parameters corresponding to the real scene image to be processed;

[0008] According to the real scene acquisition parameters, coordinate correction is performed on the real scene image to be processed to obtain the target camera pose;

[0009] According to the target camera position, the historical real scene map model and the historical building shading map, the shading update detection area is obtained;

[0010] When there is a shading update in the shading update detection area, an updated shading image is extracted from the real scene image to be processed;

[0011] Texture remapping is performed on the updated shading image to update the historical real-life map model.

[0012] A model texture updating device, the device comprising:

[0013] An acquisition module, used to acquire a real scene image to be processed and real scene acquisition parameters corresponding to the real scene image to be processed;

[0014] A correction module is used to perform coordinate correction on the real scene image to be processed according to the real scene acquisition parameters to obtain the target camera pose;

[0015] A processing module is used to obtain a texture update detection area according to the target camera posture, the historical real scene map model and the historical building texture map;

[0016] An extraction module, used for extracting an updated shading image from the real scene image to be processed when there is a shading update in the shading update detection area;

[0017] The mapping module is used to remap the texture of the updated background image and update the historical real-life map model.

[0018] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0019] Acquire a real scene image to be processed and real scene acquisition parameters corresponding to the real scene image to be processed;

[0020] According to the real scene acquisition parameters, coordinate correction is performed on the real scene image to be processed to obtain the target camera pose;

[0021] According to the target camera position, the historical real scene map model and the historical building shading map, the shading update detection area is obtained;

[0022] When there is a shading update in the shading update detection area, an updated shading image is extracted from the real scene image to be processed;

[0023] Texture remapping is performed on the updated shading image to update the historical real-life map model.

[0024] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0025] Acquire a real scene image to be processed and real scene acquisition parameters corresponding to the real scene image to be processed;

[0026] According to the real scene acquisition parameters, coordinate correction is performed on the real scene image to be processed to obtain the target camera pose;

[0027] According to the target camera position, the historical real scene map model and the historical building shading map, the shading update detection area is obtained;

[0028] When there is a shading update in the shading update detection area, an updated shading image is extracted from the real scene image to be processed;

[0029] Texture remapping is performed on the updated shading image to update the historical real-life map model.

[0030] The above-mentioned model texture updating method, device, computer equipment and storage medium can determine the target camera posture by acquiring the real scene image to be processed and performing coordinate correction on the real scene image to be processed according to the real scene acquisition parameters, so as to obtain the texture update detection area according to the target camera posture, the historical real scene map model and the historical building texture map. When there is a texture update in the texture update detection area, the historical real scene map model can be updated by extracting the updated texture image from the real scene image to be processed and remapping the texture of the updated texture image. In the whole process, the historical real scene map model is updated according to the updated texture image by extracting the updated texture image with clear texture by using the real scene image to be processed, so that the surface texture of the updated historical real scene map model can be made clear, thereby improving the quality of the real scene map model. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A diagram showing an application environment of a model texture updating method in an embodiment;

[0032] Figure 2 A schematic diagram of a flow chart of a model texture updating method in one embodiment;

[0033] Figure 3 is a flow chart of a model texture updating method in another embodiment;

[0034] Figure 4 is a flow chart of a model texture updating method in yet another embodiment;

[0035] Figure 5 is a structural block diagram of a model texture updating device in one embodiment;

[0036] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0038] The model texture updating method provided in this application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The user uses the terminal 102 to collect real-scene data and uploads it to the server 104. The server 104 obtains the real-scene image to be processed and the real-scene acquisition parameters corresponding to the real-scene image to be processed according to the real-scene data, and performs coordinate correction on the real-scene image to be processed according to the real-scene acquisition parameters to obtain the target camera posture. According to the target camera posture, the historical real-scene map model and the historical building shading map, the shading update detection area is obtained. When there is a shading update in the shading update detection area, the updated shading image is extracted from the real-scene image to be processed, and the updated shading image is texture remapped to update the historical real-scene map model. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers and portable wearable devices, and the server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0039] In one embodiment, Figure 2 As shown, a model texture updating method is provided, which is applied to Figure 1 The server in the example is used to illustrate the following steps:

[0040] Step 202: Acquire a real scene image to be processed and real scene acquisition parameters corresponding to the real scene image to be processed.

[0041] Among them, the real-scene image to be processed refers to the real-scene data after data cleaning and image preprocessing, and the real-scene data refers to the data collected by the user through the terminal. For example, the real-scene data can specifically refer to the real-scene image, real-scene video, etc. collected by the user through the terminal. The terminal can specifically be a mobile phone, a handheld camera, a video camera, etc., and this embodiment is not specifically limited here. The real-scene acquisition parameters refer to the parameters when collecting the real-scene image to be processed, including acquisition device parameters, acquisition coordinates, and acquisition time, etc., wherein the acquisition coordinates can specifically refer to geographic coordinates and acquisition device posture. For example, geographic coordinates can specifically refer to GPS (Global Positioning System) coordinates.

[0042] Specifically, the server will first obtain the real-scene data collected and uploaded by the user through the terminal, and after data cleaning and image preprocessing of the real-scene data, obtain the real-scene image to be processed. Among them, when the real-scene data is a real-scene image, it can be processed directly. When the real-scene data is a real-scene video, it is necessary to extract frames of the real-scene video, obtain the video frames corresponding to the real-scene video, and then process the video frames. Among them, data cleaning refers to removing images with poor shooting quality. The poor shooting quality here includes various situations such as blur, overexposure, black screen, occlusion, and GPS no signal. This embodiment does not make specific limitations here. Image preprocessing refers to image enhancement, grayscale image generation, multi-level scale image, etc. of the image after data cleaning to facilitate subsequent processing. This embodiment does not make specific limitations on the specific methods of data cleaning and image preprocessing here.

[0043] Step 204 , coordinate correction is performed on the real scene image to be processed according to the real scene acquisition parameters to obtain the target camera pose.

[0044] Among them, coordinate correction refers to the correction and calculation of the acquisition coordinates of each image at the time of acquisition based on the image features, computer vision, least squares principle, optimization principle, etc. of the real scene image to be processed, so that it is closest to the true value at the time of acquisition. The acquisition coordinates include geographic coordinates and acquisition device posture. The target camera posture refers to the corrected camera posture.

[0045] Specifically, the terminal will use the collection coordinates in the real-scene collection parameters to perform collection simulation in the historical real-scene map model to obtain a simulated observation image, and then perform feature matching on the simulated observation image and the real-scene image to be processed to correct the collection coordinates when collecting the real-scene image to be processed and obtain the target camera pose.

[0046] Step 206, obtaining a texture update detection area according to the target camera position, the historical real scene map model and the historical building texture map.

[0047] Among them, the historical real-life map model refers to a constructed real-life map model. For example, the historical real-life map model may specifically refer to a constructed real-life three-dimensional map. The historical building shading map refers to a shading map constructed based on the historical real-life map model, which is consistent with the building model in the historical real-life map model in terms of spatial position, size, attributes, etc. The historical building shading map includes the geometric shape, spatial position and corresponding pictures of the bottom texture. For example, the shading here may specifically refer to the bottom merchant, and the historical real-life map model and the historical building shading map may be pre-constructed manually. The shading update detection area refers to the area corresponding to the real-life image to be processed where a shading update may exist, that is, the shading area actually referred to by the real-life image to be processed. Further shading update detection is required for this area to confirm whether there is a shading update.

[0048] Specifically, the server can restore the collection scene according to the target camera posture and obtain the real-scene collection space range. By comparing the historical real-scene map model and the historical building background map according to the real-scene collection space range, the corresponding background update detection area can be obtained.

[0049] Step 208: When there is a texture update in the texture update detection area, an updated texture image is extracted from the real scene image to be processed.

[0050] Specifically, after obtaining the texture update detection area, the server will further determine whether there is a texture update in the texture update detection area. When there is a texture update in the texture update detection area, the server will extract an updated texture image from the real scene image to be processed and implement the texture update according to the updated texture image.

[0051] Step 210: remap the texture of the updated background image to update the historical real-scene map model.

[0052] Among them, texture remapping refers to mapping the updated texture image onto the historical real-life map model to achieve the update of the historical real-life map model.

[0053] Specifically, the server will determine the spatial position and shape features of the shading area corresponding to the updated shading image based on the historical real-life map model and the historical building shading map, calculate the origin of the rendering coordinate system and the projection transformation matrix based on the spatial position and shape features, construct the rendering space, and use the updated shading image as texture data and the vertex projection coordinates of the historical real-life map model as UV coordinates to render the updated shading image for updating, merge the updated texture image with the original texture through a preset algorithm, obtain a new texture image, and update the historical real-life map model. For example, the preset algorithm can be a greedy algorithm.

[0054] The above-mentioned model texture updating method can determine the target camera posture by obtaining the real scene image to be processed and performing coordinate correction on the real scene image to be processed according to the real scene acquisition parameters, so as to obtain the texture update detection area according to the target camera posture, the historical real scene map model and the historical building texture map. When there is a texture update in the texture update detection area, the historical real scene map model can be updated by extracting the updated texture image from the real scene image to be processed and remapping the texture of the updated texture image. In the whole process, the updated texture image with clear texture is extracted by using the real scene image to be processed, and the historical real scene map model is updated according to the updated texture image, so that the surface texture of the updated historical real scene map model can be made clear, thereby improving the quality of the real scene map model.

[0055] In one embodiment, coordinate correction is performed on the real scene image to be processed according to the real scene acquisition parameters to obtain the target camera pose, including:

[0056] According to the real scene acquisition parameters and the historical real scene map model, a simulated observation image is obtained;

[0057] Perform feature matching on the simulated observation image and the real scene image to be processed to obtain feature points of the same name;

[0058] The reprojection error is determined based on the characteristic same-name points to obtain the target camera pose.

[0059] Among them, the simulated observation image refers to the image obtained by performing acquisition simulation in the historical real scene map model according to the real scene acquisition parameters. The acquisition simulation is mainly simulated based on the acquisition coordinates in the real scene acquisition parameters. Feature homonymous points refer to multiple pairs of similar feature points that match each other in the simulated observation image and the real scene image to be processed. The reprojection error refers to the error when projecting the second feature point in the real scene image to be processed to the first feature point in the simulated observation image.

[0060] Specifically, the server will determine the geographic coordinates and pose of the acquisition device according to the acquisition coordinates in the real scene acquisition parameters, perform acquisition simulation on the historical real scene map model according to the geographic coordinates and the pose of the acquisition device, obtain a simulated observation image, perform feature matching on the simulated observation image and the real scene image to be processed, determine multiple pairs of similar feature points of the simulated observation image and the real scene image to be processed, obtain feature homonymous points, obtain a coordinate transformation matrix according to multiple pairs of similar feature points in the feature homonymous points, obtain a reprojection error according to the coordinate transformation matrix, adjust the coordinate transformation matrix according to the reprojection error, and when the reprojection error meets the preset conditions, obtain the target camera pose according to the latest coordinate transformation matrix and the acquisition coordinates. The preset condition may specifically be that the reprojection error converges.

[0061] In this embodiment, by performing acquisition simulation according to the real-scene acquisition parameters and the historical real-scene map model, a simulated observation image is obtained, and feature matching is performed on the simulated observation image and the real-scene image to be processed to obtain feature same-name points. The reprojection error is determined based on the feature same-name points to obtain the target camera pose, thereby realizing the determination of the target camera pose.

[0062] In one embodiment, feature matching is performed on the simulated observation image and the real scene image to be processed to obtain feature points of the same name, including:

[0063] Extracting features from the simulated observation image and the real scene image to be processed respectively to obtain a first feature point set of the simulated observation image and a second feature point set of the real scene image to be processed;

[0064] Performing a similarity comparison on feature points in the first feature point set and the second feature point set to determine multiple pairs of similar feature points;

[0065] Based on multiple pairs of similar feature points, feature points with the same name are obtained.

[0066] Specifically, the server will first perform feature extraction on the simulated observation image and the real scene image to be processed respectively, to obtain a first feature point set of the simulated observation image and a second feature point set of the real scene image to be processed, and then perform a similarity comparison on the feature points in the first feature point set and the second feature point set to determine multiple pairs of similar feature points, and use the multiple pairs of similar feature points as feature homonymous points.

[0067] Furthermore, the feature matching in this embodiment may specifically refer to SIFT (Scale-invariant feature transform, SIFT, scale-invariant feature transform) feature matching, and the obtained feature homonymous points are SIFT feature homonymous points. The server will first perform SIFT feature extraction on the simulated observation image and the real scene image to be processed respectively, and obtain the first feature point set of the simulated observation image and the second feature point set of the real scene image to be processed. The feature points in the first feature point set and the second feature point set are both represented by feature vectors, and then the feature points in the first feature point set and the second feature point set are compared for similarity based on the feature vectors to determine multiple pairs of similar feature points, and use the multiple pairs of similar feature points as feature homonymous points. Among them, performing similarity comparison on feature points based on feature vectors refers to calculating the vector distance between feature points based on feature vectors, and using feature points whose vector distance is less than a preset distance threshold as similar feature points. The vector distance may specifically refer to the cosine distance, and the preset distance threshold can be set as needed.

[0068] In this embodiment, by performing feature extraction on the simulated observation image and the real scene image to be processed respectively, a feature point set is obtained, and a similarity comparison is performed on the feature point set to determine multiple pairs of similar feature points. Based on the multiple pairs of similar feature points, feature homonymous points are obtained, thereby achieving the acquisition of feature homonymous points.

[0069] In one embodiment, determining the reprojection error according to the characteristic same-name points to obtain the target camera pose includes:

[0070] According to multiple pairs of similar feature points among the feature points with the same name, a coordinate transformation matrix is ​​obtained;

[0071] Obtaining a coordinate transformation feature point according to the coordinate transformation matrix and the second feature point among the similar feature points;

[0072] Compare the coordinate transformation feature point with the first feature point in the similar feature points to obtain the reprojection error, and adjust the coordinate transformation matrix;

[0073] Return to the step of obtaining the coordinate transformation feature point according to the coordinate transformation matrix and the second feature point in the similar feature points, until the reprojection error converges, and obtain the target camera pose according to the latest coordinate transformation matrix.

[0074] The coordinate transformation matrix refers to a matrix for transforming feature points in the simulated observation image and the real scene image to be processed, that is, transforming feature points in the real scene image to be processed into feature points in the simulated observation image.

[0075] Specifically, the server will first obtain a coordinate transformation matrix that can realize the coordinate transformation of the feature points based on multiple pairs of similar feature points in the feature points of the same name, and then perform coordinate transformation on the second feature point in the similar feature points according to the coordinate transformation matrix to obtain the corresponding coordinate transformation feature point, compare the coordinate transformation feature point with the first feature point in the similar feature points, obtain the reprojection error according to the coordinate error between the coordinate transformation feature point and the first feature point, and adjust the coordinate transformation matrix, and return to the step of obtaining the coordinate transformation feature point according to the coordinate transformation matrix and the second feature point in the similar feature points, until the reprojection error converges, and obtain the target camera pose according to the latest coordinate transformation matrix and the collected coordinates. Among them, adjusting the coordinate transformation matrix refers to making slight adjustments to the matrix parameters in the coordinate transformation matrix so that the reprojection error is continuously reduced to convergence. The specific adjustment method is not specifically limited in this embodiment.

[0076] It should be noted that there are deviations in the acquisition coordinates corresponding to the real scene image to be processed. Therefore, it is necessary to correct the acquisition coordinates through coordinate correction to determine the target camera pose corresponding to the acquisition coordinates, so as to determine the accurate real scene acquisition space range according to the target camera pose.

[0077] In this embodiment, a coordinate transformation matrix is ​​obtained based on multiple pairs of similar feature points, coordinate transformation feature points are obtained based on the coordinate transformation matrix and the second feature points, the coordinate transformation feature points are compared with the first feature points to obtain the reprojection error, and the coordinate transformation matrix is ​​adjusted. The step of obtaining the coordinate transformation feature points based on the coordinate transformation matrix and the second feature points among the similar feature points is returned until the reprojection error converges, and the target camera pose is obtained based on the latest coordinate transformation matrix, thereby achieving the acquisition of the target camera pose.

[0078] In one embodiment, according to the target camera position, the historical real scene map model and the historical building shading map, obtaining the shading update detection area includes:

[0079] According to the target camera position, the real scene acquisition space range is obtained;

[0080] According to the spatial scope of real-scene acquisition, the historical real-scene map model and the historical building texture map are compared to obtain the texture update detection area.

[0081] The real scene acquisition space range refers to the actual space range when acquiring the real scene determined according to the target camera posture.

[0082] Specifically, the server will perform acquisition simulation according to the target camera pose to obtain the real scene acquisition space range, and compare the historical real scene map model and the historical building texture map according to the real scene acquisition space range to obtain the corresponding texture update detection area. Among them, by comparing the historical real scene map model according to the real scene acquisition space range, the corresponding real scene map area can be obtained, and then the real scene map area can be compared with the historical building texture map to obtain the corresponding texture update detection area.

[0083] In this embodiment, by obtaining the real scene acquisition space range according to the target camera posture, and comparing the historical real scene map model and the historical building background map according to the real scene acquisition space range, the background update detection area is obtained, thereby realizing the determination of the background update detection area.

[0084] In one embodiment, after obtaining the texture update detection area according to the target camera position, the historical real scene map model and the historical building texture map, the method further includes:

[0085] Acquire the texture feature data corresponding to the texture update detection area and the texture feature data of the adjacent area;

[0086] Performing image feature matching on the real scene image to be processed according to the shading feature data and the shading feature data of the adjacent area;

[0087] When the image features do not match, it is determined that there is a texture update in the texture update detection area.

[0088] Among them, the shading feature data refers to the data used to characterize the characteristics of the shading update detection area, including the collection time, house number address, spatial position, geometric shape, shading character features and shading image features, etc. The adjacent area shading feature data refers to the shading feature data of the adjacent area corresponding to the shading update detection area. For example, when the shading update detection area is the bottom merchant area, the adjacent area can specifically refer to the adjacent bottom merchant area.

[0089] Specifically, the preset texture feature library stores the texture feature data of all texture areas in the historical building texture map. After obtaining the texture update detection area, the server will obtain the corresponding texture feature data and the adjacent area texture feature data from the preset texture feature library according to the texture update detection area. After obtaining the texture feature data and the adjacent area texture feature data, the server will perform image feature matching on the real scene image to be processed according to the texture feature data and the adjacent area texture feature data. When the image features do not match, it is determined that there is a texture update in the texture update detection area. Among them, the real scene image to be processed includes a target image area corresponding to the texture update detection area and an adjacent image area other than the target image area. When performing image feature matching, the server will match the target image area with the texture feature data, and match the adjacent image area with the adjacent area texture feature data.

[0090] Among them, when the adjacent image area matches the shading image features in the shading feature data of the adjacent area, and the target image area does not match the shading feature data, it is determined that the image features do not match, and there is a shading update in the shading update detection area. When the adjacent image area does not match the shading image features in the shading feature data of the adjacent area, it indicates that there may be a positioning error. At this time, no subsequent processing will be performed, and the step of obtaining the real scene image to be processed will be returned. When the adjacent image area matches the shading image features in the shading feature data of the adjacent area, and the target image area matches the shading feature data, it is determined that the image features match, and there is no shading update in the shading update detection area.

[0091] For example, the shading may specifically refer to the bottom merchant. When the shading update detection area represents the bottom merchant A, and the bottom merchant A has adjacent merchants B and C, there will be a target image area corresponding to the bottom merchant A and adjacent image areas corresponding to merchants B and C in the real scene image to be processed. When it is necessary to determine whether there is a shading update in the shading update detection area, the server will first compare the shading image features in the adjacent image area with the corresponding adjacent area shading feature data to determine whether the current positioning is accurate. When the adjacent image area matches the shading image features in the corresponding adjacent area shading feature data, it indicates that the positioning is accurate. The server will further compare the target image area and the shading feature data. When the target image area does not match the shading feature data, it indicates that there is a change in the bottom merchant A here, that is, there is a shading update in the shading update detection area. When the target image area matches the shading feature data, it indicates that there is no change in the bottom merchant A here, that is, there is no shading update in the shading update detection area.

[0092] In this embodiment, by performing image feature matching based on the shading feature data and the shading feature data of the adjacent region, it is possible to determine whether there is a shading update in the shading update detection region based on the matching result.

[0093] In one embodiment, performing image feature matching on the real scene image to be processed according to the shading feature data and the shading feature data of the adjacent region includes:

[0094] Extract characters from the real scene image to be processed to obtain the features of the characters to be compared;

[0095] Compare the shading character features and the character features to be compared in the shading feature data.

[0096] The character features to be compared refer to the characters extracted from the real scene image to be processed.

[0097] Specifically, when performing image feature matching, the server will match the real scene image to be processed according to the shading image features and shading character features in the shading feature data. Wherein, when matching the real scene image to be processed according to the shading character features, the server will extract characters from the real scene image to be processed, obtain the character features to be compared, compare the shading character features and the character features to be compared, and when the shading character features and the character features to be compared match, it indicates that there is no shading update in the shading update detection area, and return to the step of obtaining the real scene image to be processed. When the shading character features and the character features to be compared do not match, it indicates that there may be shading updates in the shading update detection area, and the server needs to further combine the results of image feature matching of the real scene image to be processed according to the shading feature data of the adjacent area to further determine whether there is a shading update in the shading update detection area. Wherein, the character feature matching method can be based on simhash (local sensitive hash) coding comparison, etc., and this embodiment is not specifically limited here.

[0098] In this embodiment, image feature matching can be performed on the real scene image to be processed by performing character feature matching.

[0099] In one embodiment, when there is a shading update in the shading update detection area, extracting an updated shading image from the real scene image to be processed includes:

[0100] When there is a shading update in the shading update detection area, obtaining shading feature data corresponding to the shading update detection area;

[0101] According to the shading feature data and the target camera position, determine the effective position of the updated shading image in the real scene image to be processed;

[0102] An updated shading image is extracted from the real scene image to be processed according to the effective position.

[0103] The effective position refers to the pixel coordinates of the updated background image in the real scene image to be processed.

[0104] Specifically, when there is a background update in the background update detection area, the server will obtain the background feature data corresponding to the background update detection area from the preset background feature library, and estimate the effective position of the updated background image in the real scene image to be processed based on the spatial position, geometric size and other information in the background feature data and the target camera pose. Based on the effective position, the updated background image can be extracted from the real scene image to be processed.

[0105] In this embodiment, by acquiring the background feature data corresponding to the background update detection area, determining the effective position of the updated background image in the real scene image to be processed according to the background feature data and the target camera posture, and extracting the updated background image from the real scene image to be processed according to the effective position, the extraction of the updated background image can be achieved.

[0106] In one embodiment, Figure 3 As shown, a flow chart is provided to illustrate the model texture updating method of the present application, and the method comprises the following steps:

[0107] S302: base map construction, i.e., manually constructing a three-dimensional map (i.e., a historical real scene map model) and a shading map (i.e., a historical building shading map);

[0108] S304: constructing a texture feature library;

[0109] S306: The user collects real-scene pictures or real-scene videos through the terminal and uploads them to the server;

[0110] S308: The server obtains a real-scene image or a real-scene video, and performs raw data calculation on the real-scene image or the real-scene video, including image filtering and error correction (i.e., obtains a real-scene image to be processed and real-scene acquisition parameters corresponding to the real-scene image to be processed, and performs coordinate correction on the real-scene image to be processed according to the real-scene acquisition parameters to obtain a target camera pose);

[0111] S310: The server performs a shading update detection, that is, obtains a shading update detection area according to the target camera posture, the historical real scene map model, and the historical building shading map;

[0112] S312, when there is a texture update in the texture update detection area, the server performs texture remapping, that is, extracts an updated texture image from the real scene image to be processed, performs texture remapping on the updated texture image, and updates the historical real scene map model.

[0113] In one embodiment, Figure 4 As shown, a flow chart is provided to illustrate the model texture updating method of the present application, and the method comprises the following steps:

[0114] Step 402, obtaining a real scene image to be processed and real scene acquisition parameters corresponding to the real scene image to be processed;

[0115] Step 404, obtaining a simulated observation image according to the real scene acquisition parameters and the historical real scene map model;

[0116] Step 406, performing feature extraction on the simulated observation image and the real scene image to be processed respectively, to obtain a first feature point set of the simulated observation image and a second feature point set of the real scene image to be processed;

[0117] Step 408, performing a similarity comparison on the feature points in the first feature point set and the second feature point set to determine a plurality of pairs of similar feature points;

[0118] Step 410, obtaining feature points with the same name based on multiple pairs of similar feature points;

[0119] Step 412, obtaining a coordinate transformation matrix according to multiple pairs of similar feature points among the feature points with the same name;

[0120] Step 414, obtaining a coordinate transformation feature point according to the coordinate transformation matrix and the second feature point among the similar feature points;

[0121] Step 416, comparing the coordinate transformation feature point and the first feature point in the similar feature points to obtain a reprojection error, and adjusting the coordinate transformation matrix;

[0122] Step 418, returning to the step of obtaining the coordinate transformation feature point according to the coordinate transformation matrix and the second feature point in the similar feature points, until the reprojection error converges, and obtaining the target camera pose according to the latest coordinate transformation matrix;

[0123] Step 420, obtaining the real scene acquisition space range according to the target camera position;

[0124] Step 422, according to the spatial range of the real scene collection, compare the historical real scene map model and the historical building shading map to obtain the shading update detection area;

[0125] Step 424, obtaining the shading feature data corresponding to the shading update detection area and the shading feature data of the adjacent area;

[0126] Step 426, performing image feature matching on the real scene image to be processed according to the shading feature data and the shading feature data of the adjacent area;

[0127] Step 428, when the image features do not match, determining that there is a shading update in the shading update detection area;

[0128] Step 430, when there is a shading update in the shading update detection area, obtaining shading feature data corresponding to the shading update detection area;

[0129] Step 432, determining a valid position of the updated shading image in the real scene image to be processed according to the shading feature data and the target camera position and posture;

[0130] Step 434, extracting an updated shading image from the real scene image to be processed according to the effective position;

[0131] Step 436, remapping the texture of the updated background image to update the historical real-life map model.

[0132] It should be understood that, although the steps in each flow chart involved in the above-described embodiment are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a portion of the steps in each flow chart involved in the above-described embodiment may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the steps or stages in other steps.

[0133] In one embodiment, Figure 5 As shown, a model texture updating device is provided, comprising: an acquisition module 502, a correction module 504, a processing module 506, an extraction module 508 and a mapping module 510, wherein:

[0134] An acquisition module 502 is used to acquire a real scene image to be processed and real scene acquisition parameters corresponding to the real scene image to be processed;

[0135] A correction module 504 is used to perform coordinate correction on the real scene image to be processed according to the real scene acquisition parameters to obtain the target camera pose;

[0136] The processing module 506 is used to obtain a texture update detection area according to the target camera position, the historical real scene map model and the historical building texture map;

[0137] An extraction module 508 is used to extract an updated shading image from the real scene image to be processed when there is a shading update in the shading update detection area;

[0138] The mapping module 510 is used to perform texture remapping on the updated background image to update the historical real scene map model.

[0139] The above-mentioned model texture updating device can determine the target camera posture by acquiring the real scene image to be processed and performing coordinate correction on the real scene image to be processed according to the real scene acquisition parameters, so as to obtain the texture update detection area according to the target camera posture, the historical real scene map model and the historical building texture map. When there is a texture update in the texture update detection area, the historical real scene map model can be updated by extracting the updated texture image from the real scene image to be processed and remapping the texture of the updated texture image. In the whole process, the updated texture image with clear texture is extracted by using the real scene image to be processed, and the historical real scene map model is updated according to the updated texture image, so that the surface texture of the updated historical real scene map model can be made clear, thereby improving the quality of the real scene map model.

[0140] In one embodiment, the correction module is also used to obtain a simulated observation image based on the real-scene acquisition parameters and the historical real-scene map model, perform feature matching on the simulated observation image and the real-scene image to be processed, obtain feature homonymous points, determine the reprojection error based on the feature homonymous points, and obtain the target camera pose.

[0141] In one embodiment, the correction module is also used to perform feature extraction on the simulated observation image and the real scene image to be processed respectively, to obtain a first feature point set of the simulated observation image and a second feature point set of the real scene image to be processed, to perform similarity comparison on the feature points in the first feature point set and the second feature point set, to determine multiple pairs of similar feature points, and to obtain feature homonymous points based on the multiple pairs of similar feature points.

[0142] In one embodiment, the correction module is also used to obtain a coordinate transformation matrix based on multiple pairs of similar feature points in the feature points with the same name, obtain coordinate transformation feature points based on the coordinate transformation matrix and the second feature points among the similar feature points, compare the coordinate transformation feature points with the first feature points among the similar feature points to obtain a reprojection error, and adjust the coordinate transformation matrix, return to the step of obtaining the coordinate transformation feature points based on the coordinate transformation matrix and the second feature points among the similar feature points, until the reprojection error converges, and obtain the target camera pose based on the latest coordinate transformation matrix.

[0143] In one embodiment, the processing module is also used to obtain the real scene acquisition space range according to the target camera posture, and compare the historical real scene map model and the historical building texture map according to the real scene acquisition space range to obtain the texture update detection area.

[0144] In one embodiment, the processing module is also used to obtain the background feature data corresponding to the background update detection area and the background feature data of the adjacent area, and perform image feature matching on the real scene image to be processed based on the background feature data and the background feature data of the adjacent area. When the image features do not match, it is determined that there is a background update in the background update detection area.

[0145] In one embodiment, the processing module is further used to extract characters from the real scene image to be processed, obtain the character features to be compared, and compare the shading character features in the shading feature data with the character features to be compared.

[0146] In one embodiment, the extraction module is also used to obtain the texture feature data corresponding to the texture update detection area when there is a texture update in the texture update detection area, determine the effective position of the updated texture image in the real scene image to be processed according to the texture feature data and the target camera posture, and extract the updated texture image from the real scene image to be processed according to the effective position.

[0147] The specific definition of the model texture updating device can be found in the definition of the model texture updating method above, which will not be repeated here. Each module in the above-mentioned model texture updating device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0148] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store shading feature data, etc. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a model texture updating method is implemented.

[0149] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0150] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0151] Acquire a real scene image to be processed and real scene acquisition parameters corresponding to the real scene image to be processed;

[0152] According to the real scene acquisition parameters, coordinate correction is performed on the real scene image to be processed to obtain the target camera pose;

[0153] According to the target camera position, the historical real scene map model and the historical building shading map, the shading update detection area is obtained;

[0154] When there is a shading update in the shading update detection area, an updated shading image is extracted from the real scene image to be processed;

[0155] Texture remapping is performed on the updated shading image to update the historical real-life map model.

[0156] In one embodiment, when the processor executes the computer program, the following steps are also implemented: according to the real scene acquisition parameters and the historical real scene map model, a simulated observation image is obtained, feature matching is performed on the simulated observation image and the real scene image to be processed to obtain feature homonymous points, and a reprojection error is determined according to the feature homonymous points to obtain the target camera pose.

[0157] In one embodiment, when the processor executes the computer program, the following steps are also implemented: feature extraction is performed on the simulated observation image and the real scene image to be processed respectively to obtain a first feature point set of the simulated observation image and a second feature point set of the real scene image to be processed, similarity comparison is performed on the feature points in the first feature point set and the second feature point set to determine multiple pairs of similar feature points, and feature homonymous points are obtained based on the multiple pairs of similar feature points.

[0158] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining a coordinate transformation matrix based on multiple pairs of similar feature points in the feature points with the same name, obtaining a coordinate transformation feature point based on the coordinate transformation matrix and the second feature point in the similar feature points, comparing the coordinate transformation feature point with the first feature point in the similar feature points to obtain a reprojection error, and adjusting the coordinate transformation matrix, returning to the step of obtaining a coordinate transformation feature point based on the coordinate transformation matrix and the second feature point in the similar feature points, until the reprojection error converges, and obtaining the target camera pose based on the latest coordinate transformation matrix.

[0159] In one embodiment, when the processor executes the computer program, the following steps are also implemented: according to the target camera posture, the real scene acquisition space range is obtained; according to the real scene acquisition space range, the historical real scene map model and the historical building texture map are compared to obtain the texture update detection area.

[0160] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining the background feature data corresponding to the background update detection area and the background feature data of the adjacent area, performing image feature matching on the real scene image to be processed according to the background feature data and the background feature data of the adjacent area, and when the image features do not match, determining that there is a background update in the background update detection area.

[0161] In one embodiment, when the processor executes the computer program, the following steps are also implemented: extracting characters from the real scene image to be processed to obtain the character features to be compared, and comparing the shading character features in the shading feature data with the character features to be compared.

[0162] In one embodiment, the processor further implements the following steps when executing the computer program: when there is a texture update in the texture update detection area, obtaining the texture feature data corresponding to the texture update detection area, determining the effective position of the updated texture image in the real scene image to be processed based on the texture feature data and the target camera posture, and extracting the updated texture image from the real scene image to be processed based on the effective position.

[0163] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0164] Acquire a real scene image to be processed and real scene acquisition parameters corresponding to the real scene image to be processed;

[0165] According to the real scene acquisition parameters, coordinate correction is performed on the real scene image to be processed to obtain the target camera pose;

[0166] According to the target camera position, the historical real scene map model and the historical building shading map, the shading update detection area is obtained;

[0167] When there is a shading update in the shading update detection area, an updated shading image is extracted from the real scene image to be processed;

[0168] Texture remapping is performed on the updated shading image to update the historical real-life map model.

[0169] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: according to the real scene acquisition parameters and the historical real scene map model, a simulated observation image is obtained, feature matching is performed on the simulated observation image and the real scene image to be processed to obtain feature homonymous points, and the reprojection error is determined according to the feature homonymous points to obtain the target camera pose.

[0170] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: feature extraction is performed on the simulated observation image and the real scene image to be processed respectively to obtain a first feature point set of the simulated observation image and a second feature point set of the real scene image to be processed, similarity comparison is performed on the feature points in the first feature point set and the second feature point set to determine multiple pairs of similar feature points, and feature homonymous points are obtained based on the multiple pairs of similar feature points.

[0171] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining a coordinate transformation matrix based on multiple pairs of similar feature points in the feature points with the same name, obtaining a coordinate transformation feature point based on the coordinate transformation matrix and the second feature point in the similar feature points, comparing the coordinate transformation feature point with the first feature point in the similar feature points to obtain a reprojection error, and adjusting the coordinate transformation matrix, returning to the step of obtaining a coordinate transformation feature point based on the coordinate transformation matrix and the second feature point in the similar feature points, until the reprojection error converges, and obtaining the target camera pose based on the latest coordinate transformation matrix.

[0172] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: according to the target camera posture, the real scene acquisition space range is obtained; according to the real scene acquisition space range, the historical real scene map model and the historical building texture map are compared to obtain the texture update detection area.

[0173] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining the background feature data corresponding to the background update detection area and the background feature data of the adjacent area, performing image feature matching on the real scene image to be processed based on the background feature data and the background feature data of the adjacent area, and when the image features do not match, determining that there is a background update in the background update detection area.

[0174] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: extracting characters from the real scene image to be processed to obtain the character features to be compared, and comparing the shading character features in the shading feature data with the character features to be compared.

[0175] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when there is a texture update in the texture update detection area, the texture feature data corresponding to the texture update detection area is obtained, and based on the texture feature data and the target camera posture, the effective position of the updated texture image in the real scene image to be processed is determined, and the updated texture image is extracted from the real scene image to be processed based on the effective position.

[0176] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0177] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0178] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A model texture updating method, It is characterized in that The method comprises: Acquire a real scene image to be processed and real scene acquisition parameters corresponding to the real scene image to be processed; Performing coordinate correction on the real scene image to be processed according to the real scene acquisition parameters to obtain the target camera pose; Obtaining a texture update detection area according to the target camera position, the historical real-scene map model, and the historical building texture map; When there is a shading update in the shading update detection area, extracting an updated shading image from the real scene image to be processed; Texture remapping is performed on the updated shading image to update the historical real-scene map model.

2. The method according to claim 1, It is characterized in that The coordinate correction of the real scene image to be processed according to the real scene acquisition parameters to obtain the target camera pose comprises: According to the real scene acquisition parameters and the historical real scene map model, a simulated observation image is obtained; Performing feature matching on the simulated observation image and the real scene image to be processed to obtain feature points of the same name; The reprojection error is determined according to the characteristic same-name points to obtain the target camera pose.

3. The method according to claim 2, It is characterized in that The performing feature matching on the simulated observation image and the real scene image to be processed to obtain feature points of the same name includes: Extracting features from the simulated observation image and the real scene image to be processed respectively to obtain a first feature point set of the simulated observation image and a second feature point set of the real scene image to be processed; Performing a similarity comparison on feature points in the first feature point set and the second feature point set to determine multiple pairs of similar feature points; According to the multiple pairs of similar feature points, feature points with the same name are obtained.

4. The method according to claim 2, It is characterized in that Determining the reprojection error according to the characteristic same-name points to obtain the target camera pose comprises: Obtaining a coordinate transformation matrix according to a plurality of pairs of similar feature points among the feature points of the same name; Obtaining a coordinate transformation feature point according to the coordinate transformation matrix and a second feature point among the similar feature points; Comparing the coordinate transformation feature point with a first feature point among the similar feature points to obtain a reprojection error, and adjusting the coordinate transformation matrix; Return to the step of obtaining the coordinate transformation feature point according to the coordinate transformation matrix and the second feature point among the similar feature points, until the reprojection error converges, and obtain the target camera pose according to the latest coordinate transformation matrix.

5. The method according to claim 1, It is characterized in that The obtaining of the texture update detection area according to the target camera position, the historical real scene map model and the historical building texture map comprises: According to the target camera position and posture, a real scene acquisition space range is obtained; According to the real scene acquisition space range, the historical real scene map model and the historical building shading map are compared to obtain the shading update detection area.

6. The method according to claim 1, It is characterized in that After obtaining the texture update detection area according to the target camera position, the historical real scene map model and the historical building texture map, the method further includes: Acquire the shading feature data corresponding to the shading update detection area and the shading feature data of the adjacent area; Performing image feature matching on the real scene image to be processed according to the shading feature data and the shading feature data of the adjacent area; When the image features do not match, it is determined that there is a shading update in the shading update detection area.

7. The method according to claim 6, It is characterized in that The performing image feature matching on the real scene image to be processed according to the shading feature data and the shading feature data of the adjacent area comprises: Extracting characters from the real scene image to be processed to obtain features of the characters to be compared; The shading character features in the shading feature data are compared with the character features to be compared.

8. The method according to claim 1, It is characterized in that When there is a shading update in the shading update detection area, extracting an updated shading image from the real scene image to be processed comprises: When there is a shading update in the shading update detection area, acquiring shading feature data corresponding to the shading update detection area; Determine, according to the shading feature data and the target camera position, a valid position of updating the shading image in the real scene image to be processed; An updated shading image is extracted from the real scene image to be processed according to the effective position.

9. A model texture updating device, It is characterized in that The device comprises: An acquisition module, used for acquiring a real scene image to be processed, wherein the real scene image to be processed carries real scene acquisition parameters; A correction module, used for performing coordinate correction on the real scene image to be processed according to the real scene acquisition parameters to obtain a target camera pose; A processing module, used for obtaining a texture update detection area according to the target camera posture, the historical real scene map model and the historical building texture map; An extraction module, configured to extract an updated shading image from the real scene image to be processed when there is a shading update in the shading update detection area; A mapping module is used to perform texture remapping on the updated background image to update the historical real-scene map model.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program. It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images