Map updating method, device, electronic device and computer-readable storage medium

Abstract

Embodiments of the present application provide a map update method, apparatus, electronic device, and computer-readable storage medium, relating to the field of computer vision technology. The method includes: obtaining a current image and historical images of a map corresponding to a region to be inspected; determining distinguishing feature information between the current image and the historical images based on a trained Siamese network; determining at least one change location frame of the historical image relative to the current image based on the distinguishing feature information; and updating the map based on the at least one change location frame. Embodiments of the present application achieve efficient and accurate map updates.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology. Specifically, the present application relates to a map updating method, device, electronic device and computer-readable storage medium. Background Art

[0002] Maps are widely used in resource exploration, engineering construction, and daily life. With the development of internet technology, electronic devices, primarily computers, are becoming increasingly popular in cartography. Maps can now be presented digitally on a variety of devices, providing richer information and a wider range of applications.

[0003] As time goes by, geographical features also undergo periodic changes, such as the addition or demolition of buildings. Therefore, maps need to be updated in a timely manner to provide users with more accurate data information and services.

[0004] Currently, map changes are usually detected and updated based on semantic segmentation algorithms. However, semantic segmentation algorithms predict small changes based on the pixel level and are sensitive to color and light, resulting in a large number of false detections and reducing the accuracy of map updates. Summary of the Invention

[0005] The present application provides a map updating method, device, electronic device and computer-readable storage medium, which can solve the problem of low map updating accuracy.

[0006] In a first aspect of the present application, a map updating method is provided, the method comprising:

[0007] Obtain the current image and historical images of the map corresponding to the area to be detected;

[0008] Determine the distinguishing feature information between the current image and the historical image based on the trained Siamese network;

[0009] determining at least one change positioning frame of the historical image relative to the current image based on the distinguishing feature information;

[0010] The map is updated based on the at least one changed location box.

[0011] In one possible implementation, determining the distinguishing feature information between the current image and the historical image includes:

[0012] Perform feature extraction on the current image to obtain a first feature point set;

[0013] Perform feature extraction on the historical image to obtain a second feature point set;

[0014] determining a set of distinguishing points between the first set of feature points and the second set of feature points;

[0015] Based on the set of distinguishing points, distinguishing feature information is generated.

[0016] In one possible implementation, determining at least one change positioning frame of the historical image relative to the current image based on the distinguishing feature information includes:

[0017] determining at least one target feature point based on the distinguishing feature information;

[0018] At least one change positioning frame of a preset shape is determined based on the target feature points.

[0019] In another possible implementation, the determining of at least one target feature point based on the distinguishing feature information includes:

[0020] Upsample the distinguishing feature information to obtain fused feature information;

[0021] At least one target feature point is obtained based on the fused feature information.

[0022] In another possible implementation, the shape of the change positioning frame is a rectangle; and determining at least one change positioning frame of a preset shape based on the target feature points includes:

[0023] Determine the category and margin data corresponding to each target feature point respectively;

[0024] Based on the category and margin data, the rectangular change positioning box corresponding to each target feature point is determined respectively.

[0025] In yet another possible implementation, updating the map based on at least one changed positioning frame includes:

[0026] determining a change profile based on each change positioning frame;

[0027] receiving correction information, and correcting the changed contour based on the correction information to generate feature contour data;

[0028] Update the map based on the feature contour data.

[0029] In another possible implementation, the above-mentioned twin network is trained in the following way:

[0030] Obtain sample images of the same sample area at different times; the sample images are provided with corresponding sample positioning frames;

[0031] Input the sample image into the initial twin network and obtain the real-time positioning box label output by the initial twin network;

[0032] Determine the multi-branch loss function based on the sample positioning box and positioning box label;

[0033] The parameters of the initial twin network are adjusted based on the multi-branch loss function to obtain the twin network.

[0034] In a second aspect of the present application, a map updating device is provided, the device comprising:

[0035] An acquisition module is used to obtain the current image and historical images of the map corresponding to the area to be detected;

[0036] An extraction module, which is used to determine the distinguishing feature information between the current image and the historical images based on the trained Siamese network;

[0037] a determination module, configured to determine at least one change positioning frame of the historical image relative to the current image based on the distinguishing feature information;

[0038] An updating module is configured to update a map based on at least one changed positioning frame.

[0039] In one possible implementation, the extraction module is specifically configured to:

[0040] Perform feature extraction on the current image to obtain a first feature point set;

[0041] Perform feature extraction on the historical image to obtain a second feature point set;

[0042] determining a set of distinguishing points between the first set of feature points and the second set of feature points;

[0043] Based on the set of distinguishing points, distinguishing feature information is generated.

[0044] In one possible implementation, the determining module specifically includes:

[0045] a first determining unit, configured to determine at least one target feature point based on the distinguishing feature information;

[0046] The second determining unit is configured to determine at least one change positioning frame of a preset shape based on the target feature points.

[0047] In another possible implementation, the first determining unit is further configured to:

[0048] Upsample the distinguishing feature information to obtain fused feature information;

[0049] At least one target feature point is obtained based on the fused feature information.

[0050] In another possible implementation, the shape of the change positioning frame may be a rectangle, and the second determining unit is further configured to:

[0051] Determine the category and margin data corresponding to each target feature point respectively;

[0052] Based on the category and margin data, the rectangular change positioning box corresponding to each target feature point is determined respectively.

[0053] In yet another possible implementation, the update module is further configured to:

[0054] determining a change profile based on each change positioning frame;

[0055] receiving correction information, and correcting the changed contour based on the correction information to generate feature contour data;

[0056] Update the map based on the feature contour data.

[0057] In yet another possible implementation, the apparatus further includes a training module, specifically configured to:

[0058] Obtain sample images of the same sample area at different times; the sample images are provided with corresponding sample positioning frames;

[0059] Input the sample image into the initial twin network and obtain the real-time positioning box label output by the initial twin network;

[0060] Determine the multi-branch loss function based on the sample positioning box and positioning box label;

[0061] The parameters of the initial twin network are adjusted based on the multi-branch loss function to obtain the twin network.

[0062] The third aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the map updating method shown in the first aspect of the present application is implemented.

[0063] In a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the map updating method shown in the first aspect of the present application is implemented.

[0064] The beneficial effects of the technical solution provided by this application are:

[0065] This application uses a twin network to determine the distinguishing feature information between the current image and the historical image. The twin network has strong anti-interference ability, so that the obtained distinguishing feature information has better feature representation ability. Based on the distinguishing feature information, the change positioning frame is determined to update the map, which can effectively reduce false detections of map changes and improve the accuracy of map updates.

[0066] In addition, area change recognition based on the change positioning frame can quickly lock the map change area. Compared with the pixel-level change recognition of semantic segmentation in the existing technology, it greatly improves the efficiency of map data update, reduces the network model's requirements for image resolution and image quality, and ensures the accuracy of map data change positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments of the present application.

[0068] Figure 1 A diagram illustrating an application scenario of a map updating method provided in an embodiment of the present application;

[0069] Figure 2 A flowchart of a map updating method provided in an embodiment of the present application;

[0070] Figure 3 A schematic diagram of a twin network architecture based on the FCOS algorithm provided as an example of this application;

[0071] Figure 4 A schematic diagram of a scheme for determining distinguishing characteristic information provided as an example of this application;

[0072] Figure 5 Flowchart of the fusion feature output solution provided for an example of this application

[0073] Figure 6 A schematic diagram of a twin network training process provided in an embodiment of the present application;

[0074] Figure 7 A flowchart of a map updating method provided as an example of this application;

[0075] Figure 8 A schematic diagram of the structure of a map updating device provided in an embodiment of the present application;

[0076] Figure 9 A schematic structural diagram of a map updating device provided as an example of this application;

[0077] Figure 10 A schematic structural diagram of an electronic device for map updating provided in an embodiment of the present application. DETAILED DESCRIPTION

[0078] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and are not to be construed as limiting the present application.

[0079] It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any units and all combinations of one or more associated listed items.

[0080] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0081] A map is a graphic or image that selectively represents certain phenomena of the earth (or other planets) in two-dimensional or multi-dimensional forms and means on a plane or sphere according to certain rules. It has a strict mathematical basis, symbol system, text annotations, and can use map generalization principles to scientifically reflect the distribution characteristics of natural and socio-economic phenomena and their interrelationships.

[0082] Map updates involve revising map content to improve accuracy and maintain map currency based on the changing realities of the area depicted. The shorter the update cycle, the more current the map and its value. There are two methods for updating: comprehensive resurvey (or editing) and partial resurvey (or editing).

[0083] The map updating method provided in this application can achieve efficient and accurate map updating, thereby ensuring the freshness of map data.

[0084] Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0085] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0086] Computer vision (CV) is the science of making machines "see." Specifically, it refers to the use of cameras and computers to replace the human eye in identifying and measuring objects, and further processing the images to create images more suitable for human observation or transmission to instrumentation. As a scientific discipline, computer vision studies related theories and technologies, aiming to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies generally include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0087] The solution provided in the embodiments of this application involves artificial intelligence map update technology, which is specifically illustrated by the following embodiments.

[0088] The key to map updates is change detection between images of the same area corresponding to different time periods. Existing deep learning-based change detection algorithms mostly use deep learning features from different images to generate differential images, or use strategies to learn change relationships between pixel blocks to complete change detection. These change detection algorithms are typical semantic segmentation-based algorithms, assigning different categories to changed and unchanged pixels. Taking the U-net + FPN + basic model model commonly used in existing semantic segmentation solutions as an example, the following problems exist in detecting changed buildings:

[0089] (1) Pixel-level classification has too many false positives, lacks integrity, and increases manual workload. This end-to-end semantic segmentation algorithm can accurately predict small changes at the pixel level, but is sensitive to color and light, resulting in a large number of false positives. Although the recall rate is guaranteed, the accuracy rate is greatly reduced.

[0090] (2) It is unable to filter out minor changes in individual buildings that are not important, resulting in a lot of useless information in the detection process. For example, in many large villages, individual house renovations often occur. This level of change does not need to be detected and does not contribute to the background data.

[0091] (3) The semantic segmentation solution is based on pixel-level detection and cannot complete instance-level building detection. Pixel blocks can only extract local features and cannot be directly applied to data fusion. At the same time, pixel-level classification does not fully utilize the context of the image. Features from different regions of the image cannot be shared, and classification performance is limited.

[0092] (4) Data annotation costs are too high. Semantic segmentation solutions require high-quality satellite images as training data, and pixel-based annotation of each building change is too expensive for observation and operation.

[0093] The map update method provided in this application is based on a deep learning target detection model, combined with a twin network structure to extract change features, and uses end-to-end detection of map change areas to achieve efficient and accurate map updates.

[0094] The map updating method, device, electronic device, and computer-readable storage medium provided in this application are intended to solve the above technical problems in the prior art.

[0095] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0096] like Figure 1 As shown, the map updating method of this application can be applied to Figure 1 In the scenario shown, specifically, after the server 103 receives the current image 101 and the historical image 102 of the map corresponding to the area to be detected sent by the terminal, it identifies the distinguishing feature information between the current image 101 and the historical image 102 based on the twin network 104, and determines at least one change positioning frame 105 of the historical image 102 relative to the current image 101, so as to update the map based on the at least one change positioning frame 105 to obtain an updated map 106.

[0097] Figure 1 In the scenario shown, the above map updating method can be performed in a server, and in other scenarios, it can also be performed in a terminal.

[0098] Those skilled in the art will understand that the “terminal” used here may be a mobile phone, a tablet computer, a PDA (Personal Digital Assistant), a MID (Mobile Internet Device), etc.; the “server” may be implemented as an independent server or a server cluster consisting of multiple servers.

[0099] The present application provides a method for updating a map. Figure 2 As shown, the method may include the following steps:

[0100] S201, obtaining a current image and a historical image of a map corresponding to an area to be detected.

[0101] In an embodiment of the present application, after determining the area to be detected that requires map update, map data corresponding to the area to be detected is extracted to obtain map data of the same area at different times, that is, the current image and historical image of the map corresponding to the area to be detected.

[0102] Among them, the current image or historical image can be obtained through existing GIS (Geographic Information System) data, or can be directly obtained based on remote sensing data.

[0103] S202, determining distinguishing feature information between the current image and the historical image based on the trained twin network.

[0104] Among them, the Siamese network is a detection network based on the twin neural network architecture; the Siamese neural network (Siameseneural network), also known as the twin neural network, is a coupled architecture based on two artificial neural networks. It takes two samples as input and outputs their representation embedded in a high-dimensional space to compare the similarity of the two samples.

[0105] Specifically, the detection network can be based on target detection algorithms such as FCOS (Fully Convolutional One-Stage Object Detection), SSD (Single Shot MultiBox Detector), or Faster R-CNN (Faster Region-CNN).

[0106] S203: Determine at least one change positioning frame of the historical image relative to the current image based on the distinguishing feature information.

[0107] The change positioning frame represents the area change of the historical image relative to the current image, that is, the changes in the same area position on the map. The changes may include: construction of buildings, reconstruction of roads or construction of bridges.

[0108] The shape of the change positioning frame may be a rectangle, a circle, or a polygon generated based on the contour of the change object, which is not specifically limited in this embodiment.

[0109] S204: Update the map based on at least one changed positioning frame.

[0110] In the above embodiment, the distinguishing feature information between the current image and the historical image is determined by the twin network. The twin network has strong anti-interference ability, so that the obtained distinguishing feature information has better feature representation ability. The change positioning frame is determined based on the distinguishing feature information, and the map is updated. This can effectively reduce false detections of map changes and improve the accuracy of map updates.

[0111] Next, the map updating method provided by this embodiment is specifically described by taking the twin network based on the FCOS algorithm as an example. Figure 3 As shown in the figure, the model architecture of the twin network based on the FCOS algorithm can include a Backbone (backbone network) module, an FPN (Feature Pyramid Network) module and a multi-branch loss function module.

[0112] The Backbone module extracts image information at the front end and generates feature maps for subsequent network use. Backbone typically employs the ResNet101 or VGGNet architecture. The FPN module performs feature fusion upsampling, processing features based on the multi-scale characteristics of objects in the image. The multi-branch loss function module predicts the changing localization box, accurately targeting the changing areas of the map.

[0113] A possible implementation method is provided in the embodiment of the present application, such as Figure 4 As shown, step S202 of determining the distinguishing feature information between the current image and the historical image may include:

[0114] (1) Extract features from the current image to obtain the first feature point set;

[0115] (2) Extract features from historical images to obtain a second feature point set;

[0116] (3) determining a set of distinguishing points between the first feature point set and the second feature point set;

[0117] (4) Generate distinguishing feature information based on the distinguishing point set.

[0118] In this embodiment, the current image and the comparison image are used as the input of the twin network, and feature parameters are extracted respectively based on the Backbone module, where the Backbone module can adopt the Resnet101 structure.

[0119] Specifically, the current image and the comparison image share the Resnet101 structure and extract corresponding features. Downsampling based on the Resnet101 structure network is divided into five stages: C1, C2, C3, C4, and C5. The scale of the feature map output by each stage is different. Because the semantic information of the C1 and C2 stages is too little, the feature output feature maps of the C3, C4, and C5 stages are respectively selected to represent the feature map of the current image or the historical image. The feature map within the same stage is subtracted, and the distinguishing feature information is determined based on the feature output feature maps of the deleted C3, C4, and C5 stages.

[0120] Among them, the five stages are composed of multiple residual units, each of which consists of a convolutional layer, a normalization layer (Batch Normalization), an activation layer (ReLU), and an identity mapping:

[0121] The bottom convolutional layer extracts basic features such as image edges and texture, while the higher-level convolutional layers abstract and combine these texture features. The normalization layer normalizes the features to a normal distribution. The activation layer applies nonlinear mapping to the extracted features, enhancing the model's generalization capabilities. The identity mapping is a shortcut link that does not generate additional parameters or increase computational complexity, ensuring efficient gradient propagation during subsequent Siamese network training.

[0122] In order to obtain the distinguishing feature information between the current image and the comparison image, this application introduces a twin network feature extraction strategy. Compared with the conventional technical means of splicing pictures based on the channel dimension or subtracting two pictures to generate residual maps, the twin network architecture improves the expression ability of the extracted features, and the twin network has strong anti-interference ability.

[0123] An embodiment of the present application provides a possible implementation method. In step S203, determining at least one change positioning frame of the historical image relative to the current image based on the distinguishing feature information may include:

[0124] (1) Determine at least one target feature point based on the distinguishing feature information.

[0125] Another possible implementation method provided in the embodiment of the present application is to determine at least one target feature point based on the distinguishing feature information, which may include:

[0126] a. Upsample the distinguishing feature information to obtain fused feature information;

[0127] b. Based on the fused feature information, obtain at least one target feature point.

[0128] Specifically, the FPN module adopts a feature pyramid strategy to perform multi-scale and hierarchical upsampling operations on the distinguishing feature information extracted by the Backbone module to achieve feature map scale amplification and feature information fusion. Based on the feature outputs of the C3, C4 and C5 stages of the Resnet101 network, the fused feature outputs of the FPN module P3, P4, P5, P6 and P7, a total of five levels, are determined.

[0129] like Figure 5 As shown, we first extract the feature outputs of the resnet101 network C3, C4 and C5 stages, and then pass through 1 A convolution operation of 1 is performed to obtain feature maps as shown in M3, M4, and M5; a convolution operation with a step size of 2 is performed on the basis of the P5 layer to generate feature maps of the P6 and P7 layers respectively;

[0130] Then, the top-down fusion upsampling can be performed by using the nearest neighbor interpolation method. The initial P5 level features are upsampled and added to the initial P4 level feature map. Finally, after 3 3 convolutions to obtain the final P4-level feature output; the final P4-level feature is upsampled and added to M3, and finally 3 The final P3-level feature output is obtained through three convolutions. Following the same methodology, the P3 level is also upsampled by the upsampling module to achieve scale amplification and integrate necessary feature information. The P7-level feature output has the largest single-pixel receptive field and is responsible for predicting large areas, such as an entire industrial park. The P3-level feature output has a single-pixel receptive field that is 32 times smaller than that of the P7 level and is responsible for predicting small objects, such as individual buildings or bridges.

[0131] Finally, the regression target corresponding to each pixel point on all levels is calculated, and the pixels that meet the preset conditions are selected as target feature points based on the regression target; the above preset conditions can be set separately for different levels where the pixels are located.

[0132] Through fused upsampling based on a feature pyramid strategy, the input to each upsampling module comes not only from the output of the previous upsampling layer, but also from the shared feature layer of the same size corresponding to the Backbone feature extraction process. Convolution operations are used to achieve information fusion, achieving better integration of hierarchical feature information. Furthermore, the feature map outputs of different layers are responsible for detecting objects of different sizes, enhancing the accuracy of subsequent object detection.

[0133] (2) Determine at least one change positioning frame of a preset shape based on the target feature points.

[0134] An embodiment of the present application provides a possible implementation method, in which the shape of the above-mentioned changing positioning frame may be a rectangle; determining at least one changing positioning frame of a preset shape based on the target feature points may include:

[0135] a. Determine the category and margin data corresponding to each target feature point;

[0136] b. Based on the category and margin data, determine the rectangular change positioning box corresponding to each target feature point.

[0137] Among them, the twin network classifies, predicts and regresses the target feature points at all levels, determines the object category to which each point belongs and the distance from each point to the four boundaries of the object, so as to determine the category of the object and a corresponding rectangular change positioning box.

[0138] An embodiment of the present application provides a possible implementation method, in which step S204 of updating the map based on at least one changed positioning frame may include:

[0139] determining a change profile based on each change positioning frame;

[0140] receiving correction information, and correcting the changed contour based on the correction information to generate feature contour data;

[0141] Update the map based on the feature contour data.

[0142] Specifically, taking building change updates as an example, based on the building instance segmentation algorithm, an accurate building outline can be generated according to the change positioning box; then, based on manual operations, the building outline can be edited and corrected for a second time, and finally the corrected building outline data can be used as map update data to update the buildings on the map.

[0143] A possible implementation method is provided in the embodiment of the present application, such as Figure 6 As shown, the twin network can be trained through the following steps:

[0144] S601, obtaining sample images of the same sample area at different times; the sample images are provided with corresponding sample positioning frames;

[0145] S602: Input the sample image into the initial twin network to obtain the real-time positioning box label output by the initial twin network;

[0146] S603, determining a multi-branch loss function based on the sample positioning frame and the positioning frame label;

[0147] S604: Adjust the parameters of the initial twin network based on the multi-branch loss function to obtain the twin network.

[0148] Specifically, the embodiment of the present application uses an anchor-free strategy to directly predict the target for each pixel, measure the distance above, below, left, and right of the change positioning box, and the category of the target, obtain the real-time positioning box label, and then train the twin network based on the multi-branch loss function, where the multi-branch loss function includes classification loss, regression loss, and center loss:

[0149] (1) The classification loss can calculate the category score corresponding to the target feature point, which represents whether the target feature point falls within the corresponding sample positioning box;

[0150] (2) The regression loss can characterize the difference between the real-time positioning box label and the sample positioning box;

[0151] (3) Centerness loss In order to effectively screen the positioning box labels, the centerness of each target feature point is first calculated. The centerness data can represent the degree of deviation of the current point from the target center. Then, the centerness data is directly multiplied by the category score in the above classification loss as the final classification score.

[0152] The embodiment of the present application completely avoids the complex calculations related to anchors through an anchor-free strategy. Through multi-level data processing, the training samples of the twin network are richer, and various folded, extremely large, extremely small or irregular target objects can be detected. At the same time, the center loss is introduced to effectively limit the positioning box labels generated by points far away from the target center (the center of the sample positioning box), thereby reducing these relatively low-quality points, and using a multi-branch loss function to enhance the accuracy of the twin network prediction.

[0153] To better understand the above map update method, such as Figure 7 As shown, an example of the map updating method of the present application is described in detail below:

[0154] like Figure 7 As shown, in one example, the map updating method provided by this application may include the following steps:

[0155] S701, obtaining a current image and a historical image of a map corresponding to the area to be detected;

[0156] S702, extracting features from the current image based on the trained twin network to obtain a first feature point set; extracting features from the historical image to obtain a second feature point set; the twin network can be an FCOS or SSD network based on a twin neural network architecture;

[0157] S703, determining a distinguishing point set between the first feature point set and the second feature point set by downsampling the feature information, and generating distinguishing feature information based on the distinguishing point set; specifically, the downsampling of the feature information can be performed based on a Resnet101 network structure;

[0158] S704, upsampling the distinguishing feature information using a feature pyramid strategy to obtain fused feature information;

[0159] S705, obtaining at least one target feature point based on the fused feature information;

[0160] S706, respectively determining the category and margin data corresponding to each target feature point;

[0161] S707, based on the category and margin data, determine the rectangular change positioning frame corresponding to each target feature point; wherein the change positioning frame represents the area change of the historical image relative to the current image, that is, the change in the same area position on the map, which may include: construction of buildings, reconstruction of roads, or construction of bridges, etc.

[0162] S708: Update the map based on the at least one changed positioning frame.

[0163] The map updating method of the present application determines the distinguishing feature information between the current image and the historical image through the twin network. The twin network has strong anti-interference ability, so that the obtained distinguishing feature information has better feature representation ability. The change positioning frame is determined based on the distinguishing feature information, thereby updating the map. It can effectively reduce false detections of map changes and improve the accuracy of map updates.

[0164] In addition, area change recognition based on the change positioning frame can quickly lock the map change area. Compared with the pixel-level change recognition of semantic segmentation in the existing technology, it greatly improves the efficiency of map data update, reduces the network model's requirements for image resolution and image quality, and ensures the accuracy of map data change positioning.

[0165] The embodiment of the present application provides a map updating device, such as Figure 8As shown, the map updating device 80 may include: an acquisition module 801, an extraction module 802, a determination module 803 and an update module 804, wherein:

[0166] The acquisition module 801 is used to acquire the current image and historical images of the map corresponding to the area to be detected;

[0167] An extraction module 802 is used to determine distinguishing feature information between the current image and the historical image based on the trained Siamese network;

[0168] A determination module 803 is configured to determine at least one change positioning frame of the historical image relative to the current image based on the distinguishing feature information;

[0169] The updating module 804 is configured to update the map based on at least one changed positioning frame.

[0170] The present application provides a possible implementation method, in which the extraction module 802 can be specifically used to:

[0171] Perform feature extraction on the current image to obtain a first feature point set;

[0172] Perform feature extraction on the historical image to obtain a second feature point set;

[0173] determining a set of distinguishing points between the first set of feature points and the second set of feature points;

[0174] Based on the set of distinguishing points, distinguishing feature information is generated.

[0175] This embodiment of the present application provides a possible implementation method, and the determination module 803 may specifically include:

[0176] a first determining unit, configured to determine at least one target feature point based on the distinguishing feature information;

[0177] The second determining unit is configured to determine at least one change positioning frame of a preset shape based on the target feature points.

[0178] An embodiment of the present application provides a possible implementation manner, in which the first determining unit may further be configured to:

[0179] Upsample the distinguishing feature information to obtain fused feature information;

[0180] At least one target feature point is obtained based on the fused feature information.

[0181] An embodiment of the present application provides a possible implementation method, in which the shape of the change positioning frame may be a rectangle, and the second determining unit may further be configured to:

[0182] Determine the category and margin data corresponding to each target feature point respectively;

[0183] Based on the category and margin data, the rectangular change positioning box corresponding to each target feature point is determined respectively.

[0184] An embodiment of the present application provides a possible implementation method, and the update module 804 can also be used to:

[0185] determining a change profile based on each change positioning frame;

[0186] receiving correction information, and correcting the changed contour based on the correction information to generate feature contour data;

[0187] Update the map based on the feature contour data.

[0188] An embodiment of the present application provides a possible implementation method, wherein the device further includes a training module, which can be specifically used to:

[0189] Obtain sample images of the same sample area at different times; the sample images are provided with corresponding sample positioning frames;

[0190] Input the sample image into the initial twin network and obtain the real-time positioning box label output by the initial twin network;

[0191] Determine the multi-branch loss function based on the sample positioning box and positioning box label;

[0192] The parameters of the initial twin network are adjusted based on the multi-branch loss function to obtain the twin network.

[0193] In order to better understand the above map updating device, an example of the map updating device of the present application is described in detail below. Figure 9 As shown, the device 90 of this embodiment may include:

[0194] An acquisition module 901 is used to acquire a current image and a historical image of a map corresponding to the area to be detected;

[0195] Extraction module 902, for determining distinguishing feature information between the current image and the historical image based on the trained Siamese network;

[0196] A determination module 903 is configured to determine at least one change positioning frame of the historical image relative to the current image based on the distinguishing feature information;

[0197] An updating module 904, configured to update a map based on at least one changed positioning frame;

[0198] Training module 905 is used to obtain sample images of the same sample area at different times; the sample images are set with corresponding sample positioning frames; the sample images are input into the initial twin network to obtain the real-time positioning frame labels output by the initial twin network; the multi-branch loss function is determined based on the sample positioning frame and the positioning frame label; the parameters of the initial twin network are adjusted based on the multi-branch loss function to obtain the twin network.

[0199] The map updating device of this embodiment can execute the map updating method shown in the above embodiments of this application. The implementation principles are similar and will not be repeated here.

[0200] This application uses a twin network to determine the distinguishing feature information between the current image and the historical image. The twin network has strong anti-interference ability, so that the obtained distinguishing feature information has better feature representation ability. Based on the distinguishing feature information, the change positioning frame is determined to update the map, which can effectively reduce false detections of map changes and improve the accuracy of map updates.

[0201] In addition, area change recognition based on the change positioning frame can quickly lock the map change area. Compared with the pixel-level change recognition of semantic segmentation in the existing technology, it greatly improves the efficiency of map data update, reduces the network model's requirements for image resolution and image quality, and ensures the accuracy of map data change positioning.

[0202] An embodiment of the present application provides an electronic device, which includes: a memory and a processor; at least one program, stored in the memory, for being executed by the processor, which can achieve, compared with the prior art: determining the distinguishing feature information between the current image and the historical image through the twin network, and the twin network has strong anti-interference ability, so that the obtained distinguishing feature information has better feature representation ability; at the same time, based on the change positioning frame, area change recognition is realized, which can quickly lock the map change area. Compared with the pixel-level change recognition of semantic segmentation in the prior art, the update efficiency of map data is greatly improved, the requirements of the network model on image resolution and image quality are reduced, and the accuracy of map data change positioning is ensured; the embodiment of the present application can effectively reduce false detection and misdetection of map changes, and improve the accuracy of map updates.

[0203] In an alternative embodiment, an electronic device is provided, such as Figure 10 As shown, Figure 10The electronic device 4000 shown includes: a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which may be used for data exchange between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the number of transceivers 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0204] Processor 4001 can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0205] Bus 4002 may include a path for transmitting information between the above components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. Bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 10 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0206] The memory 4003 may be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0207] The memory 4003 is used to store the application code (computer program) for executing the solution of the present application, and the execution is controlled by the processor 4001. The processor 4001 is used to execute the application code stored in the memory 4003 to implement the content shown in the above method embodiment.

[0208] Among them, electronic devices include but are not limited to: mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 10 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0209] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer-readable storage medium is run on a computer, the computer can execute the corresponding contents of the aforementioned method embodiment.

[0210] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0211] The above descriptions are only partial embodiments of the present invention. It should be pointed out that ordinary technicians in this technical field can make several improvements and modifications without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A map updating method, characterized in that: include: Obtain the current image and historical images of the map corresponding to the area to be detected; Determining distinguishing feature information between the current image and the historical image at multiple scales based on the trained Siamese network; Performing a multi-scale, hierarchical upsampling operation on the distinguishing feature information of the multiple scales to obtain fused feature information of multiple levels, wherein the fused feature information of different levels has different receptive fields; Acquire at least one target feature point based on the fused feature information of the multiple levels; determining at least one change positioning frame of the historical image relative to the current image based on the target feature point; The map is updated based on the at least one changed location box.

2. The map updating method according to claim 1, wherein: The determining of distinguishing feature information between the current image and the historical image at multiple scales includes: Performing feature extraction on the current image to obtain a first feature point set at multiple scales; Performing feature extraction on the historical image to obtain a second feature point set at multiple scales; determining a set of distinguishing points between a first set of feature points at the multiple scales and a second set of feature points at the multiple scales; Based on the set of distinguishing points, distinguishing feature information at multiple scales is generated.

3. The map updating method according to claim 1 or 2, characterized in that: The determining, based on the target feature points, at least one change positioning frame of the historical image relative to the current image comprises: At least one change positioning frame of a preset shape is determined based on the target feature point.

4. The map updating method according to claim 3, wherein: The shape of the change positioning frame is a rectangle; and the step of determining at least one change positioning frame of a preset shape based on the target feature points includes: Determining the category and margin data corresponding to each target feature point respectively; Based on the category and the margin data, a rectangular change positioning frame corresponding to each target feature point is determined respectively.

5. The map updating method according to claim 1, wherein: The updating of the map based on the at least one changed positioning frame includes: determining a change profile based on each of the change positioning frames; receiving correction information, and correcting the changed contour based on the correction information to generate feature contour data; The map is updated according to the feature contour data.

6. The map updating method according to claim 1, wherein: The twin network is trained in the following way: Acquire sample images of the same sample area at different times; the sample images are provided with corresponding sample positioning frames; Inputting the sample image into the initial twin network to obtain a real-time positioning frame label output by the initial twin network; Determine a multi-branch loss function based on the sample positioning frame and the positioning frame label; The parameters of the initial twin network are adjusted based on the multi-branch loss function to obtain the twin network.

7. A map updating device, characterized in that: include: An acquisition module is used to obtain the current image and historical images of the map corresponding to the area to be detected; an extraction module, configured to determine distinguishing feature information between the current image and the historical image at multiple scales based on the trained Siamese network; a determination module configured to perform a multi-scale, hierarchical upsampling operation on the distinguishing feature information at multiple scales to obtain fused feature information at multiple levels, wherein the fused feature information at different levels has different receptive fields; obtain at least one target feature point based on the fused feature information at the multiple levels; and determine at least one change positioning frame of the historical image relative to the current image based on the target feature point; An updating module is configured to update the map based on the at least one changed positioning frame.

8. The device according to claim 7, characterized in that When determining the distinguishing feature information between the current image and the historical image at multiple scales, the extraction module is specifically configured to: Performing feature extraction on the current image to obtain a first feature point set at multiple scales; Performing feature extraction on the historical image to obtain a second feature point set at multiple scales; determining a set of distinguishing points between a first set of feature points at the multiple scales and a second set of feature points at the multiple scales; Based on the set of distinguishing points, distinguishing feature information at multiple scales is generated.

9. The device according to claim 7 or 8, characterized in that The determination module includes: The second determining unit is configured to determine at least one change positioning frame of a preset shape based on the target feature point.

10. The device according to claim 9, characterized in that When determining the location of the change of at least one preset shape based on the target feature point, the second determining unit is specifically configured to: Determining the category and margin data corresponding to each target feature point respectively; Based on the category and the margin data, a rectangular change positioning frame corresponding to each target feature point is determined respectively.

11. The device according to claim 7, characterized in that When updating the map based on the at least one changed positioning frame, the updating module is specifically configured to: determining a change profile based on each of the change positioning frames; receiving correction information, and correcting the changed contour based on the correction information to generate feature contour data; The map is updated according to the feature contour data.

12. The device according to claim 7, characterized in that The apparatus further comprises a training module, wherein the training module is configured to: Acquire sample images of the same sample area at different times; the sample images are provided with corresponding sample positioning frames; Inputting the sample image into the initial twin network to obtain a real-time positioning frame label output by the initial twin network; Determine a multi-branch loss function based on the sample positioning frame and the positioning frame label; The parameters of the initial twin network are adjusted based on the multi-branch loss function to obtain the twin network.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the map updating method according to any one of claims 1 to 6 is implemented.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the map updating method according to any one of claims 1 to 6.

Images