Method, device and readable storage medium for training a map feature mining network model

Abstract

The application discloses a kind of training map element mining network model method, device and readable storage medium.Map element mining network model is used to perceive map element in photo, method includes obtaining initial photo for marking high-precision map element;Initial photo is segmented into multiple picture blocks;According to preset proportion, multiple picture blocks are randomly occluded to obtain occlusion area and visible area;According to visible area, occlusion area is completed to obtain completed picture;When the pixel value of the same position of completed picture and initial photo exists difference value, difference value is used to adjust map element mining network model, and the difference value of the pixel value of the same position of the picture completed by the map element mining network model after adjustment and initial photo is less than set value, can save the cost of manual labeling map element, and make the map element mining network model after training can accurately perceive the new and unusual map element in photo.

Description

Technical Field

[0001] This specification relates to the field of high-precision mapping technology, and in particular to a method, apparatus, and readable storage medium for training a map feature mining network model. Background Technology

[0002] Currently, the creation of high-precision maps requires image acquisition equipment (such as handheld cameras, vehicle dashcams, or vehicles equipped with cameras) to continuously collect image information of roads and their surroundings. Then, workers annotate the map features in the images, and the annotated images are used to train a map feature mining network model. After training, the trained network model identifies map features in the images. However, manually annotating map features in images is not only labor-intensive, but also often lacks relevant annotation data for new and uncommon map features, causing the trained network model to be unable to accurately perceive these new and uncommon features. Summary of the Invention

[0003] In view of this, embodiments of this specification provide a method, apparatus, and readable storage medium for training a map feature mining network model, which addresses the problems of high manual costs in training map feature mining network models and the inability of trained map feature mining network models to accurately perceive newly added and uncommon map features in the prior art.

[0004] The embodiments in this specification adopt the following technical solutions:

[0005] This specification provides an embodiment of a method for training a map feature mining network model, wherein the map feature mining network model is used to perceive map features in a photograph, and the method includes:

[0006] Obtain initial photographs for labeling high-precision map features;

[0007] The initial photo is divided into multiple image blocks;

[0008] According to a preset ratio, multiple image blocks are randomly occluded to obtain occluded and visible areas;

[0009] The occluded area is filled in based on the visible area to obtain a completed image;

[0010] When there is a difference in pixel values ​​at the same position between the completed image and the initial photo, the difference value is used to adjust the map feature mining network model. The difference in pixel values ​​at the same position between the completed image and the initial photo after adjustment is less than a set value.

[0011] Optionally, the step of completing the occluded area based on the visible area to obtain a completed image includes:

[0012] For each image block in the visible region: calculate the self-attention of the image block with respect to other image blocks in the visible region, and sum the obtained attentions to obtain the self-attention of the image block;

[0013] The occluded area is filled in using the self-attention of each image block in the visible area to obtain a completed image.

[0014] Optionally, adjusting the map feature mining network model using the difference values ​​includes:

[0015] The product of the difference value, the preset learning rate of the map feature mining network model, and the gradient of the mapping function after the map feature mining network model is trained is determined; wherein, the mapping function after the map feature mining network model is trained is used to complete the occluded area according to the visible area to obtain a completed image.

[0016] The adjusted parameters of the map feature mining network model are obtained by subtracting the product from the parameters of the trained model.

[0017] Optionally, subtracting the product from the trained parameters of the map feature mining network model to obtain the adjusted parameters of the map feature mining network model includes:

[0018] The adjusted parameters of the map feature mining network model are obtained by the following formula (1);

[0019] θ t+1 =θ t -δ*L*F'(θ t (1)

[0020] Where δ represents the preset learning rate, L represents the difference value, and θ t+1 This represents the adjusted parameter, θ. t F'(θ) represents the parameters of the trained map feature mining network model. t ) represents the gradient of the mapping function after the map feature mining network model has been trained.

[0021] Optionally, dividing the initial photo into multiple image blocks includes:

[0022] The initial photo is divided into multiple image blocks of the same size.

[0023] Optionally, the shape of the image block is a regular graphic.

[0024] This specification also provides an embodiment of a method for perceiving map features in a photograph, applied to the adjusted map feature mining network model described in any of the above-mentioned embodiments, the method comprising:

[0025] Obtain initial photographs for labeling high-precision map features;

[0026] The initial photo is divided into multiple image blocks;

[0027] Calculate the self-attention of each of the image blocks;

[0028] Map elements in the initial photograph are perceived based on the self-attention of each of the image blocks;

[0029] Generate a self-attention heatmap of the map features in the initial photograph.

[0030] This specification also provides an apparatus for training a map feature mining network model, the map feature mining network model being used to perceive map features in a photograph, the apparatus comprising:

[0031] The photo acquisition module is used to acquire initial photos for labeling high-precision map features;

[0032] The photo segmentation module is used to segment the initial photo into multiple image blocks;

[0033] The photo occlusion module is used to randomly occlude multiple image blocks according to a preset ratio to obtain occlusion areas and visible areas;

[0034] The photo completion module is used to complete the occluded area based on the visible area to obtain a completed image;

[0035] The adjustment module is used to adjust the map feature mining network model when there is a difference in pixel values ​​at the same position between the completed image and the initial photo. The difference in pixel values ​​at the same position between the completed image and the initial photo is less than a set value after adjustment.

[0036] This specification also provides a non-volatile computer-readable storage medium storing program instructions or a computer program thereon; and / or, a computer program that, when executed by a processor, implements the method for training a map feature mining network model as described above.

[0037] This specification also provides a computer device including a memory and a processor, wherein the memory stores a control program, which, when executed by the processor, is used to implement the method for training a map feature mining network model according to any of the preceding claims.

[0038] The above-described at least one technical solution adopted in the embodiments of this specification can achieve the following beneficial effects:

[0039] The process involves: acquiring initial photos for labeling high-precision map features; dividing the initial photos into multiple image blocks; randomly occluding multiple image blocks according to a preset ratio to obtain occluded and visible areas; completing the occluded areas based on the visible areas to obtain a completed image; and adjusting the map feature mining network model based on the difference in pixel values ​​at the same location between the completed image and the initial photo when there is a difference in pixel values ​​at the same location. The adjusted map feature mining network model's completed image has a pixel value difference of less than a set value at the same location as the initial photo. This embodiment can significantly reduce the cost of manual map feature labeling and enables the trained map feature mining network model to accurately perceive newly added and uncommon map features in the photos. Attached Figure Description

[0040] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0041] Figure 1 This is a flowchart of a method for training a map feature mining network model in one embodiment of this specification;

[0042] Figure 2 This is a schematic diagram of an embodiment of the present specification before the initial photograph is obscured;

[0043] Figure 3 for Figure 2 A diagram showing the initial photo after it has been obscured;

[0044] Figure 4 for Figure 3 A diagram illustrating the completion of the initial photo after occlusion.

[0045] Figure 5 This is a schematic diagram of an initial photograph in one embodiment of this specification;

[0046] Figure 6 for Figure 5 A schematic diagram of the self-attention heatmap of the initial image;

[0047] Figure 7 This is a schematic diagram of an apparatus for training a map feature mining network model in one embodiment of this specification;

[0048] Figure 8 This is a schematic diagram of a computer device in one embodiment of this specification. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0050] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0051] Figure 1 This is a flowchart illustrating a method for training a map feature mining network model in one embodiment of this specification. The map feature mining network model is a type of neural network model used to perceive map features in a photograph. Map features may include, but are not limited to, lane lines, road signs, traffic lights, and traffic signs, etc. See also... Figure 1 The method is shown below and may include the following steps:

[0052] Step S102: Obtain the initial photographs for labeling high-precision map features;

[0053] In this step, high-definition maps (HD maps) are typically used for autonomous driving assistance. Generally, ordinary navigation maps have an accuracy error of at least 5-10 meters, while high-definition maps have centimeter-level accuracy. High-definition maps offer richer road elements and traffic-related dynamic elements, such as detailed lane lines, road signs, traffic signs, traffic lights, lane curvature, slope, and real-time lane-level traffic dynamic information. This primarily serves autonomous driving environment assessment, decision-making, and control. Ordinary navigation only provides road-level road elements; besides the accuracy and rationality of the navigation route, users prioritize visual effects and voice guidance experience. Initial photos can be captured using image acquisition devices, which can include handheld cameras, in-vehicle dashcams, and vehicles equipped with cameras. Using initial photos that do not require manual annotation significantly saves on the cost of manual annotation. Therefore, a large number of initial photos can be used to train the map feature mining network model, making the trained map feature mining network model more likely to "discover" the co-occurrence relationship of pixels between new and uncommon map features and other common map features in the same photo. By utilizing the "pixel co-occurrence relationship" feature of new and uncommon map features and other common map features, the model can accurately perceive new and uncommon map features.

[0054] Step S104: Divide the initial photo into multiple image blocks.

[0055] In this step, the image block can be a regular shape. A regular shape means that each side and each angle is equal; it can also be understood as a centrally symmetrical shape. For example, the image block can be a square, a rectangle, an equilateral triangle, a regular pentagon, or other shapes. Having a regular shape for the image block simplifies subsequent steps of occlusion according to a preset ratio and completing the occluded area based on the visible area.

[0056] Step S106: According to a preset ratio, randomly occlude multiple image blocks to obtain the occluded area and the visible area;

[0057] In this step, the preset ratio can be set, for example, to 75%, 70%, 65%, 55%, 45%, or 30%. Generally, the same preset ratio can be selected for each training session, although different preset ratios can also be selected for each training session. See [link / reference] for more information. Figure 2 and Figure 3 , Figure 2 This is a schematic diagram of the initial photograph before it is obscured in one embodiment of this specification. Figure 3 for Figure 2 A schematic diagram of the initial photo after it has been occluded. Assume... Figure 2 The resolution is 448*448; it is divided into 16*16=256 identical image blocks, each with a resolution of 28*28. Figure 3 This is the result after randomly masking 75% of the image patches. Generally, multiple image patches are randomly masked according to a preset ratio. This can be understood as masking a predetermined percentage of complete image patches. For example, if there are 100 image patches and the preset ratio is 75%, then 75 image patches will be masked. Typically, some image patches will not be completely masked while others are only partially masked. This masking method enables the trained map feature mining network model to more accurately perceive map features in the photo.

[0058] Step S108: Complete the occluded area based on the visible area to obtain the completed image;

[0059] Specifically, in this step, please refer to Figure 4 , Figure 4 for Figure 3 A schematic diagram of initial photo completion after occlusion. This step enables the trained map feature mining network model to learn the relationships between a large number of "image patches" in an image: for example, it can perceive traffic lights and also find the "pedestrian crossing" area in the images used for training; then, for a new image with traffic lights, the "traffic light" element naturally attracts attention to the "pedestrian crossing".

[0060] Step S110: When there is a difference in pixel values ​​at the same position between the completed image and the initial photo, the difference value is used to adjust the map feature mining network model. The difference value between the pixel values ​​at the same position between the completed image and the initial photo is less than the set value after adjustment.

[0061] In this step, if the difference in pixel values ​​at the same location between the image completed by the adjusted map feature mining network model and the initial photo is less than a set value, it means that the pixel values ​​at the same location in the completed image and the initial photo are accurate, which means that the completed image and the initial photo are consistent at the same location.

[0062] In this embodiment, an initial photograph for labeling high-precision map features is obtained; the initial photograph is divided into multiple image blocks; multiple image blocks are randomly occluded according to a preset ratio to obtain occluded and visible areas; the occluded areas are completed based on the visible areas to obtain a completed image; when there is a difference in pixel values ​​at the same position between the completed image and the initial photograph, the difference value is used to adjust the map feature mining network model. The difference in pixel values ​​at the same position between the completed image and the initial photograph after adjustment is less than a set value. This embodiment can greatly save the cost of manually labeling map features and enable the trained map feature mining network model to accurately perceive newly added and uncommon map features in the photograph.

[0063] In the embodiments of this specification, the method of completing the occluded area based on the visible area to obtain a completed image may include:

[0064] For each image patch in the visible region: calculate the self-attention of the image patch with respect to other image patches in the visible region, and sum the obtained attentions to get the self-attention of the image patch;

[0065] The occluded area is filled in by using the self-attention of each image block in the visible area to obtain the completed image.

[0066] In this embodiment, self-attention is defined as follows: the map feature mining network model can calculate an "attention" coefficient for each pixel in the photo based on the pixel feature distribution of the photo itself. Generally, the weight coefficient of the image region where the map feature is located in a high-precision map is higher than the weight coefficient of other regions, which facilitates subsequent heatmap visualization of the image. Regions with high weight coefficients are also more highlighted after visualization. For each image block in the visible region: calculate the self-attention of the image block with respect to other image blocks in the visible region, and sum the obtained attentions to obtain the self-attention of the image block; based on the self-attention of each image block in the visible region, the occluded region is completed to obtain the completed image, so that the trained map feature mining network model has a greater chance of "discovering" the co-occurrence relationship of pixels between newly added and uncommon map features and other common map features in the same photo, thereby accurately perceiving newly added and uncommon map features by utilizing the "pixel co-occurrence relationship" feature of newly added and uncommon map features and other common map features.

[0067] In the embodiments of this specification, adjusting the map feature mining network model using difference values ​​may include:

[0068] The product of the difference value, the preset learning rate of the map feature mining network model, and the gradient of the mapping function after the map feature mining network model is trained is determined; wherein, the mapping function after the map feature mining network model is trained is used to complete the occluded area based on the visible area to obtain the completed image.

[0069] The adjusted parameters of the map feature mining network model are obtained by subtracting the product from the parameters trained on the model.

[0070] In this embodiment, the product of the difference value, the preset learning rate of the map feature mining network model, and the gradient of the mapping function after the map feature mining network model is determined. The adjusted parameters of the map feature mining network model are obtained by subtracting the product from the parameters after the map feature mining network model is trained, so that the adjusted map feature mining network model can accurately perceive the map features in the photo.

[0071] In the embodiments of this specification, subtracting the product from the parameters trained on the map feature mining network model to obtain the adjusted parameters of the map feature mining network model may include:

[0072] The adjusted parameters of the map feature mining network model are obtained by the following formula (1);

[0073] θ t+1 =θ t -δ*L*F'(θ t (1)

[0074] Where δ represents the preset learning rate, L represents the difference value, and θ t+1 This represents the adjusted parameter, θ. t F'(θ) represents the parameters of the map feature mining network model after training. t ) represents the gradient of the mapping function after the map feature mining network model has been trained.

[0075] In the embodiments of this specification, dividing the initial photograph into multiple image blocks may include:

[0076] The initial photo is divided into multiple image blocks of the same size.

[0077] In this embodiment, the initial photo is divided into multiple image blocks of the same size, which makes it easier to convert each block into a feature vector, facilitates the sharing of the same parameters among the image blocks, makes it easier to calculate the self-attention of each image block in the visible area, and makes it easier to complete the occluded area to obtain the completed image based on the self-attention of each image block in the visible area.

[0078] In the embodiments of this specification, after adjusting the map feature mining network model using difference values, the following may also be included:

[0079] Calculate the self-attention of each image patch;

[0080] Based on the self-attention perception of each image patch, map elements in the initial photo are perceived;

[0081] Generate a self-attention heatmap of map features in the initial photograph.

[0082] In this embodiment, after adjusting the map feature mining network model using difference values, for the initial photograph used to label high-precision map features, after the initial photograph is segmented into multiple image blocks, the self-attention of each segmented image block can be calculated. Based on the self-attention of each image block, the map features in the initial photograph can be perceived, accurately identifying common, uncommon, and newly added map features in the photograph. A self-attention heatmap of map features in the initial photograph can also be generated, providing a visual representation of the map features in the photograph and facilitating subsequent selection and labeling of map features in the photograph. Specifically, see... Figure 5 and Figure 6 , Figure 5 This is a schematic diagram of an initial photograph in one embodiment of this specification; Figure 6 for Figure 5 A schematic diagram of the self-attention heatmap of the initial image.

[0083] The following is a detailed description of this solution through a specific embodiment:

[0084] Step 1: Obtain initial photos for labeling high-precision map features. The set of initial photos can be represented by Z; the initialization parameters of the mapping function (map feature mining network model) can be represented by θ, and the initial map feature mining network model can output the pixel parameters of the photos.

[0085] Step 2: Divide the initial photo into multiple image blocks of the same size. For example, obtain an initial photo Z of size x*y. k ∈R x×y Divide the image into m*n blocks. In this m*n matrix, define the image block at position (i, j) as O(n). i，j O i，j This can be expressed by the following formula (2).

[0086] Where i≤m&j≤n; (2)

[0087] Step 3: According to a preset ratio, randomly occlude multiple image blocks to obtain the occluded area and the visible area. Specifically, randomly occlude p% of the image blocks; that is, only retain m*n*(100%-p%) of the image blocks.

[0088] Step 4: Complete the occluded area based on the visible area to obtain the completed image, which is to predict the occluded area based on the visible area. Specifically, use the visible Z-axis... k *(100%-p%), through the mapping function F(θ) t The masked p% image block is "completed" to obtain the completed image Z'. k The specific completion method is shown in formula (3) below.

[0089] Z′ k =Z k *(100% - p%)*F(θ) t (3)

[0090] Where, θ t This represents the parameters of the map feature mining network model at the end of the t-th training round. The occluded region is filled in based on the visible region to obtain the completed image. Specifically, for each image block in the visible region: the self-attention of that image block with respect to other image blocks in the visible region is calculated, and the individual attentions are summed to obtain the self-attention of that image block; then, the occluded region is filled in based on the self-attention of each image block in the visible region to obtain the completed image. Assume the overall parameters of the map feature mining network model are θ. t It consists of m*n different image blocks (O i，j The corresponding region parameter is Ω. i，j Define image block O. i，j After passing through other image blocks (such as: O) m，nThe self-attention calculation is used to obtain image block O. i，j Self-attention O′ i，j Self-attention O′ i，j This is expressed by the following formula (4),

[0091] O′ i，j =∑ m，n O i，j ×Softmax(O i，j Ω i，j O m，n (4)

[0092] Z′ k To stitch all O's together according to the order of the image blocks. i，j The result.

[0093] Step 5: When there are differences in pixel values ​​at the same location between the completed image and the initial photo, the parameters of the map feature mining network model are adjusted using the differences. Specifically, the differences between all the "completed" images and the corresponding original images are calculated using formula (5).

[0094] Difference L=∑||Z′ k -Z k || (5)

[0095] Then, the map feature mining network model is adjusted using formula (1) from the above embodiments.

[0096] Step Six: After the map feature mining network model training round is completed, the initial photo is obtained and segmented into multiple image patches of the same size using the map feature mining network model. For each image patch, the self-attention of the other image patches is determined, and the self-attention of the other image patches is superimposed to obtain the self-attention of the target image patch, which is obtained by using the function Softmax(O) i,j ,Ω i,j O m,n The self-attention of the image patches is obtained. The result after stacking is a numerical matrix with the exact same size as the original image, where each element represents "self-attention".

[0097] Step 7: Output a self-attention heatmap of the initial photo based on the self-attention of each image patch in the initial photo. Generally, map features have higher weight coefficients, and the heatmap effect of map features in the self-attention heatmap is relatively brighter than other areas.

[0098] Based on the same concept, one embodiment of this specification also provides a method for perceiving map features in a photograph, which can be applied to the adjusted map feature mining network model described above. The method may include:

[0099] Obtain initial photographs for labeling high-precision map features;

[0100] The initial photo is divided into multiple image blocks;

[0101] Calculate the self-attention of each image patch;

[0102] Based on the self-attention perception of each image patch, map elements in the initial photo are perceived;

[0103] Generate a self-attention heatmap of map features in the initial photograph.

[0104] The solution in this embodiment can accurately perceive common, uncommon, and newly added map elements in the photo. It can also generate a self-attention heatmap of map elements in the initial photo, which can visually display the map elements in the photo and is also beneficial for subsequent selection and annotation of map elements in the photo.

[0105] Figure 7 This is a schematic diagram of an apparatus for training a map feature mining network model according to one embodiment of this specification. Based on the same concept, the present invention also provides an apparatus for training a map feature mining network model. The map feature mining network model is used to perceive map features in a photograph. The apparatus includes a photograph acquisition module 601, a photograph segmentation module 602, a photograph occlusion module 603, a photograph completion module 604, and an adjustment module 605.

[0106] The photo acquisition module 601 is used to acquire initial photos for labeling high-precision map features;

[0107] The photo segmentation module 602 is used to segment the initial photo into multiple image blocks;

[0108] The photo occlusion module 603 is used to randomly occlude multiple image blocks according to a preset ratio to obtain the occluded area and the visible area;

[0109] The photo completion module 604 is used to complete the occluded area based on the visible area to obtain a completed image;

[0110] The adjustment module 605 is used to adjust the map feature mining network model when there is a difference in pixel values ​​at the same position between the completed image and the initial photo. The difference in pixel values ​​at the same position between the completed image and the initial photo is less than a set value after adjustment.

[0111] Based on the same concept, the present invention also provides a non-volatile computer-readable storage medium having program instructions or a computer program stored thereon; and / or a computer program, wherein when the program instructions or the computer program are executed by a processor, a method for training a map feature mining network model according to any of the above embodiments is provided.

[0112] Figure 8 This is a schematic diagram of a computer device according to one embodiment of this specification. Based on the same concept, the present invention also provides a computer device 800, which may include a memory 830 and a processor 810. The memory 830 stores a control program 820, which, when executed by the processor 810, is used to implement the method for training a map feature mining network model according to any of the above embodiments.

[0113] It should be noted that the apparatus provided in this application corresponds one-to-one with the method provided in this application. Therefore, the apparatus also has similar beneficial technical effects as the method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the apparatus will not be repeated here. The various embodiments described above in this application can be combined arbitrarily.

[0114] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using a hardware physical module. For example, a Programmable Logic Device (PLD) (e.g., a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program a digital system themselves to "integrate" it onto a PLD, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0115] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0116] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0117] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0118] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0119] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0120] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0121] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0122] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0123] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0124] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0125] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0126] This application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0127] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0128] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for training a map feature mining network model, characterized in that, The map feature mining network model is used to perceive map features in a photograph, and the method includes: Obtain initial photographs for labeling high-precision map features; The initial photo is divided into multiple image blocks; According to a preset ratio, multiple image blocks are randomly occluded to obtain occluded and visible areas; The occluded area is filled in based on the visible area to obtain a completed image; When there is a difference in pixel value at the same position between the completed image and the initial photo, the difference value is used to adjust the map feature mining network model. The difference value between the pixel value at the same position between the completed image and the initial photo is less than a set value after adjustment. The adjustment of the map feature mining network model using the difference values ​​includes: The parameter update amount of the map feature mining network model is determined based on the difference value; The adjusted parameters of the map feature mining network model are obtained by subtracting the parameter update amount of the map feature mining network model from the parameters after training.

2. The method according to claim 1, characterized in that, The step of completing the occluded area based on the visible area to obtain the completed image includes: For each image block in the visible region: calculate the self-attention of the image block with respect to other image blocks in the visible region, and sum the obtained attentions to obtain the self-attention of the image block; The occluded area is filled in using the self-attention of each image block in the visible area to obtain a completed image.

3. The method according to claim 1, characterized in that, Determining the parameter update amount of the map feature mining network model based on the difference value includes: determining the product of the difference value, the preset learning rate of the neural network, and the gradient of the mapping function after the neural network is trained; the product is the parameter update amount of the map feature mining network model; wherein, the mapping function after the neural network is trained is used to complete the occluded area based on the visible area to obtain a completed image.

4. The method according to claim 3, characterized in that, The step of subtracting the product from the trained parameters of the map feature mining network model to obtain the adjusted parameters of the map feature mining network model includes: The adjusted parameters of the map feature mining network model are obtained by the following formula (1); （1） in, L represents the preset learning rate, and L represents the difference value. This indicates the adjusted parameters. This represents the parameters of the map feature mining network model after training. This represents the gradient of the mapping function after the neural network has been trained.

5. The method according to claim 1, characterized in that, The step of dividing the initial photo into multiple image blocks includes: The initial photo is divided into multiple image blocks of the same size.

6. The method according to claim 1, characterized in that, The shape of the image block is a regular graphic.

7. A method for perceiving map elements in a photograph, characterized in that, The method, applied to the adjusted map feature mining network model according to any one of claims 1 to 6, comprises: Obtain initial photographs for labeling high-precision map features; The initial photo is divided into multiple image blocks; Calculate the self-attention of each of the image blocks; Map elements in the initial photograph are perceived based on the self-attention of each of the image blocks; Generate a self-attention heatmap of the map features in the initial photograph.

8. An apparatus for training a map feature mining network model, characterized in that, The map feature mining network model is used to perceive map features in a photograph, and the device includes: The photo acquisition module is used to acquire initial photos for labeling high-precision map features; The photo segmentation module is used to segment the initial photo into multiple image blocks; The photo occlusion module is used to randomly occlude multiple image blocks according to a preset ratio to obtain occlusion areas and visible areas; The photo completion module is used to complete the occluded area based on the visible area to obtain a completed image; The adjustment module is used to adjust the map feature mining network model when there is a difference in pixel values ​​at the same position between the completed image and the initial photo. The difference in pixel values ​​at the same position between the completed image and the initial photo after adjustment is less than a set value. The adjustment module is specifically used to determine the parameter update amount of the map feature mining network model based on the difference value; and to obtain the adjusted parameters of the map feature mining network model by subtracting the parameter update amount of the map feature mining network model from the parameters after training.

9. A non-volatile computer-readable storage medium having program instructions or a computer program stored thereon; and / or, a computer program, characterized in that, When the program instructions or computer program are executed by a processor, the method for training a map feature mining network model as described in any one of claims 1 to 6 is implemented.

10. A computer device, comprising a memory and a processor, characterized in that, The memory stores a control program, which, when executed by the processor, is used to implement the method for training a map feature mining network model according to any one of claims 1 to 6.

Images