Model training method, map drawing method, device, computer equipment and medium

Abstract

The present disclosure provides a model training method and device, the method comprising: constructing an initial network model, including: a backbone network, a region proposal network, a mask network and a recognition network based on an attention mechanism. The backbone network performs feature extraction on the input image to obtain a feature map. The region proposal network is used to generate a candidate target frame. The mask network obtains a mask prediction result based on the feature map and the candidate target frame. The recognition network obtains a classification prediction result and an outer box prediction result based on the above-mentioned feature map, the candidate target frame and the mask prediction result. Obtain multiple training images and labels of each of the multiple training images, and the label of any training image includes: the category, outer box and mask of the target object in any of the training images. Use the multiple training images and the labels of each of the multiple training images to train the initial network model to obtain a target network model. The present disclosure also provides a map drawing method and device, a computer device and a medium.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer technology, and more specifically, to a model training method and device, a map drawing method and device, a computer device, and a medium. Background Art

[0002] In the production process of high-precision maps, various target objects on the road need to be drawn.

[0003] In the process of realizing the present invention, the inventors found that there are at least the following problems in the prior art: if various target objects are manually identified, extracted and drawn from road pictures, it will cost a lot of labor costs and the efficiency is extremely low, and an automated alternative is urgently needed. However, in the existing automated recognition solutions for target objects, there are often problems such as the positioning deviation of the outer frame of the target object due to the influence of the background area, which in turn leads to the recognition deviation of the target object. Summary of the invention

[0004] In view of this, embodiments of the present invention provide a model training method and apparatus, a map drawing method and apparatus, a computer device and a medium to more accurately identify and segment target objects in road images, thereby achieving high-precision map drawing containing various target objects.

[0005] One aspect of an embodiment of the present invention provides a model training method, including: on the one hand, constructing an initial network model. The initial network model includes: a backbone network, a region proposal network, a mask network and a recognition network based on an attention mechanism. Among them, the backbone network is used to extract features from the input image to obtain a feature map. The region proposal network is used to generate a candidate target frame. The mask network is used to obtain a mask prediction result based on the feature map and the candidate target frame. The recognition network is used to obtain a classification prediction result and an outer box prediction result based on the above-mentioned feature map, the candidate target frame and the mask prediction result. On the other hand, a plurality of training images and respective labels of the plurality of training images are obtained. Among them, the label of any training image among the obtained plurality of training images includes: the category, outer box and mask of the target object in the any training image. Then, the initial network model is trained using the plurality of training images and the respective labels of the plurality of training images to obtain a target network model.

[0006] According to an embodiment of the present invention, the above-mentioned use of multiple training pictures and the labels of each of the multiple training pictures to train the initial network model includes: for any training picture among the multiple training pictures, the any training picture is input into the backbone network so that the backbone network outputs a first feature map for the any training picture. And the any training picture is input into the region proposal network so that the region proposal network outputs a first candidate target box for the any training picture. Then, the first candidate target box is applied to the first feature map to obtain a first region of interest feature map. Then, the first region of interest feature map is input into the mask network so that the mask network outputs a mask prediction result for the any training picture. Then, based on the mask prediction result for the any training picture and the label of the any training picture, the loss value of the first function is calculated. Therefore, based on the loss value of the first function, the parameters of at least one of the backbone network, the region proposal network and the mask network can be adjusted until the first function converges.

[0007] According to an embodiment of the present invention, the mask prediction result for any training image includes multiple mask prediction results. The calculation of the loss value of the first function based on the mask prediction result of any training image and the label of any training image includes: determining the category and mask of any training image according to the label of any training image. Then, based on the mask of any training image and the first function, the loss value of the mask prediction result corresponding to the category of any training image among the multiple mask prediction results for any training image is calculated.

[0008] According to an embodiment of the present invention, the above-mentioned use of multiple training images and the labels of each of the multiple training images to train the initial network model also includes: based on the mask of any training image, the multiple mask prediction results for the any training image and the first function, calculating the loss value of each of the multiple mask prediction results. Then, based on the loss value of each of the multiple mask prediction results, a confidence map is converted. Then, the confidence map and the first region of interest feature map are input into the recognition network so that the recognition network outputs the classification prediction result and the outer box prediction result for the any training image. Next, based on the classification prediction result, the outer box prediction result and the label of the any training image, the loss value of the second function is calculated. Then, based on the loss value of the second function, the parameters of the recognition network are adjusted until the second function converges.

[0009] According to an embodiment of the present invention, the above-mentioned conversion based on the loss values ​​of each of the multiple mask prediction results to obtain a confidence map includes: for any mask prediction result among the multiple mask prediction results, the loss value of each pixel in the any mask prediction result is inverted and added by 1 to obtain a confidence value of each pixel. Then, based on the confidence value of each pixel in the any mask prediction result, the confidence map of the any mask prediction result is determined.

[0010] According to an embodiment of the present invention, the method further comprises: before inputting the first region of interest feature map into the mask network, performing scale normalization processing on the first region of interest feature map. And / or, before inputting the first region of interest feature map into the recognition network, performing scale normalization processing on the first region of interest feature map.

[0011] According to an embodiment of the present invention, the method further includes: before the confidence map and the first region of interest feature map are input into the recognition network, the confidence map is subjected to dimensionality reduction processing. The inputting of the confidence map and the first region of interest feature map into the recognition network includes: accumulating the confidence map subjected to dimensionality reduction processing and the first region of interest feature map in the channel dimension to obtain a first feature map to be recognized. Then the first feature map to be recognized is input into the recognition network.

[0012] Another aspect of an embodiment of the present invention provides a map drawing method, comprising: on the one hand, obtaining a road picture. On the other hand, obtaining a target network model trained according to the model training method described in any of the above embodiments. Then, the road picture is processed using the target network model so that: the mask network in the target network model outputs the mask of the target object in the road picture, and the recognition network based on the attention mechanism in the target network model outputs the category and outer box of the target object in the road picture. Then, based on the mask, category and outer box of the target object in the road picture, the target object is drawn in the map.

[0013] According to an embodiment of the present invention, the above-mentioned processing of the road image using the target network model includes: inputting the road image into the backbone network of the target network model so that the backbone network outputs a second feature map for the road image, and inputting the road image into the region proposal network of the target network model so that the region proposal network outputs a second candidate target frame for the road image. Then, the second feature map of the road image is input into the mask network so that the mask network outputs a mask prediction result for the road image. The second candidate target frame is applied to the second feature map and the mask prediction result for the road image respectively to obtain a second region of interest feature map and a region of interest mask prediction result. Next, the second region of interest feature map and the region of interest mask prediction result are input into the recognition network so that the recognition network outputs the category and outer box of the target object in the road image.

[0014] According to an embodiment of the present invention, the method further includes: before the second region of interest feature map and the region of interest mask prediction result are input into the recognition network, the second region of interest feature map and the region of interest mask prediction result are respectively scale normalized. The input of the second region of interest feature map and the region of interest mask prediction result into the recognition network includes: accumulating the second region of interest feature map and the region of interest mask prediction result after scale normalization in the channel dimension to obtain a second feature map to be identified for the road image. Then, the second feature map to be identified is input into the recognition network so that the recognition network outputs the category and outer bounding box of the target object in combination with the non-maximum suppression algorithm.

[0015] According to an embodiment of the present invention, the method further includes: based on the category of the target object in the road picture, determining a mask prediction result corresponding to the category of the target object from the mask prediction results for the road picture as a mask of the target object in the road picture.

[0016] According to an embodiment of the present invention, the target object includes: a lane marking line.

[0017] Another aspect of an embodiment of the present invention provides a model training device, including: a construction module, a sample acquisition module and a training module. The construction module is used to construct an initial network model. The initial network model includes: a backbone network, a region proposal network, a mask network and a recognition network based on an attention mechanism. Among them, the backbone network is used to extract features from the input image to obtain a feature map. The region proposal network is used to generate a candidate target frame. The mask network is used to obtain a mask prediction result based on the feature map and the candidate target frame. The recognition network is used to obtain a classification prediction result and an outer box prediction result based on the above feature map, the candidate target frame and the mask prediction result. The sample acquisition module is used to obtain multiple training images and their respective labels. Among them, the label of any training image among the multiple training images obtained includes: the category, outer box and mask of the target object in any training image. The training module is used to train the initial network model using multiple training images and their respective labels to obtain a target network model.

[0018] Another aspect of an embodiment of the present invention provides a map drawing device, including: a first acquisition module, a second acquisition module, a processing module and a drawing module. The first acquisition module is used to acquire a road picture. The second acquisition module is used to acquire a target network model trained according to the model training method described in any of the above embodiments. The processing module is used to process the road picture using the target network model so that: the mask network in the target network model outputs the mask of the target object in the road picture, and the recognition network based on the attention mechanism in the target network model outputs the category and outer box of the target object in the road picture. The drawing module is used to draw the target object in the map based on the mask, category and outer box of the target object in the road picture.

[0019] Another aspect of an embodiment of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above method when executing the program.

[0020] Another aspect of an embodiment of the present invention provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the method described above when executed.

[0021] Another aspect of an embodiment of the present invention provides a computer program, wherein the computer program includes computer executable instructions, and the instructions are used to implement the method described above when executed.

[0022] According to an embodiment of the present invention, a target network model is obtained by supervised training of an initial network model. The initial model is composed of a backbone network, a region proposal network, a mask network, and a recognition network based on an attention mechanism, and includes three output branches: an output branch of the mask network regarding mask prediction results and an output branch of the recognition network regarding classification prediction results and outer box prediction results. The attention mechanism is introduced into the recognition network, and the output of the mask network is used as part of the output of the recognition network, so that the recognition network can pay more attention to the spatial position distribution of the target object and locate the target object. In this way, a target network model with better performance can be trained to avoid positioning deviation of the target object and reduce recognition bias of the target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above and other objects, features and advantages of the embodiments of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0024] Figure 1 Schematically illustrates an exemplary system architecture of an application model training method and device and a map drawing method and device according to an embodiment of the present invention;

[0025] Figure 2 A flowchart of a model training method according to an embodiment of the present invention is schematically shown;

[0026] Figure 3 Schematically shows an embodiment of the present invention Figure 2 An example flow chart of operation S230;

[0027] Figure 4 Schematically shows an example structural diagram of an initial network model according to an embodiment of the present invention;

[0028] Figure 5 A flowchart of a map drawing method according to an embodiment of the present invention is schematically shown;

[0029] Figure 6 Schematically shows an embodiment of the present invention Figure 5 An example flow chart of operation S530;

[0030] Figure 7 An example structural diagram of a road picture according to an embodiment of the present invention is schematically shown;

[0031] Figure 8 A block diagram of a model training device according to an embodiment of the present invention is schematically shown;

[0032] Fig. 9 A block diagram schematically shows a map drawing device according to an embodiment of the present invention; and

[0033] Fig.10 A block diagram of a computer device according to an embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0034] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the embodiments of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the embodiments of the present invention.

[0035] The terms used herein are only for describing specific embodiments and are not intended to limit the embodiments of the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0036] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0037] In the case of using expressions such as "at least one of A, B, and C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.). In the case of using expressions such as "at least one of A, B, or C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, or C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0038] Embodiments of the present invention provide a model training method and device, a map drawing method and device, a computer device and a medium. The model training method may include a construction process, a sample acquisition process and a training process. In the construction process, an initial network model is constructed. The initial network model may include: a backbone network, a region proposal network, a mask network and a recognition network based on an attention mechanism. The backbone network is used to extract features from an input image to obtain a feature map. The region proposal network is used to generate a candidate target frame. The mask network is used to obtain a mask prediction result based on the feature map and the candidate target frame. The recognition network is used to obtain a classification prediction result and an outer box prediction result based on the feature map, the candidate target frame and the mask prediction result. In the sample acquisition process, multiple training images and labels of each of the multiple training images are obtained. The label of any training image in the multiple training images obtained includes: the category, outer box and mask of the target object in the any training image. Then, a training process is performed, and the initial network model constructed above is trained using the multiple training images obtained above and the labels of each of the multiple training images to obtain a target network model.

[0039] In the production process of high-precision maps, it is necessary to draw various target objects on the road. If various target objects are manually identified, extracted and drawn from road pictures, it will cost a lot of labor costs and be extremely inefficient, and an automated alternative is urgently needed. In one processing method, the target objects are identified and extracted based on object detection technology, such as R-CNN (Region Convolutional Neural Network), Fast R-CNN (Fast Region Convolutional Neural Network), Faster R-CNN (Faster Region Convolutional Neural Network), etc. In another processing method, the target objects are identified and extracted based on semantic segmentation technology, such as SCNN (Spatial Convolutional Neural Network). In another processing method, the target objects are identified and extracted based on instance segmentation technology, such as Mask R-CNN (Mask Region Convolutional Neural Network).

[0040] However, the bounding box position of the target object that can only be extracted based on the target detection technology is not suitable for higher precision positioning requirements, such as the inability to locate the specific outline and position of the target object more precisely. Although semantic segmentation technology can usually meet the requirements of pixel-level positioning and segmentation of target objects, there are problems such as discontinuity and insufficient segmentation of small target objects due to scale problems. Based on instance segmentation technology, the position of the bounding box of the target object can be determined first, and then the target object in the bounding box can be extracted and segmented. However, in the process of determining the bounding box, for slender target objects, more content in the bounding box is the background area. Due to the influence of the background area, the positioning of the bounding box of the target object deviates, which leads to the recognition deviation of the target object. Based on the many problems existing in the above related technologies, various types of target objects in road pictures cannot be accurately identified and extracted, which in turn affects the drawing of high-precision maps.

[0041] According to an embodiment of the present invention, a model training method and device and a map drawing method and device are provided. The model training method according to an embodiment of the present invention can train a target network model that can more accurately identify and extract various types of target objects. The map drawing method according to an embodiment of the present invention can use the target network model to more accurately identify and extract various target objects from road images to draw them in the map, so as to draw a map with higher accuracy.

[0042] Figure 1 The exemplary system architecture 100 to which the model training method and apparatus and the map drawing method and apparatus according to the embodiment of the present invention can be applied is schematically shown. It should be noted that: Figure 1 The examples shown are merely examples of system architectures to which the embodiments of the present invention may be applied, to help those skilled in the art understand the technical content of the embodiments of the present invention, but do not mean that the embodiments of the present invention may not be used in other devices, systems, environments or scenarios.

[0043] like Figure 1 As shown, the system architecture 100 according to the embodiment of the present invention may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0044] The terminal devices 101, 102, 103 communicate with the server 105 via the network 104 to receive or send messages, etc. The terminal devices 101, 102, 103 may be installed with client applications with various functions, such as navigation applications, music applications, shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0045] The terminal devices 101 , 102 , and 103 may be various electronic devices, including but not limited to vehicle navigation, smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0046] The server 105 may be a server that provides various services, such as a background management server that provides support for various client applications in the terminal devices 101, 102, and 103. The background management server may receive request messages sent by the terminal devices 101, 102, and 103, analyze and process the received request messages, and feed back the response results (such as web pages, information, or data generated by obtaining or processing the request messages) to the terminal devices 101, 102, and 103, and the terminal devices 101, 102, and 103 output these response results to the user.

[0047] It should be noted that the model training method according to the embodiment of the present invention can be implemented in the terminal devices 101, 102, and 103, and accordingly, the model training device according to the embodiment of the present invention can be set in the terminal devices 101, 102, and 103. Alternatively, the model training method according to the embodiment of the present invention can also be implemented in the server 105, and accordingly, the model training device according to the embodiment of the present invention can be set in the server 105. Alternatively, the model training method according to the embodiment of the present invention can also be implemented in other computer devices that can communicate with the terminal devices 101, 102, 103 and / or the server 105, and accordingly, the model training device according to the embodiment of the present invention can be set in other computer devices that can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0048] The map drawing method according to the embodiment of the present invention may be implemented in the terminal devices 101, 102, 103, and accordingly, the map drawing apparatus according to the embodiment of the present invention may be arranged in the terminal devices 101, 102, 103. Alternatively, the map drawing method according to the embodiment of the present invention may also be implemented in the server 105, and accordingly, the map drawing apparatus according to the embodiment of the present invention may be arranged in the server 105. Alternatively, the map drawing method according to the embodiment of the present invention may also be implemented in other computer devices capable of communicating with the terminal devices 101, 102, 103 and / or the server 105, and accordingly, the map drawing apparatus according to the embodiment of the present invention may be arranged in other computer devices capable of communicating with the terminal devices 101, 102, 103 and / or the server 105.

[0049] The model training method according to the embodiment of the present invention and the map drawing method according to the embodiment of the present invention can be implemented in the same computer device or in different computer devices, and there is no limitation here.

[0050] It should be understood that Figure 1 The number and type of terminal devices, networks and servers in the embodiment are only for illustration purposes. According to actual needs, any number and type of terminal devices, networks and servers may be provided.

[0051] According to an embodiment of the present invention, a model training method is provided. The method is exemplarily described below by means of a figure. It should be noted that the sequence number of each operation in the following method is only used as a representation of the operation for the purpose of description, and should not be regarded as indicating the execution order of the various operations. Unless explicitly stated, the method does not need to be executed in the order shown.

[0052] Figure 2 The flowchart of the model training method according to an embodiment of the present invention is schematically shown.

[0053] like Figure 2 As shown, the method may include operations S210 to S230.

[0054] In operation S210, an initial network model is constructed.

[0055] Exemplarily, the constructed initial network model may include: a backbone network, a region proposal network (RPN), a mask network, and a recognition network based on an attention mechanism. Among them, the backbone network is used to extract features from the input image to obtain a feature map. The region proposal network is used to generate a candidate target box (prebox), which represents the recommendation result of the outer box of the target object. The mask network is used to obtain a mask prediction result based on the feature map and candidate target box obtained above. The recognition network is used to obtain a classification prediction result and an outer box prediction result based on the feature map, candidate target box and mask prediction result obtained above. In this example, the classification prediction result is used to characterize the prediction result for the category of the target object, the outer box prediction result is used to characterize the prediction result for the location of the outer box of the target object, and the mask prediction result is used to characterize the prediction result for the mask of the target object. The mask of the target object, for example, is encoded in binary to more finely reflect the spatial position distribution of the target object.

[0056] Then, in operation S220, a plurality of training images and labels of the plurality of training images are obtained.

[0057] The label of any training image among the obtained multiple training images includes: a category (class), an outer bounding box and a mask of a target object in the training image.

[0058] Next, in operation S230, the initial network model is trained using the plurality of training images and the labels of the plurality of training images to obtain a target network model.

[0059] Those skilled in the art can understand that the model training method according to the embodiment of the present invention obtains the target network model through supervised training of the initial network model. Among them, the initial model is composed of a backbone network, a region proposal network, a mask network and a recognition network based on an attention mechanism, including three output branches: an output branch of the mask network about the mask prediction result and an output branch of the recognition network about the classification prediction result and the outer box prediction result. The attention mechanism is introduced into the recognition network, and the output of the mask network is used as part of the output of the recognition network, so that the recognition network can pay more attention to the spatial position distribution of the target object and locate the target object. In this way, a target network model with better performance can be trained to avoid positioning deviation of the target object and reduce the recognition bias of the target object.

[0060] Figure 3 Schematically shows an embodiment of the present invention Figure 2 An example flowchart of operation S230 in FIG.

[0061] like Figure 3 As shown, operation S230 may include the following operations to train the initial network model using multiple training images and their respective labels.

[0062] In operation S2301, for any training picture among a plurality of training pictures, the any training picture is input into a backbone network, so that the backbone network outputs a first feature map for the any training picture.

[0063] Exemplarily, the backbone network can be a convolutional neural network, for example, composed of at least one convolutional layer, at least one activation function (ReLU) layer and at least one pooling layer. The backbone network extracts a first feature map of any training image, and the first feature map is used in a subsequent region proposal network.

[0064] Then, in operation S2302, any one of the training images is input into a region proposal network, so that the region proposal network outputs a first candidate target box for any one of the training images.

[0065] Then, in operation S2303, the first candidate target box is applied to the first feature map to obtain a first region of interest (RoI) feature map.

[0066] In operation S2304, the first region of interest feature map is input into the mask network, so that the mask network outputs a mask prediction result for any training image.

[0067] Exemplarily, the mask network can be a fully convolutional network (FCN), which is applied to any region of interest feature map to predict a pixel-granular mask of the corresponding target object.

[0068] Next, in operation S2305 , a loss value of the first function is calculated based on the mask prediction result for the any training picture and the label of the any training picture.

[0069] Exemplarily, the first function may be a weighted cross entropy loss (Sigmoid Cross Entropy Loss) function, etc., which is used to measure the difference between the mask prediction result output by the mask network and the mask of the target object in the actual training image.

[0070] Next, in operation S2306, based on the loss value of the first function, the parameters of at least one network among the backbone network, the region proposal network and the mask network are adjusted until the first function converges.

[0071] It can be understood that through iterative optimization of network parameters, when the first function converges, the difference between the mask prediction result output by the mask network and the mask of the target object in the actual training image tends to be minimized, so that the target network model can more accurately predict the spatial position distribution of the target object.

[0072] According to an embodiment of the present invention, the mask prediction result for any training image includes multiple mask prediction results for the any training image. On this basis, the calculation of the loss value of the first function based on the mask prediction result of the any training image and the label of the any training image includes: determining the actual category and the actual mask of the any training image according to the label of the any training image. The mask prediction result corresponding to the category of the any training image is selected from the multiple mask prediction results for the any training image. Then, based on the actual mask of the any training image and the first function, the loss value of the mask prediction result corresponding to the category of the any training image is calculated to measure the difference between the actual mask of the any training image and the mask prediction result of the corresponding category. The above process only needs to consider the loss of the mask prediction results of the same category, which can improve the efficiency of model training on the basis of ensuring the quality of model training.

[0073] For example, the mask network outputs C mask prediction results M1~M for a training image. C , C is an integer greater than 1, representing the total number of categories of target objects in all training images. If the category of the target object in the above training image is the i-th category, i is an integer greater than or equal to 1 and less than or equal to C, then the mask prediction result M is selected iThe loss value of the first function is determined by comparing the actual mask of the target object marked in the label.

[0074] Continue to refer Figure 3 According to an embodiment of the present invention, operation S230 uses multiple training images and the labels of the multiple training images to train the initial network model, which may include not only the above operations S2301 to S2306, but also the following operations.

[0075] In operation S2307 , based on the mask of any training picture, a plurality of mask prediction results for the any training picture, and the first function, a loss value of each of the plurality of mask prediction results is calculated.

[0076] In operation S2308, a confidence map is converted based on the loss values ​​of each of the plurality of mask prediction results.

[0077] Then, in operation S2309, the confidence map and the first region of interest feature map obtained above are input into the recognition network, so that the recognition network outputs the classification prediction result and the outer box prediction result for any training image.

[0078] Next, in operation S2310, a loss value of the second function is calculated based on the classification prediction result for the any training picture, the outer box prediction result, and the label of the any training picture.

[0079] Exemplarily, the second function can be used to measure the difference between the classification prediction result output by the recognition network and the category of the target object in the actual training image, and the difference between the outer box prediction result output by the recognition network and the outer box of the target object in the actual training image.

[0080] Next, in operation S2311, based on the loss value of the second function, the parameters of the recognition network are adjusted until the second function converges.

[0081] It can be understood that the input of the above-mentioned recognition network includes the confidence map converted from the mask prediction result and the feature map of the region of interest of the training picture, so that the processing process of the recognition network pays more attention to the spatial position distribution of the target object in the training picture. Through iterative optimization of the network parameters, when the second function converges, the difference between the classification prediction result output by the recognition network and the category of the target object in the actual training picture tends to be minimized, and the difference between the outer box prediction result output by the recognition network and the outer box of the target object in the actual training picture tends to be minimized. Thereby, the target network model can more accurately predict the outer box position and category of the target object and avoid positioning deviation.

[0082] According to an embodiment of the present invention, the above-mentioned conversion based on the loss values ​​of each of the multiple mask prediction results to obtain a confidence map includes: for any mask prediction result among the multiple mask prediction results, the loss value of each pixel in the any mask prediction result is inverted and added by 1 to obtain a confidence value of each pixel. Then, based on the confidence value of each pixel in the any mask prediction result, the confidence map of the any mask prediction result is determined.

[0083] For example, the mask prediction results are encoded using binary values, and the mask network described above is used to output C mask prediction results. Combined with the first function, an error loss value of n×n×C dimensions can be obtained, with a value interval of [0,1], where n is a positive integer and n×n represents the pixel resolution. Here, all loss values ​​are inverted and added with 1, so that the loss values ​​are converted into confidence values. The larger the confidence value, the higher the prediction accuracy, thus obtaining a confidence map of n×n×C dimensions.

[0084] Further, according to an embodiment of the present invention, the model training method may further include: before inputting the first region of interest feature map into the mask network, performing scale normalization processing on the first region of interest feature map. And / or, before inputting the first region of interest feature map into the recognition network, performing scale normalization processing on the first region of interest feature map.

[0085] In this embodiment, the process of normalizing the scale of the feature map of the region of interest can be called the region of interest alignment (RoIAlign) process. It should be noted that the region of interest alignment process is different from the region of interest pooling (RoIPool) process in Faster R-CNN. The region of interest pooling process maps the region of interest to the corresponding position according to the original input image. In the mapping process, quantization (quantize) is required, that is, rounding up, and then the original image area is divided into different parts according to the set final output size. If a decimal is generated during segmentation, quantization is performed again, and finally the segmented parts are subjected to the maximum pooling (max pooling) operation. It can be understood that the region of interest pooling process will lose a lot of spatial position information after multiple quantizations, resulting in errors in the mask prediction results at the pixel level. For this reason, this embodiment adopts the region of interest alignment process, which does not perform a quantization operation, but after setting the number of sampling points, bilinear interpolation is performed according to the number of sampling points as the value of the corresponding part. No loss of spatial position information can achieve pixel-to-pixel alignment.

[0086] According to an embodiment of the present invention, the model training method may further include: before the confidence map and the first region of interest feature map are input into the recognition network, the confidence map is subjected to dimensionality reduction processing. The inputting of the confidence map and the first region of interest feature map into the recognition network includes: accumulating the confidence map subjected to dimensionality reduction processing and the first region of interest feature map in the channel dimension to obtain a first feature map to be recognized, and then inputting the first feature map to be recognized into the recognition network.

[0087] Figure 4 An example structural diagram of an initial network model according to an embodiment of the present invention is schematically shown.

[0088] like Figure 4 As shown, the initial network model includes: a backbone network 410, a region proposal network (not shown in the figure), a mask network 420 and a recognition network 430 based on an attention mechanism. A training picture is input into the backbone network 410, and the corresponding feature map is extracted by the backbone network. The candidate target box generated by the region proposal network acts on the feature map to obtain a first region of interest feature map. The first region of interest feature map is scale-normalized by the region of interest alignment operation, and then respectively input into the mask network 420 and the recognition network 430. The mask network 420 is, for example, a fully convolutional network. The mask network 420 outputs C mask prediction results. In this example, combined with the first function, an error loss value of 28×28×C dimensions can be obtained, and the value interval is [0,1]. Here, all loss values ​​are inverted and added 1, so as to convert the loss value into a confidence value. The larger the confidence value, the higher the prediction accuracy, thereby obtaining a confidence map of 28×28×C dimensions.

[0089] Then, the recognition network 430 based on the attention mechanism is trained using the confidence map. The confidence map of 28×28×C dimensions can be first reduced in dimension through the maximum pooling operation to obtain an updated confidence map of 7×7×C dimensions. The updated confidence map is accumulated with the scale-normalized first region of interest feature map in the channel dimension to obtain the first feature map to be identified. If the size of the scale-normalized first region of interest feature map is 7×7×256, the size of the first feature map to be identified is 7×7×(256+C). The first feature map to be identified is then input into the recognition network 430 for training, so that the recognition network outputs the classification prediction results and the outer box prediction results for the above training images.

[0090] According to an embodiment of the present invention, a map drawing method is provided. The method is exemplarily described below through a legend. It should be noted that the sequence number of each operation in the following method is only used as a representation of the operation for the purpose of description, and should not be regarded as indicating the execution order of the various operations. Unless explicitly stated, the method does not need to be executed in the order shown.

[0091] Figure 5 A flowchart of a map drawing method according to an embodiment of the present invention is schematically shown.

[0092] like Figure 5 As shown, the method may include operations S510 to S540.

[0093] In operation S510, a road picture is acquired.

[0094] In operation S520, a target network model is acquired.

[0095] The target network model is obtained by training according to the model training method of any of the embodiments described above.

[0096] Then, in operation S530, the road image is processed using the target network model so that: the mask network in the target network model outputs the mask of the target object in the road image, and the recognition network based on the attention mechanism in the target network model outputs the category and outer box of the target object in the road image.

[0097] Next, in operation S540 , the target object is drawn in the map based on the mask, the category, and the bounding box of the target object in the road picture.

[0098] As described above, the model training method according to the embodiment of the present invention can train a target network model with better performance, avoid positioning deviation for the target object, and reduce the recognition deviation of the target object. The map drawing method according to the embodiment of the present invention uses the target network model obtained by the above training to process the target objects in the road image, and more accurately determines the mask, category and outer box of the target object in the road image, avoiding the positioning deviation of the outer box. It can more accurately extract and segment target objects of various types and shapes, which is conducive to the drawing of various target objects in high-precision maps.

[0099] Figure 6 Schematically shows an embodiment of the present invention Figure 5 An example flowchart of operation S530 in FIG.

[0100] like Figure 6 As shown, the process of processing the road picture using the target network model in operation S530 may include the following operations.

[0101] In operation S5301, a road picture is input into a backbone network of a target network model so that the backbone network outputs a second feature map for the road picture.

[0102] In operation S5302, the road picture is input to the region proposal network of the target network model, so that the region proposal network outputs a second candidate target box for the road picture.

[0103] Then, in operation S5303, the second feature map of the road image is input into the mask network, so that the mask network outputs a mask prediction result for the road image.

[0104] In operation S5304, the second candidate target box is applied to the second feature map and the mask prediction result for the road image respectively to obtain a second region of interest feature map and a region of interest mask prediction result.

[0105] Next, in operation S5305, the second ROI feature map and the ROI mask prediction result are input into the recognition network, so that the recognition network outputs the category and bounding box of the target object in the road image.

[0106] According to an embodiment of the present invention, the above-mentioned map drawing method also includes: before the second region of interest feature map and the region of interest mask prediction result are input into the recognition network, the second region of interest feature map and the region of interest mask prediction result are respectively scale normalized. Exemplarily, the scale normalization process for the second region of interest feature map and the region of interest mask prediction result is the same as the principle of the scale normalization process for the first region of interest mentioned above, and will not be repeated here. On this basis, the above-mentioned input of the second region of interest feature map and the region of interest mask prediction result into the recognition network includes: accumulating the second region of interest feature map and the region of interest mask prediction result after scale normalization in the channel dimension to obtain a second feature map to be identified for the road image. Then, the second feature map to be identified is input into the recognition network so that the recognition network outputs the category and outer bounding box of the target object in combination with the non-maximum suppression algorithm.

[0107] According to an embodiment of the present invention, the above-mentioned map drawing method also includes: based on the category of the target object in the road image, determining a mask prediction result corresponding to the category of the target object from the mask prediction results for the road image as a mask of the target object in the road image.

[0108] For example, after training the target network model, a road image is input into the backbone network, and the backbone network extracts the corresponding feature map. The region proposal network generates a candidate target frame. The extracted feature map is directly input into the mask network so that the mask network outputs a mask prediction result (maskmap) of n×n×C dimensions for the road image. The candidate target frame is applied to the above feature map to obtain a second region of interest feature map, and then the second region of interest feature map is scale-normalized by the region of interest alignment operation. At the same time, the candidate target frame is applied to the above mask prediction result to obtain a region of interest mask prediction result, and then the region of interest mask prediction result is scale-normalized by the region of interest alignment operation.

[0109] The scale-normalized second region of interest feature map and the scale-normalized region of interest mask prediction result are accumulated in the channel dimension to obtain the second feature map to be identified. This process is the same as the process of accumulating the updated confidence map and the scale-normalized first region of interest feature map in the channel dimension in the above text, and will not be repeated here. The second feature map to be identified is then processed using the recognition network in combination with the non-maximum suppression algorithm to obtain the outer box prediction result and classification prediction result of the road image, and the outer box prediction result is used as the outer box of the target object in the road image, and the classification prediction result is used as the category of the target object in the road image. Then, according to the category of the target object, the mask prediction result corresponding to the category is selected from the above mask prediction results as the mask of the target object in the road image.

[0110] The target network model according to the embodiment of the present invention is suitable for the recognition and extraction of various types of target objects. Generally, slender target objects in road images are prone to positioning deviation, while the target network model according to the embodiment of the present invention can also accurately locate the outer frame of the target object and segment the mask for slender target objects.

[0111] Figure 7 The example structure diagram of a road image according to an embodiment of the present invention is schematically shown. The road image 700 includes various types of lane markings. In order to be able to identify and segment various lane markings from the road image, various lane markings are labeled as target objects when training the target network model. The target network model of the embodiment of the present invention is used to process the road image 700, and the outer box 710, mask 720 and category (such as lane marking) of the lane markings contained in the road image 700 can be identified. Figure 7 The lane marking line can be accurately segmented. The lane marking line can be accurately located and drawn in the map to avoid drawing errors caused by recognition deviation.

[0112] Figure 8 A block diagram of a model training device according to an embodiment of the present invention is schematically shown.

[0113] like Figure 8 As shown, the model training device 800 may include: a construction module 810, a sample acquisition module 820 and a training module 830.

[0114] The construction module 810 is used to construct an initial network model. The initial network model includes: a backbone network, a region proposal network, a mask network, and a recognition network based on an attention mechanism. Among them, the backbone network is used to extract features from the input image to obtain a feature map. The region proposal network is used to generate candidate target frames. The mask network is used to obtain mask prediction results based on the feature map and the candidate target frame. The recognition network is used to obtain classification prediction results and outer box prediction results based on the above feature map, candidate target frame and mask prediction results.

[0115] The sample acquisition module 820 is used to acquire multiple training images and labels of each of the multiple training images. The label of any training image among the acquired multiple training images includes: the category, bounding box and mask of the target object in the training image.

[0116] The training module 830 is used to train the initial network model using multiple training images and the labels of the multiple training images to obtain a target network model.

[0117] Fig. 9 A block diagram of a map drawing device according to an embodiment of the present invention is schematically shown.

[0118] like Fig. 9 As shown, the map drawing device 900 may include: a first acquisition module 910 , a second acquisition module 920 , a processing module 930 and a drawing module 940 .

[0119] The first acquisition module 910 is used to acquire a road picture.

[0120] The second acquisition module 920 is used to obtain the target network model trained by the model training method described in any of the above embodiments.

[0121] The processing module 930 is used to process the road image using the target network model so that: the mask network in the target network model outputs the mask of the target object in the road image, and the recognition network based on the attention mechanism in the target network model outputs the category and outer box of the target object in the road image.

[0122] The drawing module 940 is used to draw the target object in the map based on the mask, category and bounding box of the target object in the road picture.

[0123] It should be noted that the implementation methods, technical problems solved, functions realized, and technical effects achieved of each module / unit / sub-unit in the device part embodiment are the same or similar to the implementation methods, technical problems solved, functions realized, and technical effects achieved of each corresponding step in the method part embodiment, and will not be repeated here.

[0124] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware, and firmware, or by a suitable combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be performed.

[0125] For example, any multiple of the construction module 810, the sample acquisition module 820, and the training module 830 can be combined in one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the construction module 810, the sample acquisition module 820, and the training module 830 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, at least one of the construction module 810, the sample acquisition module 820, and the training module 830 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.

[0126] Fig.10A block diagram of a computer device suitable for implementing the model training method and / or map drawing method described above according to an embodiment of the present invention is schematically shown. Fig.10 The computer device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0127] like Fig.10 As shown, the computer device 1000 according to an embodiment of the present invention includes a processor 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1008 into a random access memory (RAM) 1003. The processor 1001 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1001 may also include an onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0128] In RAM 1003, various programs and data required for the operation of device 1000 are stored. Processor 1001, ROM 1002 and RAM 1003 are connected to each other via bus 1004. Processor 1001 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 1002 and / or RAM 1003. It should be noted that the program can also be stored in one or more memories other than ROM 1002 and RAM 1003. Processor 1001 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0129] According to an embodiment of the present invention, the device 1000 may further include an input / output (I / O) interface 1005, which is also connected to the bus 1004. The device 1000 may further include one or more of the following components connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage section 1008 as needed.

[0130] According to an embodiment of the present invention, the method flow according to an embodiment of the present invention can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the processor 1001, the above-mentioned functions defined in the system of the embodiment of the present invention are executed. According to an embodiment of the present invention, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0131] The embodiment of the present invention further provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the model training method and / or map drawing method according to the embodiment of the present invention is implemented.

[0132] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In an embodiment of the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 1002 and / or RAM 1003 described above and / or one or more memories other than ROM 1002 and RAM 1003.

[0133] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0134] Those skilled in the art will appreciate that the features described in the various embodiments and / or claims of the embodiments of the present invention may be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the embodiments of the present invention. In particular, without departing from the spirit and teachings of the embodiments of the present invention, the features described in the various embodiments and / or claims of the embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the embodiments of the present invention.

[0135] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the embodiments of the present invention. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination. The scope of the embodiments of the present invention is defined by the attached claims and their equivalents. Without departing from the scope of the embodiments of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the embodiments of the present invention.

Claims

1. A model training method, comprising: Constructing an initial network model, wherein the initial network model includes: a backbone network, a region proposal network, a mask network, and a recognition network based on an attention mechanism, wherein the backbone network is used to extract features from an input image to obtain a feature map, the region proposal network is used to generate a candidate target frame, the mask network is used to obtain a mask prediction result based on the feature map and the candidate target frame, and the recognition network is used to obtain a classification prediction result and an outer box prediction result based on the feature map, the candidate target frame, and the mask prediction result; Acquire multiple training images and labels of the multiple training images, wherein the label of any training image in the multiple training images includes: a category, a bounding box, and a mask of a target object in the any training image; and The initial network model is trained using the multiple training images and their respective labels to obtain a target network model, wherein the recognition network is obtained by adjusting the loss value obtained based on the classification prediction results of the training images, the outer box prediction results and the labels of the training images, and the classification prediction results and the outer box prediction results are generated by using the recognition network to process the confidence maps corresponding to the multiple mask prediction results and the first region of interest feature map, and the first region of interest feature map is generated by applying the candidate target box to the feature map output by the backbone network.

2. The method according to claim 1, wherein: The training of the initial network model using the multiple training pictures and the labels of the multiple training pictures includes: for any training picture among the multiple training pictures, Inputting any one of the training images into the backbone network so that the backbone network outputs a first feature map for any one of the training images; Inputting any one of the training images into the region proposal network so that the region proposal network outputs a first candidate target box for any one of the training images; Applying the first candidate target frame to the first feature map to obtain a first region of interest feature map; Inputting the first region of interest feature map into the mask network, so that the mask network outputs a mask prediction result for any one of the training images; Calculating a loss value of a first function based on the mask prediction result for any one of the training images and the label of any one of the training images; and Based on the loss value of the first function, parameters of the backbone network, the region proposal network and / or the mask network are adjusted until the first function converges.

3. The method according to claim 2, wherein: The mask prediction result for any one of the training images includes multiple mask prediction results; The calculating the loss value of the first function based on the mask prediction result of any one of the training images and the label of any one of the training images includes: Determine the category and mask of any training image according to the label of any training image; as well as Based on the mask of any one of the training images and the first function, a loss value of a mask prediction result corresponding to the category of any one of the training images among the multiple mask prediction results is calculated.

4. The method according to claim 3, wherein: The training of the initial network model using the plurality of training images and the labels of the plurality of training images further comprises: Based on the mask of any one of the training images, the plurality of mask prediction results and the first function, calculating the loss value of each of the plurality of mask prediction results; Based on the loss values ​​of each of the plurality of mask prediction results, converting to obtain a confidence map; Inputting the confidence map and the first region of interest feature map into the recognition network, so that the recognition network outputs a classification prediction result and a bounding box prediction result for any one of the training images; Calculating a loss value of a second function based on the classification prediction result, the outer bounding box prediction result and the label of the any training image for the any training image; and Based on the loss value of the second function, the parameters of the recognition network are adjusted until the second function converges.

5. The method according to claim 4, wherein: The converting the loss values ​​of the plurality of mask prediction results into a confidence map comprises: For any mask prediction result among the multiple mask prediction results, negate the loss value of each pixel in the any mask prediction result and add 1 to obtain a confidence value of each pixel; and Based on the confidence value of each pixel in any mask prediction result, a confidence map of any mask prediction result is determined.

6. The method according to claim 4, further comprising: Before inputting the first region of interest feature map into the mask network, performing scale normalization processing on the first region of interest feature map; and / or Before inputting the first region of interest feature map into the recognition network, the first region of interest feature map is scale normalized.

7. The method according to claim 4, further comprising: Before inputting the confidence map and the first region of interest feature map into the recognition network, performing dimensionality reduction processing on the confidence map; The step of inputting the confidence map and the first region of interest feature map into the recognition network comprises: Accumulating the confidence map after dimension reduction processing and the first region of interest feature map in the channel dimension to obtain a first feature map to be identified; as well as The first feature map to be identified is input into the identification network.

8. A method for drawing a map, comprising: Get road pictures; Obtain a target network model trained by the model training method according to any one of claims 1 to 7; The road image is processed using the target network model so that: the mask network in the target network model outputs the mask of the target object in the road image, and the recognition network based on the attention mechanism in the target network model outputs the category and the outer bounding box of the target object in the road image; as well as Based on the mask, category and bounding box of the target object in the road picture, the target object is drawn in the map.

9. The method according to claim 8, wherein: The processing of the road image by using the target network model includes: Inputting the road image into the backbone network of the target network model so that the backbone network outputs a second feature map for the road image; Inputting the road image into a region proposal network of the target network model so that the region proposal network outputs a second candidate target box for the road image; Inputting the second feature map of the road image into the mask network so that the mask network outputs a mask prediction result for the road image; Applying the second candidate target frame to the second feature map and the mask prediction result for the road image respectively to obtain a second region of interest feature map and a region of interest mask prediction result; and The second ROI feature map and the ROI mask prediction result are input into the recognition network, so that the recognition network outputs the category and the outer bounding box of the target object in the road picture.

10. The method according to claim 9, further comprising: Before inputting the second region of interest feature map and the region of interest mask prediction result into the recognition network, respectively performing scale normalization processing on the second region of interest feature map and the region of interest mask prediction result; The step of inputting the second ROI feature map and the ROI mask prediction result into the recognition network comprises: Accumulating the scale-normalized second ROI feature map and the ROI mask prediction result in the channel dimension to obtain a second feature map to be identified for the road image; as well as The second feature map to be identified is input into the recognition network, so that the recognition network outputs the category and the outer box in combination with a non-maximum suppression algorithm.

11. The method according to claim 9, further comprising: Based on the category of the target object in the road picture, a mask prediction result corresponding to the category of the target object is determined from the mask prediction results for the road picture to serve as a mask of the target object in the road picture.

12. The method according to claim 1 or 8, wherein: The target object includes: lane marking lines.

13. A model training device, comprising: A construction module, used to construct an initial network model, wherein the initial network model includes: a backbone network, a region proposal network, a mask network and a recognition network based on an attention mechanism, wherein the backbone network is used to extract features from an input image to obtain a feature map, the region proposal network is used to generate a candidate target frame, the mask network is used to obtain a mask prediction result based on the feature map and the candidate target frame, and the recognition network is used to obtain a classification prediction result and an outer box prediction result based on the feature map, the candidate target frame and the mask prediction result; A sample acquisition module, used to acquire a plurality of training images and labels of the plurality of training images, wherein the label of any training image in the plurality of training images includes: a category, an outer bounding box and a mask of a target object in the any training image; and A training module is used to train the initial network model using the multiple training images and their respective labels to obtain a target network model, wherein the recognition network is obtained by adjusting the loss value obtained based on the classification prediction results of the training images, the outer box prediction results and the labels of the training images, and the classification prediction results and the outer box prediction results are generated by using the recognition network to process the confidence maps corresponding to the multiple mask prediction results and the first region of interest feature map, and the first region of interest feature map is generated by applying the candidate target box to the feature map output by the backbone network.

14. A map drawing device, comprising: A first acquisition module is used to acquire a road picture; A second acquisition module is used to acquire a target network model trained by the model training method according to any one of claims 1 to 7; A processing module, configured to process the road image using the target network model, so that: the mask network in the target network model outputs the mask of the target object in the road image, and the recognition network based on the attention mechanism in the target network model outputs the category and outer bounding box of the target object in the road image; as well as The drawing module is used to draw the target object in the map based on the mask, category and outer bounding box of the target object in the road picture.

15. A computer device comprising: a memory having computer instructions stored thereon; as well as at least one processor; Wherein, when the processor executes the computer instructions, the method according to any one of claims 1 to 12 is implemented.

16. A computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions, when executed by a processor, implement the method according to any one of claims 1 to 12.

Images