Road surface marking recognition method and device, electronic equipment and computer readable storage medium
By using a pre-trained UET network model and image processing technology, road markings are automatically identified, solving the problem of high error rate in manual annotation and achieving high-precision road element recognition and high-definition map production.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHIDAO NETWORK TECH (BEIJING) CO LTD
- Filing Date
- 2023-04-28
- Publication Date
- 2026-05-19
AI Technical Summary
In existing technologies, the identification of road surface elements mainly relies on manual annotation, which leads to a high error rate and cannot be applied on a large scale to the production of high-precision maps. It is time-consuming and labor-intensive, and affects navigation accuracy.
A pre-trained UET network model with added residual modules in even-numbered layers is used to identify road marking images. By combining image preprocessing, feature extraction, and bounding polygon operations, the category of road markings is automatically identified.
It achieves accurate identification of large-scale road surface elements, reduces manual annotation, improves identification accuracy, and enhances the navigation precision of high-precision maps.
Smart Images
Figure CN116681919B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, electronic device and computer-readable storage medium for road marking recognition. Background Technology
[0002] With the increasing maturity of autonomous driving and navigation technologies, and the complexity of urban roads, using navigation tools while driving has become a driving habit for drivers. Accurate navigation is inseparable from the production of high-precision maps.
[0003] In the process of creating high-precision maps, road surface elements need to be drawn into the high-precision map. In relevant technical solutions, road surface element images are generally collected by vehicles. However, the identification of road surface elements is usually done manually by first annotating the road surface element images to determine the type of road surface element. Then, based on the determined type of road surface element, the corresponding road surface element is added to the high-precision map. This method of collecting and identifying road surface elements cannot be applied to the drawing of large-scale road markings. Furthermore, the manual annotation method has a high error rate, resulting in low accuracy of road surface element identification. Ultimately, this affects the navigation accuracy of the high-precision map and is time-consuming and labor-intensive. Summary of the Invention
[0004] To address or partially address the problems existing in related technologies, this application provides a road marking recognition method, device, electronic device, and computer-readable storage medium, which can accurately identify road elements on a large scale.
[0005] The first aspect of this application provides a method for identifying road markings, including:
[0006] The pre-acquired road marking images are pre-processed to obtain road marking images to be recognized that can be used by the road marking recognition model.
[0007] The road marking image to be identified is input into a preset road marking recognition model to obtain a binary image of the road marking image to be identified. The preset road marking recognition model is a pre-trained UET network model with residual modules added to an even number of layers.
[0008] The bounding polygon operation is performed on the binary graph to obtain the bounding polygon of the road marking to be identified. Based on the bounding polygon, the category of the road marking is determined.
[0009] In one possible implementation of this application, the preprocessing of the pre-acquired road marking image to obtain a road marking image to be recognized that can be used by the road marking recognition model includes:
[0010] The pre-acquired road marking image is processed by image segmentation, and the outline of the road marking to be identified in the pre-acquired road marking image after image segmentation is marked to obtain the road marking image to be identified.
[0011] In one possible implementation of this application, the step of inputting the road marking image to be identified into a preset road marking recognition model to obtain a binary image of the road marking image to be identified includes:
[0012] The road marking image to be identified is converted into matrix input data of a preset size;
[0013] Feature extraction is performed on the matrix input data to obtain a feature-extracted image;
[0014] The feature-extracted image is upsampled to obtain an upsampled image;
[0015] The feature-extracted image is concatenated with the upsampled image to obtain a binary image of the road marking image to be identified.
[0016] In one possible implementation of this application, feature extraction is performed on the matrix input data to obtain a feature-extracted image, including:
[0017] The matrix input data is subjected to a first feature extraction using a first feature extraction module to obtain a first feature extraction image. The first feature extraction module includes a convolution module and a residual module.
[0018] The first feature extraction image is subjected to a second feature extraction using a second feature extraction module to obtain a second feature extraction image, wherein the second feature extraction module includes a convolution module;
[0019] The second feature extraction image is subjected to a third feature extraction using a third feature extraction module to obtain a third feature extraction image. The third feature extraction module includes a convolution module and a residual module.
[0020] In one possible implementation of this application, the feature extraction image is upsampled to obtain an upsampled image, including:
[0021] The third feature extraction image is upsampled for the first time to obtain a first upsampled intermediate image. The third feature extraction image is then connected to the first upsampled intermediate image via the axial data connection module to obtain the first upsampled image.
[0022] The first upsampled image is upsampled a second time to obtain a second upsampled intermediate image. The second feature extraction image is then connected to the second upsampled intermediate image via the axial data connection module to obtain the second upsampled image.
[0023] The second upsampled image is upsampled a third time to obtain a third upsampled intermediate image. The first feature extraction image is then connected to the third upsampled intermediate image via the axial data connection module to obtain the third upsampled image.
[0024] In one possible implementation of this application, the step of concatenating the feature-extracted image with the upsampled image to obtain a binary image of the road marking image to be identified includes:
[0025] The third upsampled image is then upsampled a fourth time to obtain the fourth upsampled image;
[0026] The matrix input data is passed through the axial data connection module and then connected with the fourth upsampled image to obtain a binary image of the road marking image to be identified.
[0027] As one possible implementation of this application, in this implementation, performing a bounding polygon operation on the binary image to obtain the bounding polygon of the road marking to be identified includes:
[0028] Multiple corner points in the binary image are identified using a preset corner point recognition model;
[0029] Based on the multiple corner points, a bounding rectangle operation is performed to obtain the bounding polygon of the road marking to be identified.
[0030] A second aspect of this application provides a road marking recognition device, comprising:
[0031] The image acquisition module is used to preprocess the pre-acquired road marking images to obtain road marking images to be recognized that can be used by the road marking recognition model.
[0032] The model recognition module is used to input the road marking image to be recognized into a preset road marking recognition model to obtain a binary image of the road marking image to be recognized. The preset road marking recognition model is a pre-trained UET network model with residual modules added to an even number of layers.
[0033] The outer polygon module is used to perform an outer polygon operation on the binary image to obtain the outer polygon of the road marking to be identified, and to determine the category of the road marking based on the outer polygon.
[0034] A third aspect of this application provides an electronic device, comprising:
[0035] Processor; and
[0036] A memory that stores executable code, which, when executed by the processor, causes the processor to perform the method described above.
[0037] A fourth aspect of this application provides a computer-readable storage medium having executable code stored thereon, which, when executed by a processor of an electronic device, causes the processor to perform the method described above.
[0038] This application embodiment preprocesses the road marking image to obtain the road marking image to be identified, and uses a pre-trained UET network model with residual modules added to even-numbered layers to identify the road marking image. The identified binary image is then subjected to an outer polygon operation to determine the category of the road marking. By using the network model to identify the road marking image, a large number of road elements can be identified in a short time without manual annotation, saving manpower. Furthermore, the pre-trained UET network model with residual modules added to even-numbered layers can accurately identify the type of road marking, improving the accuracy of road marking identification and thus improving the accuracy of high-precision maps.
[0039] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description
[0040] The above and other objects, features and advantages of this application will become more apparent from the more detailed description of exemplary embodiments thereof in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments thereof.
[0041] Figure 1 This is a schematic flowchart illustrating a road marking method according to an embodiment of this application;
[0042] Figure 2 This is a schematic diagram of the structure of a road marking recognition model shown in an embodiment of this application;
[0043] Figure 3 This is a flowchart illustrating a method for determining a binary image according to an embodiment of this application;
[0044] Figure 4 This is a schematic flowchart illustrating a feature extraction method according to an embodiment of this application;
[0045] Figure 5 This is a schematic flowchart illustrating an upsampling method according to an embodiment of this application;
[0046] Figure 6 This is a flowchart illustrating a graphical connection method according to an embodiment of this application;
[0047] Figure 7 This is a flowchart illustrating an embodiment of the external polygon method of this application;
[0048] Figure 8 This is a road marking recognition effect diagram shown in an embodiment of this application;
[0049] Figure 9 This is a schematic diagram of the structure of a road marking recognition device shown in an embodiment of this application;
[0050] Figure 10 This is a schematic diagram of the structure of an electronic device shown in an embodiment of this application.
[0051] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale. Detailed Implementation
[0052] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to make this application more thorough and complete, and to fully convey the scope of this application to those skilled in the art.
[0053] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.
[0054] It should be understood that although the terms "first," "second," "third," etc., may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.
[0055] With the increasing maturity of autonomous driving and navigation technologies, and the complexity of urban roads, using navigation tools while driving has become a driving habit for drivers. Accurate navigation is inseparable from the production of high-precision maps.
[0056] In the process of creating high-precision maps, road surface elements need to be drawn into the high-precision map. In relevant technical solutions, road surface element images are generally collected by vehicles. However, the identification of road surface elements is usually done manually by first annotating the road surface element images to determine the type of road surface element. Then, based on the determined type of road surface element, the corresponding road surface element is added to the high-precision map. This method of collecting and identifying road surface elements cannot be applied to the drawing of large-scale road markings. Furthermore, the manual annotation method has a high error rate, resulting in low accuracy of road surface element identification. Ultimately, this affects the navigation accuracy of the high-precision map and is time-consuming and labor-intensive.
[0057] To address the aforementioned issues, this application provides a road marking recognition method that can accurately identify road elements on a large scale.
[0058] The technical solutions of the embodiments of this application are described in detail below with reference to the accompanying drawings.
[0059] Figure 1 This is a schematic flowchart illustrating the road marking recognition method in an embodiment of this application.
[0060] See Figure 1 The road marking recognition method provided in this application includes:
[0061] Step S101: Preprocess the pre-acquired road marking image to obtain a road marking image to be recognized that can be used by the road marking recognition model.
[0062] In this embodiment of the application, the road marking image refers to an image containing road markings to be identified, wherein the road markings to be identified include road arrows, road diamonds, pedestrian crossings, etc., which can be acquired by an in-vehicle image acquisition device, such as an in-vehicle camera or in-vehicle camera.
[0063] As one possible implementation of this application, when constructing a high-precision map, it is necessary to improve the road information, which may include adding road elements to the high-precision map. The road elements include road markings. In this embodiment of the application, road marking images of the road where the vehicle is traveling can be acquired by a collection vehicle. The collection vehicle refers to a vehicle used to collect road information. The collection vehicle is equipped with an image acquisition device that can collect road marking images of the road where the vehicle is traveling.
[0064] In one possible implementation of this application, the preprocessing of the pre-acquired road marking image to obtain a road marking image to be recognized that can be used by the road marking recognition model includes:
[0065] The pre-acquired road marking image is processed by image segmentation, and the outline of the road marking to be identified in the pre-acquired road marking image after image segmentation is marked to obtain the road marking image to be identified.
[0066] In this embodiment of the application, the pre-acquired road marking image is preprocessed. The preprocessing process includes removing images that do not conform to the preset specifications, such as images that are not clear and whose road marking category cannot be identified, and performing binarization processing on the road marking image after removing non-standard images. The outline of the road marking to be identified in the road marking image after binarization is then marked to obtain the road marking image to be identified.
[0067] As one possible implementation of this application, after binarizing the image, the binarized image can be combined with JSON data, and the binarized image can be annotated using the JSON data to obtain the outline of the road marking to be identified. If the annotation effect is not good or the annotation fails, the image can be re-annotated, or the image with poor annotation effect or annotation failure can be removed to determine the image of the road marking to be identified.
[0068] Step S102: Input the road marking image to be identified into a preset road marking recognition model to obtain a binary image of the road marking image to be identified. The preset road marking recognition model is a pre-trained UET network model with residual modules added to even-numbered layers.
[0069] In this embodiment, following the previous embodiment, after obtaining the image of the road marking to be identified, the image is input into a preset road marking recognition model to obtain a binary image of the road marking to be identified. This road marking recognition model is a pre-trained UNET network model with residual modules added to even-numbered layers; its structure can be referenced. Figure 2 .
[0070] As one possible implementation of this application, in this implementation, such as Figure 3 As shown, the step of inputting the road marking image to be identified into a preset road marking recognition model to obtain a binary image of the road marking image to be identified includes:
[0071] Step S301: Convert the road marking image to be identified into matrix input data of a preset size.
[0072] In this embodiment, before inputting the road marking image to be recognized into the road marking recognition model, the size of the target input image needs to be adjusted. In this embodiment, a matrix of 1920*1080*3 can be selected. Of course, the specific size of the matrix can be determined according to the actual situation, and this application does not impose any restrictions on it.
[0073] Step S302: Perform feature extraction on the matrix input data to obtain a feature-extracted image.
[0074] In this embodiment, feature extraction of the matrix input data can be performed multiple times to obtain multiple feature-extracted images. Specifically, during feature extraction, a feature extraction module combining multiple residual modules and convolutional modules can be used to extract features from the matrix data. Figure 4 As shown, feature extraction is performed on the matrix input data to obtain a feature-extracted image, including:
[0075] Step S401: The first feature extraction module is used to perform the first feature extraction on the matrix input data to obtain the first feature extraction image, wherein the first feature extraction module includes a convolution module and a residual module.
[0076] In the embodiments of this application, reference is made to Figure 2 , Figure 2 The left box shows the feature extraction steps, which are performed three times in total. The feature extraction steps include convolution and pooling. The convolution module can be a 3x3 convolution. In the first feature extraction of the matrix input data, the feature extraction module that integrates the residual module is used to extract features from the matrix input data to ensure the invariance and identity of the image data, and the first feature-extracted image is obtained.
[0077] Step S402: The second feature extraction module is used to perform a second feature extraction on the first feature extraction image to obtain a second feature extraction image, wherein the second feature extraction module includes a convolution module.
[0078] In this embodiment, similar to the previous embodiment, the first feature extraction image is first convolved using a convolution module, and then the first feature image after convolution using a feature extraction module that incorporates a residual module is pooled to obtain the second feature extraction image.
[0079] Step S403: The third feature extraction module is used to perform a third feature extraction on the second feature extraction image to obtain a third feature extraction image. The third feature extraction module includes a convolution module and a residual module.
[0080] In this embodiment, similar to the previous embodiment, the second feature extraction image is first convolved using a convolution module, and then the first feature image after convolution using a feature extraction module that incorporates a residual module is pooled to obtain the third feature extraction image.
[0081] This application embodiment extracts features from the road marking image to be identified by combining the feature extraction module of the residual module multiple times, thus ensuring the accuracy of the model recognition results.
[0082] Step S303: Upsample the feature extraction image to obtain an upsampled image.
[0083] In the embodiments of this application, reference is made to Figure 2 , Figure 2 The right-hand box shows the upsampling steps, which are performed three times in total. During upsampling, as shown... Figure 5 As shown, it specifically includes:
[0084] Step S501: The third feature extraction image is upsampled for the first time to obtain a first upsampled intermediate image. The third feature extraction image is then connected to the first upsampled intermediate image through the axial data connection module to obtain the first upsampled image.
[0085] Step S502: Perform a second upsampling on the first upsampled image to obtain a second upsampled intermediate image. After passing through the axial data connection module, the second feature extraction image is connected with the second upsampled intermediate image to obtain the second upsampled image.
[0086] Step S503: Perform a third upsampling on the second upsampled image to obtain a third upsampled intermediate image. After the first feature extraction image is passed through the axial data connection module, it is connected with the third upsampled intermediate image to obtain the third upsampled image.
[0087] In this embodiment of the application, the operations in steps S501 to S503 are similar. The specific upsampling process includes performing a first upsampling on the third feature extraction image to obtain a first upsampling intermediate image, concatenating the third feature extraction image with the first upsampling intermediate image via the axial data connection module to obtain a first upsampling image, performing a second upsampling on the first upsampling image to obtain a second upsampling intermediate image, concatenating the second feature extraction image with the axial data connection module to obtain a second upsampling image, performing a third upsampling on the second upsampling image to obtain a third upsampling intermediate image, and concatenating the first feature extraction image with the third upsampling intermediate image via the axial data connection module to obtain a third upsampling image.
[0088] Step S304: Connect the feature extraction image with the upsampled image to obtain a binary image of the road marking image to be identified.
[0089] In this embodiment, the feature extraction map obtained in the foregoing embodiments is concatenated with the upsampled image to obtain the model's output, i.e., a binary image of the road markings to be identified. Specifically, as shown below... Figure 6 As shown, it includes:
[0090] Step S601: Perform a fourth upsampling on the third upsampled image to obtain a fourth upsampled image.
[0091] In this embodiment of the application, following the previous embodiment, after obtaining the third upsampled image, the third upsampled image is upsampled a fourth time to obtain the fourth upsampled image.
[0092] Step S602: After the matrix input data is passed through the axial data connection module, it is connected with the fourth upsampled image to obtain a binary image of the road marking image to be identified.
[0093] In this embodiment of the application, after obtaining the fourth upsampled image, the matrix input data is concatted by the axial data connection module and then connected with the fourth upsampled image to obtain a binary image of the road marking image to be identified.
[0094] Step S103: Perform an outer polygon operation on the binary image to obtain the outer polygon of the road marking to be identified, and determine the category of the road marking based on the outer polygon.
[0095] In this embodiment of the application, after obtaining the binary image of the road marking to be identified, a bounding rectangle operation needs to be performed on the binary image. This bounding rectangle operation is implemented by enclosing a rectangle representing the shape of the road marking to be identified around it in the binary image, thus facilitating the identification of the road marking's category. Specifically, as shown... Figure 7 As shown, it includes:
[0096] Step S701: Use a preset corner recognition model to identify multiple corner points in the binary image.
[0097] In the embodiments of this application, a corner point refers to an extreme point in a binary image. The RGB value at a corner point is either the largest or the smallest compared to its surroundings. Optionally, the Moravec corner detection algorithm can be used to determine the corner points in the binary image.
[0098] Step S702: Based on the multiple corner points, perform a bounding rectangle operation to obtain the bounding polygon of the road marking to be identified.
[0099] In this embodiment of the application, after obtaining the corner points of the road markings to be identified in the binary image, an outer rectangle operation is performed based on the corner points. Specifically, the corner points can be connected in a non-repeating manner to form an outer rectangle. Based on the shape of the outer rectangle, the category of the road markings to be identified is determined.
[0100] As one possible implementation of this application, for ease of explanation, a specific embodiment is used as an example. When generating a preset road marking recognition model, it is necessary to process the pre-collected road marking image data. The pre-collected road marking images can be road marking images collected when a test vehicle is driving on the road. The pre-collected road marking images should at least contain the road markings to be recognized. The pre-collected road marking images are combined with JSON (JavaScript Object Notation) data to generate the required sample data. After obtaining the sample data, the sample data is preprocessed. The preprocessing process includes removing data from the sample data that does not meet the specifications. For example, when combining the road marking images with JSON data, the road marking images are annotated using the JSON data to obtain the outline of the road markings to be recognized. When the annotation effect is poor or the annotation fails, the images that fail to be annotated can be removed to obtain data that meets the specifications. Then, the obtained data that meets the specifications is randomly divided into test data and training data, saved to a preset MDB database, and the training data is used to test the UET network model with residual modules added in even-numbered layers. Training involves parsing data from the MDB database into a 512*512*3 matrix as input. This matrix is then used to train a UET network model with residual modules added to even-numbered layers. When this UET network model predicts and identifies road markings in images, it produces a predicted result. An outer rectangle operation is then performed on this predicted result to obtain the final recognition result and a real image label indicating the road marking category. The trained UET mesh model is then tested using test data. If the test results meet preset requirements, a trained UET network model incorporating residual modules is obtained. This mesh model can then be used to identify road markings captured by vehicles. For example... Figure 8 The image shown is an illustration of the road marking image recognition method provided in this application, demonstrating the effectiveness of road marking recognition. Figure 8 As can be seen, the solution provided in this application can accurately identify road markings, wherein, Figure 8 The white box in the image is used to indicate the category of road markings. Due to the limited display effect of the image, it is represented by a white box. In the actual recognition process, the category of road markings can be identified.
[0101] This application embodiment preprocesses the road marking image to obtain the road marking image to be identified, and uses a pre-trained UET network model with residual modules added to even-numbered layers to identify the road marking image. The identified binary image is then subjected to an outer polygon operation to determine the category of the road marking. By using the network model to identify the road marking image, a large number of road elements can be identified in a short time without manual annotation, saving manpower. Furthermore, the pre-trained UET network model with residual modules added to even-numbered layers can accurately identify the type of road marking, improving the accuracy of road marking identification and thus improving the accuracy of high-precision maps.
[0102] Corresponding to the aforementioned application function implementation method embodiments, this application also provides a road marking recognition device, electronic device, and corresponding embodiments.
[0103] Figure 9 This is a schematic diagram of the road marking recognition device shown in the embodiments of this application.
[0104] See Figure 9 The road marking recognition device 90 provided in this application embodiment includes an image acquisition module 910, a model recognition module 920, and an external polygon module 930, wherein:
[0105] The image acquisition module 910 is used to preprocess the pre-acquired road marking image to obtain a road marking image to be recognized that can be used by the road marking recognition model.
[0106] The model recognition module 920 is used to input the road marking image to be recognized into a preset road marking recognition model to obtain a binary image of the road marking image to be recognized. The preset road marking recognition model is a pre-trained UET network model with residual modules added to an even number of layers.
[0107] The circumscribed polygon module 930 is used to perform circumscribed polygon operations on the binary image to obtain the circumscribed polygon of the road marking to be identified, and to determine the category of the road marking based on the circumscribed polygon.
[0108] In one possible implementation of this application, the preprocessing of the pre-acquired road marking image to obtain a road marking image to be recognized that can be used by the road marking recognition model includes:
[0109] The pre-acquired road marking image is processed by image segmentation, and the outline of the road marking to be identified in the pre-acquired road marking image after image segmentation is marked to obtain the road marking image to be identified.
[0110] In one possible implementation of this application, the step of inputting the road marking image to be identified into a preset road marking recognition model to obtain a binary image of the road marking image to be identified includes:
[0111] The road marking image to be identified is converted into matrix input data of a preset size;
[0112] Feature extraction is performed on the matrix input data to obtain a feature-extracted image;
[0113] The feature-extracted image is upsampled to obtain an upsampled image;
[0114] The feature-extracted image is concatenated with the upsampled image to obtain a binary image of the road marking image to be identified.
[0115] In one possible implementation of this application, feature extraction is performed on the matrix input data to obtain a feature-extracted image, including:
[0116] The matrix input data is subjected to a first feature extraction using a first feature extraction module to obtain a first feature extraction image. The first feature extraction module includes a convolution module and a residual module.
[0117] The first feature extraction image is subjected to a second feature extraction using a second feature extraction module to obtain a second feature extraction image, wherein the second feature extraction module includes a convolution module;
[0118] The second feature extraction image is subjected to a third feature extraction using a third feature extraction module to obtain a third feature extraction image. The third feature extraction module includes a convolution module and a residual module.
[0119] In one possible implementation of this application, the feature extraction image is upsampled to obtain an upsampled image, including:
[0120] The third feature extraction image is upsampled for the first time to obtain a first upsampled intermediate image. The third feature extraction image is then connected to the first upsampled intermediate image via the axial data connection module to obtain the first upsampled image.
[0121] The first upsampled image is upsampled a second time to obtain a second upsampled intermediate image. The second feature extraction image is then connected to the second upsampled intermediate image via the axial data connection module to obtain the second upsampled image.
[0122] The second upsampled image is upsampled a third time to obtain a third upsampled intermediate image. The first feature extraction image is then connected to the third upsampled intermediate image via the axial data connection module to obtain the third upsampled image.
[0123] In one possible implementation of this application, the step of concatenating the feature-extracted image with the upsampled image to obtain a binary image of the road marking image to be identified includes:
[0124] The third upsampled image is then upsampled a fourth time to obtain the fourth upsampled image;
[0125] The matrix input data is passed through the axial data connection module and then connected with the fourth upsampled image to obtain a binary image of the road marking image to be identified.
[0126] As one possible implementation of this application, in this implementation, performing a bounding polygon operation on the binary image to obtain the bounding polygon of the road marking to be identified includes:
[0127] Multiple corner points in the binary image are identified using a preset corner point recognition model;
[0128] Based on the multiple corner points, a bounding rectangle operation is performed to obtain the bounding polygon of the road marking to be identified.
[0129] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated further here.
[0130] This application embodiment preprocesses the road marking image to obtain the road marking image to be identified, and uses a pre-trained UET network model with residual modules added to even-numbered layers to identify the road marking image. The identified binary image is then subjected to an outer polygon operation to determine the category of the road marking. By using the network model to identify the road marking image, a large number of road elements can be identified in a short time without manual annotation, saving manpower. Furthermore, the pre-trained UET network model with residual modules added to even-numbered layers can accurately identify the type of road marking, improving the accuracy of road marking identification and thus improving the accuracy of high-precision maps.
[0131] The following is for reference. Figure 10The diagram illustrates a structural schematic of an electronic device 1000 suitable for implementing embodiments of the present disclosure. The terminal devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 10 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.
[0132] The electronic device includes a memory and a processor, wherein the processor may be referred to as processing device 1001 as described below, and the memory may include at least one of read-only memory (ROM) 1002, random access memory (RAM) 1003, and storage device 1008 as described below, as follows:
[0133] like Figure 10 As shown, the electronic device 1000 may include a processing unit (e.g., a central processing unit, a graphics processor, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1008 into a random access memory (RAM) 1003. The RAM 1003 also stores various programs and data required for the operation of the electronic device 1000. The processing unit 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.
[0134] Typically, the following devices can be connected to the I / O interface 1005: input devices 1006 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 1007 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1008 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows electronic device 1000 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 10 An electronic device 1000 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.
[0135] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 1009, or installed from storage device 1008, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of embodiments of this disclosure.
[0136] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.
[0137] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.
[0138] The aforementioned computer-readable medium carries one or more programs. When the electronic device executes the aforementioned one or more programs, the electronic device causes the following to occur: preprocess a pre-acquired road marking image to obtain a road marking image to be recognized that can be used by a road marking recognition model; input the road marking image to be recognized into a preset road marking recognition model to obtain a binary image of the road marking image to be recognized, wherein the preset road marking recognition model is a pre-trained UET network model with residual modules added to an even number of layers; perform a bounding polygon operation on the binary image to obtain the bounding polygon of the road marking to be recognized; and determine the category of the road marking based on the bounding polygon.
[0139] Computer program code for performing the operations of this disclosure can be written in one or more programming languages or a combination thereof, including but not limited to object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0140] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0141] The modules or units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules or units do not necessarily limit the specific unit itself.
[0142] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.
[0143] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0144] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.
[0145] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.
[0146] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.
Claims
1. A method for identifying road markings, characterized in that, include: The pre-acquired road marking images are pre-processed to obtain road marking images to be recognized that can be used by the road marking recognition model. The road marking image to be identified is input into a preset road marking recognition model to obtain a binary image of the road marking image to be identified. The preset road marking recognition model is a pre-trained UET network model with residual modules added to an even number of layers. The bounding polygon operation is performed on the binary image to obtain the bounding polygon of the road marking to be identified. Based on the bounding polygon, the category of the road marking is determined. The step of inputting the road marking image to be identified into a preset road marking recognition model to obtain a binary image of the road marking image to be identified includes: The road marking image to be identified is converted into matrix input data of a preset size; Feature extraction is performed on the matrix input data to obtain a feature-extracted image; The feature-extracted image is upsampled to obtain an upsampled image; The feature-extracted image is concatenated with the upsampled image to obtain a binary image of the road marking image to be identified; The step of extracting features from the matrix input data to obtain a feature-extracted image includes: The matrix input data is subjected to a first feature extraction using a first feature extraction module to obtain a first feature extraction image. The first feature extraction module includes a convolution module and a residual module. The first feature extraction image is subjected to a second feature extraction using a second feature extraction module to obtain a second feature extraction image, wherein the second feature extraction module includes a convolution module; The second feature extraction image is subjected to a third feature extraction using a third feature extraction module to obtain a third feature extraction image. The third feature extraction module includes a convolution module and a residual module.
2. The road marking recognition method according to claim 1, characterized in that, The preprocessing of the pre-acquired road marking images to obtain road marking images to be recognized that can be used by the road marking recognition model includes: The pre-acquired road marking image is processed by image segmentation, and the outline of the road marking to be identified in the pre-acquired road marking image after image segmentation is marked to obtain the road marking image to be identified.
3. The road marking recognition method according to claim 1, characterized in that, Upsampling the feature-extracted image yields an upsampled image, including: The third feature extraction image is upsampled for the first time to obtain a first upsampled intermediate image. The third feature extraction image is then connected to the first upsampled intermediate image via the axial data connection module to obtain the first upsampled image. The first upsampled image is upsampled a second time to obtain a second upsampled intermediate image. The second feature extraction image is then connected to the second upsampled intermediate image via the axial data connection module to obtain the second upsampled image. The second upsampled image is upsampled a third time to obtain a third upsampled intermediate image. The first feature extraction image is then connected to the third upsampled intermediate image via the axial data connection module to obtain the third upsampled image.
4. The road marking recognition method according to claim 3, characterized in that, The step of concatenating the feature-extracted image with the upsampled image to obtain a binary image of the road marking image to be identified includes: The third upsampled image is then upsampled a fourth time to obtain the fourth upsampled image; The matrix input data is passed through the axial data connection module and then connected with the fourth upsampled image to obtain a binary image of the road marking image to be identified.
5. The road marking recognition method according to claim 3, characterized in that, The step of performing a bounding polygon operation on the binary image to obtain the bounding polygon of the road marking to be identified includes: Multiple corner points in the binary image are identified using a preset corner point recognition model; Based on the multiple corner points, a bounding rectangle operation is performed to obtain the bounding polygon of the road marking to be identified.
6. A road marking recognition device, characterized in that, include: The image acquisition module is used to preprocess the pre-acquired road marking images to obtain road marking images to be recognized that can be used by the road marking recognition model. The model recognition module is used to input the road marking image to be recognized into a preset road marking recognition model to obtain a binary image of the road marking image to be recognized. The preset road marking recognition model is a pre-trained UET network model with residual modules added to an even number of layers. The outer polygon module is used to perform an outer polygon operation on the binary image to obtain the outer polygon of the road marking to be identified, and to determine the category of the road marking based on the outer polygon; The step of inputting the road marking image to be identified into a preset road marking recognition model to obtain a binary image of the road marking image to be identified includes: The road marking image to be identified is converted into matrix input data of a preset size; Feature extraction is performed on the matrix input data to obtain a feature-extracted image; The feature-extracted image is upsampled to obtain an upsampled image; The feature-extracted image is concatenated with the upsampled image to obtain a binary image of the road marking image to be identified; The step of extracting features from the matrix input data to obtain a feature-extracted image includes: The matrix input data is subjected to a first feature extraction using a first feature extraction module to obtain a first feature extraction image. The first feature extraction module includes a convolution module and a residual module. The first feature extraction image is subjected to a second feature extraction using a second feature extraction module to obtain a second feature extraction image, wherein the second feature extraction module includes a convolution module; The second feature extraction image is subjected to a third feature extraction using a third feature extraction module to obtain a third feature extraction image. The third feature extraction module includes a convolution module and a residual module.
7. An electronic device, characterized in that, include: processor; as well as A memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the method as described in any one of claims 1-5.
8. A computer-readable storage medium having executable code stored thereon, which, when executed by a processor of an electronic device, causes the processor to perform the method as described in any one of claims 1-5.