Road surface arrow recognition method and device, electronic equipment and computer readable storage medium
By combining a 4-layer V-shaped network model with the difference of Gaussian operator to preprocess and extract features from road arrow images, the problem of inaccurate recognition caused by occlusion of road arrows is solved, achieving efficient road arrow recognition and improving the accuracy of high-precision maps.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHIDAO NETWORK TECH (BEIJING) CO LTD
- Filing Date
- 2023-04-28
- Publication Date
- 2026-07-03
AI Technical Summary
In the process of creating high-precision maps, road arrows may be obscured, leading to inaccurate recognition and affecting the accuracy of vehicle navigation.
A pre-trained 4-layer V-shaped network model combined with the difference of Gaussian operator is used to preprocess and extract features from the road arrow images. The difference of Gaussian operator is used for edge detection to determine the edge and type of the road arrows.
It can accurately identify obscured road arrows in a short time, improve the accuracy of road arrow recognition, enhance the precision of high-precision maps, and save labor costs.
Smart Images
Figure CN116503839B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, electronic device, and computer-readable storage medium for recognizing road arrows. Background Technology
[0002] With the increasing maturity of autonomous driving and navigation technologies, and the complexity of urban roads, using navigation tools while driving has become a driving habit for drivers. Accurate navigation is inseparable from the production of high-precision maps.
[0003] In the process of creating high-precision maps, road surface elements need to be collected, and the accuracy of high-precision maps is inextricably linked to the accuracy of road surface element collection. In related technologies, the collection of road surface arrow elements may encounter situations where the arrows are obscured, resulting in incomplete or unclear arrow images. This leads to inaccurate arrow recognition and consequently affects vehicle navigation accuracy. Summary of the Invention
[0004] To address or partially address the problems existing in related technologies, this application provides a road surface arrow recognition method, apparatus, electronic device, and computer-readable storage medium, which can accurately identify obscured road surface arrows.
[0005] The first aspect of this application provides a method for recognizing road arrows, including:
[0006] The pre-acquired road arrow images are pre-processed to obtain road arrow images to be recognized that can be used by the road arrow recognition model.
[0007] The road arrow image to be identified is input into a preset road arrow recognition model to obtain the feature map of the road arrow image to be identified. The preset road arrow recognition model is a pre-trained 4-layer V-shaped network model that incorporates the difference of Gaussian operator.
[0008] The Gaussian difference operator is used to perform edge detection on the feature map to determine the edges of the road surface arrow to be identified, thereby obtaining the type of the road surface arrow to be identified.
[0009] In one possible implementation of this application, the preprocessing of the pre-acquired road arrow image to obtain a road arrow image to be recognized that can be used by the road arrow recognition model includes:
[0010] The pre-acquired road arrow image is processed by image segmentation, and the outline of the road arrow to be identified in the pre-acquired road arrow image after image segmentation is marked to obtain the road arrow image to be identified.
[0011] In one possible implementation of this application, the step of inputting the road arrow image to be identified into a preset road arrow recognition model to obtain a feature map of the road arrow image to be identified includes:
[0012] The image of the road arrow to be identified is converted into matrix input data of a preset size;
[0013] Feature extraction is performed on the matrix input data to obtain a feature-extracted image;
[0014] The feature-extracted image is upsampled to obtain an upsampled image;
[0015] The feature-extracted image is concatenated with the upsampled image to obtain the feature map of the road arrow image to be identified.
[0016] In one possible implementation of this application, feature extraction is performed on the matrix input data to obtain a feature-extracted image, including:
[0017] The matrix input data is subjected to a first feature extraction using a first feature extraction module to obtain a first feature extraction image, wherein the first feature extraction module includes a convolution module;
[0018] The first feature extraction image is subjected to a second feature extraction using a second feature extraction module to obtain a second feature extraction image, wherein the second feature extraction module includes a convolution module;
[0019] The second feature extraction image is subjected to a third feature extraction using a third feature extraction module to obtain a third feature extraction image. The third feature extraction module includes a convolution module.
[0020] In one possible implementation of this application, the feature extraction image is upsampled to obtain an upsampled image, including:
[0021] The third feature extraction image is upsampled for the first time to obtain a first upsampled intermediate image. The third feature extraction image is then processed by the difference of Gaussian operator and concatenated with the first upsampled intermediate image to obtain the first upsampled image.
[0022] The first upsampled image is upsampled a second time to obtain a second upsampled intermediate image. The second feature-extracted image is then processed by the difference of Gaussian operator and concatenated with the second upsampled intermediate image to obtain the second upsampled image.
[0023] The second upsampled image is upsampled a third time to obtain a third upsampled intermediate image. The first feature extraction image is then processed by the difference of Gaussian operator and concatenated with the third upsampled intermediate image to obtain the third upsampled image.
[0024] As one possible implementation of this application, in this implementation, the step of performing edge detection on the feature map using the difference of Gaussian operator to determine the edge of the road surface arrow to be identified, and obtaining the type of the road surface arrow to be identified, includes:
[0025] The pixel difference between the target road arrow region and the non-target road arrow region in the feature map is calculated using the difference of Gaussian operators, and the edge of the target road arrow is determined based on the pixel difference.
[0026] The outer polygon of the road surface arrow to be identified is obtained by performing an outer polygon operation based on the edge of the arrow.
[0027] As one possible implementation of this application, in this implementation, the step of performing a circumscribed polygon operation based on the edge of the road surface arrow to be identified, to obtain the circumscribed polygon of the road surface arrow to be identified, includes...
[0028] A preset corner point recognition model is used to identify multiple corner points of the edge of the road surface arrow to be identified in the feature map;
[0029] Based on the multiple corner points, a circumscribed polygon operation is performed to obtain the circumscribed polygon of the road surface arrow to be identified.
[0030] A second aspect of this application provides a road arrow recognition device, comprising:
[0031] The image acquisition module is used to preprocess the pre-acquired road arrow image to obtain a road arrow image to be recognized that can be used by the road arrow recognition model.
[0032] The image recognition module is used to input the road arrow image to be recognized into a preset road arrow recognition model to obtain the feature map of the road arrow image to be recognized. The preset road arrow recognition model is a pre-trained 4-layer V-shaped network model that incorporates the difference of Gaussian operator.
[0033] The type determination module is used to perform edge detection on the feature map using the difference of Gaussians operator to determine the edge of the road surface arrow to be identified, and to obtain the type of the road surface arrow to be identified.
[0034] A third aspect of this application provides an electronic device, comprising:
[0035] Processor; and
[0036] A memory that stores executable code, which, when executed by the processor, causes the processor to perform the method described above.
[0037] A fourth aspect of this application provides a computer-readable storage medium having executable code stored thereon, which, when executed by a processor of an electronic device, causes the processor to perform the method described above.
[0038] This application embodiment preprocesses the road arrow image to obtain the road arrow image to be identified, and uses a pre-trained 4-layer V-shaped network model combined with the difference of Gaussian operator to identify the road arrow image. The identified feature map is then edge-detected using the difference of Gaussian operator to determine the edges of the road arrows to be identified, thus determining the type of the road arrow. By using the network model to identify the road arrow image, a large number of road elements can be identified in a short time without manual annotation, saving manpower. Furthermore, the pre-trained 4-layer V-shaped network model combined with the difference of Gaussian operator can accurately identify occluded road arrows, improving the accuracy of road arrow recognition and thus enhancing the precision of high-precision maps.
[0039] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description
[0040] The above and other objects, features and advantages of this application will become more apparent from the more detailed description of exemplary embodiments thereof in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments thereof.
[0041] Figure 1 This is a schematic flowchart illustrating a road surface arrow method according to an embodiment of this application;
[0042] Figure 2 This is a flowchart illustrating a method for determining a feature map according to an embodiment of this application;
[0043] Figure 3 This is a schematic flowchart illustrating a feature extraction method according to an embodiment of this application;
[0044] Figure 4 This is a schematic flowchart illustrating an upsampling method according to an embodiment of this application;
[0045] Figure 5 This is a flowchart illustrating a method for determining edges according to an embodiment of this application;
[0046] Figure 6This is a flowchart illustrating an embodiment of the external polygon method of this application;
[0047] Figure 7 This is a road surface arrow recognition effect diagram shown in an embodiment of this application;
[0048] Figure 8 This is a schematic diagram of the structure of a road arrow recognition device shown in an embodiment of this application;
[0049] Figure 9 This is a schematic diagram of the structure of an electronic device shown in an embodiment of this application.
[0050] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale. Detailed Implementation
[0051] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to make this application more thorough and complete, and to fully convey the scope of this application to those skilled in the art.
[0052] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.
[0053] It should be understood that although the terms "first," "second," "third," etc., may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.
[0054] With the increasing maturity of autonomous driving and navigation technologies, and the complexity of urban roads, using navigation tools while driving has become a driving habit for many drivers. Accurate navigation relies heavily on the creation of high-definition maps. The creation of high-definition maps requires the collection of road surface elements, and the accuracy of the map is inextricably linked to the accuracy of this data collection. In related technologies, the collection of road arrow elements may encounter situations where the arrows are obscured, resulting in incomplete or unclear arrow images. This leads to inaccurate arrow recognition and consequently affects the accuracy of vehicle navigation.
[0055] To address the aforementioned issues, this application provides a method for recognizing road arrows that can accurately identify obscured road arrows.
[0056] The technical solutions of the embodiments of this application are described in detail below with reference to the accompanying drawings.
[0057] Figure 1 This is a schematic flowchart illustrating the road surface arrow recognition method according to an embodiment of this application.
[0058] See Figure 1 The road arrow recognition method provided in this application includes:
[0059] Step S101: Preprocess the pre-acquired road arrow image to obtain a road arrow image to be recognized that can be used by the road arrow recognition model.
[0060] In this embodiment of the application, the road surface arrow image refers to an image containing road surface arrows to be identified. The road surface arrows to be identified include straight arrows, turning arrows, U-turn arrows, etc., which can be acquired by an in-vehicle image acquisition device, such as an in-vehicle camera or in-vehicle camera.
[0061] As one possible implementation of this application, when constructing a high-precision map, it is necessary to improve the road information, which may include adding road arrows to the high-precision map. In this embodiment of the application, road arrow images of the road where the vehicle is traveling can be acquired by a collection vehicle. The collection vehicle refers to a vehicle used to collect road information. The collection vehicle is equipped with an image acquisition device that can collect road arrow images of the road where the vehicle is traveling.
[0062] In one possible implementation of this application, the preprocessing of the pre-acquired road arrow image to obtain a road arrow image to be recognized that can be used by the road arrow recognition model includes:
[0063] The pre-acquired road arrow image is processed by image segmentation, and the outline of the road arrow to be identified in the pre-acquired road arrow image after image segmentation is marked to obtain the road arrow image to be identified.
[0064] In this embodiment of the application, the pre-acquired road arrow image is preprocessed. The preprocessing process includes removing images that do not conform to the preset specifications, such as images that are not clear and cannot identify the road arrow category. The road arrow image after removing non-standard images is binarized, and the outline of the road arrow to be identified in the binarized road arrow image is marked to obtain the road arrow image to be identified.
[0065] As one possible implementation of this application, after binarizing the image, the binarized image can be combined with JSON data, and the binarized image can be annotated using the JSON data to obtain the outline of the road arrow to be identified. If the annotation effect is not good or the annotation fails, the image can be re-annotated, or the image with poor annotation effect or annotation failure can be removed to determine the road arrow image to be identified.
[0066] Step S102: Input the road arrow image to be identified into a preset road arrow recognition model to obtain the feature map of the road arrow image to be identified. The preset road arrow recognition model is a pre-trained 4-layer V-shaped network model that incorporates the difference of Gaussians operator.
[0067] In this embodiment of the application, following the previous embodiment, after obtaining the road surface arrow image to be identified, the road surface arrow image to be identified is input into a preset road surface arrow recognition model to obtain a binary image of the road surface arrow to be identified. The road surface arrow recognition model is a pre-trained 4-layer V-shaped network model that incorporates the difference of Gaussians operator.
[0068] As one possible implementation of this application, in this implementation, such as Figure 2 As shown, the step of inputting the road arrow image to be identified into a preset road arrow recognition model to obtain a binary image of the road arrow image to be identified includes:
[0069] Step S301: Convert the road arrow image to be identified into matrix input data of a preset size.
[0070] In this embodiment, before inputting the road arrow image to be recognized into the road arrow recognition model, the size of the target input image needs to be adjusted. In this embodiment, a matrix of 1920*1080*3 can be selected. Of course, the specific size of the matrix can be determined according to the actual situation, and this application does not impose any restrictions on it.
[0071] Step S302: Perform feature extraction on the matrix input data to obtain a feature-extracted image.
[0072] In this embodiment of the application, feature extraction of the matrix input data can be performed multiple times to obtain multiple feature-extracted images. Specifically, during feature extraction, a feature extraction module combining multiple convolutional modules can be used to extract features from the matrix data. Figure 3 As shown, feature extraction is performed on the matrix input data to obtain a feature-extracted image, including:
[0073] Step S401: The first feature extraction module is used to perform the first feature extraction on the matrix input data to obtain the first feature extraction image, wherein the first feature extraction module includes a convolution module.
[0074] In this embodiment of the application, feature extraction is performed three times. The feature extraction steps include convolution and pooling. The convolution module can be a 3x3 convolution.
[0075] Step S402: The second feature extraction module is used to perform a second feature extraction on the first feature extraction image to obtain a second feature extraction image, wherein the second feature extraction module includes a convolution module.
[0076] In this embodiment, similar to the previous embodiment, a convolution module is first used to convolve the first feature extraction image, and then pooling is performed on the convolved first feature image to obtain the second feature extraction image.
[0077] Step S403: The third feature extraction module is used to perform a third feature extraction on the second feature extraction image to obtain a third feature extraction image, wherein the third feature extraction module includes a convolution module.
[0078] In this embodiment, similar to the previous embodiment, a convolution module is first used to convolve the second feature extraction image, and then pooling is performed on the convolved first feature image to obtain the third feature extraction image.
[0079] This application embodiment uses multiple feature extraction modules to extract features from the road arrow image to be identified, ensuring the accuracy of the model's recognition results.
[0080] Step S303: Upsample the feature extraction image to obtain an upsampled image.
[0081] In this embodiment of the application, upsampling is performed a total of three times, wherein, during upsampling, as follows: Figure 4 As shown, it specifically includes:
[0082] Step S501: Perform a first upsampling on the third feature extraction image to obtain a first upsampled intermediate image. After passing the third feature extraction image through the Gaussian difference operator, connect it with the first upsampled intermediate image to obtain the first upsampled image.
[0083] Step S502: Perform a second upsampling on the first upsampled image to obtain a second upsampled intermediate image. After passing the second feature extraction image through the difference of Gaussian operator, connect it with the second upsampled intermediate image to obtain the second upsampled image.
[0084] Step S503: Perform a third upsampling on the second upsampled image to obtain a third upsampled intermediate image. After passing the first feature extraction image through the difference of Gaussian operator, concatenate it with the third upsampled intermediate image to obtain the third upsampled image.
[0085] In this embodiment, the operations in steps S501 to S503 are similar. The specific upsampling process includes performing a first upsampling on the third feature extraction image to obtain a first upsampling intermediate image; passing the third feature extraction image through a Gaussian difference operator and then connecting it to the first upsampling intermediate image using a concat module to obtain a first upsampling image; performing a second upsampling on the first upsampling image to obtain a second upsampling intermediate image; passing the second feature extraction image through a Gaussian difference operator and then connecting it to the second upsampling intermediate image using a concat module to obtain a second upsampling image; performing a third upsampling on the second upsampling image to obtain a third upsampling intermediate image; and passing the first feature extraction image through a Gaussian difference operator and then connecting it to the third upsampling intermediate image using a concat module to obtain a third upsampling image.
[0086] Step S304: Connect the extracted feature image with the upsampled image to obtain the feature map of the road arrow image to be identified.
[0087] In this embodiment of the application, the feature extraction map obtained in the foregoing embodiment is connected with the upsampled image to obtain the output of the model, namely the feature map of the road arrow to be identified.
[0088] Step S103: Use the Gaussian difference operator to perform edge detection on the feature map, determine the edge of the road surface arrow to be identified, and obtain the type of the road surface arrow to be identified.
[0089] In this embodiment of the application, after obtaining the feature map of the road surface arrow to be identified, as follows: Figure 5 As shown, the step of using the Gaussian difference operator to perform edge detection on the feature map, determining the edges of the road surface arrow to be identified, and obtaining the type of the road surface arrow to be identified includes:
[0090] Step S601: The difference of Gaussians operator is used to calculate the pixel difference between the area of the road arrow to be identified and the area of the road arrow to be identified in the feature map, and the edge of the road arrow to be identified is determined based on the pixel difference;
[0091] Step S602: Perform a circumscribed polygon operation based on the edge of the road surface arrow to be identified to obtain the circumscribed polygon of the road surface arrow to be identified.
[0092] In this embodiment, after obtaining the feature map of the road surface arrow to be identified, the difference of Gaussian operator is used to calculate the pixel difference between the region of the road surface arrow to be identified and the region of the road surface arrow not to be identified in the feature map, and the edge of the road surface arrow to be identified is determined based on the pixel difference; and a circumscribed polygon operation is performed based on the edge of the road surface arrow to be identified to obtain the circumscribed polygon of the road surface arrow to be identified. Specifically, as shown... Figure 6 As shown, the circumscribed polygon operation based on the edges of the road surface arrow to be identified is used to obtain the circumscribed polygon of the road surface arrow to be identified, including...
[0093] Step S701: Use a preset corner recognition model to identify multiple corner points of the edge of the road surface arrow to be identified in the feature map.
[0094] In the embodiments of this application, a corner point refers to an extreme point in the feature map. The RGB value at the corner point is either the largest or the smallest compared to the surrounding values. Optionally, the Moravec corner detection algorithm can be used to determine the corner points in the feature map.
[0095] Step S702: Based on the multiple corner points, perform a circumscribed polygon operation to obtain the circumscribed polygon of the road surface arrow to be identified.
[0096] In this embodiment of the application, after obtaining the corner points of the road arrow to be identified in the feature map, an enclosing rectangle operation is performed based on the corner points. Specifically, the corner points can be connected in a non-repeating manner to form an enclosing polygon. Based on the shape of the enclosing polygon, the category of the road arrow to be identified is determined.
[0097] As one possible implementation of this application, for ease of explanation, a specific embodiment is used as an example. When generating the preset road arrow recognition model, it is necessary to process the pre-collected road arrow image data. The pre-collected road arrow images can be road arrow images collected when a test vehicle is driving on the road. The pre-collected road arrow images should at least contain road arrows to be recognized. The pre-collected road arrow images are combined with JSON (JavaScript Object Notation) data to generate the required sample data. After obtaining the sample data, the sample data is preprocessed. The preprocessing process includes removing data from the sample data that does not meet the specifications. For example, when combining the road arrow images with JSON data, the road arrow images are annotated using the JSON data to obtain the outline of the road arrows to be recognized. When the annotation effect is poor or the annotation fails, the images that fail to be annotated can be removed to obtain data that meets the specifications. Then, the obtained data that meets the specifications is randomly divided into test data and training data, saved to a preset MDB database, and the training data is used to train a 4-layer V-shaped network model combined with the difference of Gaussian operator. During training, data from the MDB database can be parsed into a 512*512*3 matrix input to train a 4-layer V-shaped network model incorporating the difference of Gaussian (DG) operator. This model, when predicting and recognizing road arrow images, yields a predicted result. An enclosing polygon operation is then performed on this predicted result to obtain the final recognition result and a real image label indicating the road arrow category. The trained UET mesh model is then tested using test data. When the test results meet preset requirements, the trained 4-layer V-shaped network model incorporating the DG operator is complete. This mesh model can then be used to recognize road arrow images collected from vehicles. For example... Figure 7 The image shown is an illustration of the road arrow image recognition method provided in this application, demonstrating the effectiveness of road arrow recognition. Figure 7 As can be seen, the solution provided in this application can accurately identify road surface arrows, wherein... Figure 7 The white boxes in the image are used to indicate the category of the road arrows. Due to the limited display effect of the image, they are represented by white boxes. In the actual recognition process, the category of road arrows can be identified.
[0098] This application embodiment preprocesses the road arrow image to obtain the road arrow image to be identified, and uses a pre-trained 4-layer V-shaped network model combined with the difference of Gaussian operator to identify the road arrow image. The identified feature map is then edge-detected using the difference of Gaussian operator to determine the edges of the road arrows to be identified, thus determining the type of the road arrow. By using the network model to identify the road arrow image, a large number of road elements can be identified in a short time without manual annotation, saving manpower. Furthermore, the pre-trained 4-layer V-shaped network model combined with the difference of Gaussian operator can accurately identify occluded road arrows, improving the accuracy of road arrow recognition and thus enhancing the precision of high-precision maps.
[0099] Corresponding to the aforementioned application function implementation method embodiments, this application also provides a road arrow recognition device, an electronic device, and corresponding embodiments.
[0100] Figure 8 This is a schematic diagram of the road arrow recognition device shown in the embodiments of this application.
[0101] See Figure 8 The road arrow recognition device 90 provided in this application embodiment includes an image acquisition module 910, an image recognition module 920, and a type determination module 930, wherein:
[0102] The image acquisition module 910 is used to preprocess the pre-acquired road arrow image to obtain a road arrow image to be recognized that can be used by the road arrow recognition model.
[0103] Image recognition module 920 is used to input the road arrow image to be recognized into a preset road arrow recognition model to obtain the feature map of the road arrow image to be recognized, wherein the preset road arrow recognition model is a pre-trained 4-layer V-shaped network model combined with the difference of Gaussian operator.
[0104] The type determination module 930 is used to perform edge detection on the feature map using the difference of Gaussians operator to determine the edge of the road surface arrow to be identified and obtain the type of the road surface arrow to be identified.
[0105] In one possible implementation of this application, the preprocessing of the pre-acquired road arrow image to obtain a road arrow image to be recognized that can be used by the road arrow recognition model includes:
[0106] The pre-acquired road arrow image is processed by image segmentation, and the outline of the road arrow to be identified in the pre-acquired road arrow image after image segmentation is marked to obtain the road arrow image to be identified.
[0107] In one possible implementation of this application, the step of inputting the road arrow image to be identified into a preset road arrow recognition model to obtain a feature map of the road arrow image to be identified includes:
[0108] The image of the road arrow to be identified is converted into matrix input data of a preset size;
[0109] Feature extraction is performed on the matrix input data to obtain a feature-extracted image;
[0110] The feature-extracted image is upsampled to obtain an upsampled image;
[0111] The feature-extracted image is concatenated with the upsampled image to obtain the feature map of the road arrow image to be identified.
[0112] In one possible implementation of this application, feature extraction is performed on the matrix input data to obtain a feature-extracted image, including:
[0113] The matrix input data is subjected to a first feature extraction using a first feature extraction module to obtain a first feature extraction image, wherein the first feature extraction module includes a convolution module;
[0114] The first feature extraction image is subjected to a second feature extraction using a second feature extraction module to obtain a second feature extraction image, wherein the second feature extraction module includes a convolution module;
[0115] The second feature extraction image is subjected to a third feature extraction using a third feature extraction module to obtain a third feature extraction image. The third feature extraction module includes a convolution module.
[0116] In one possible implementation of this application, the feature extraction image is upsampled to obtain an upsampled image, including:
[0117] The third feature extraction image is upsampled for the first time to obtain a first upsampled intermediate image. The third feature extraction image is then processed by the difference of Gaussian operator and concatenated with the first upsampled intermediate image to obtain the first upsampled image.
[0118] The first upsampled image is upsampled a second time to obtain a second upsampled intermediate image. The second feature-extracted image is then processed by the difference of Gaussian operator and concatenated with the second upsampled intermediate image to obtain the second upsampled image.
[0119] The second upsampled image is upsampled a third time to obtain a third upsampled intermediate image. The first feature extraction image is then processed by the difference of Gaussian operator and concatenated with the third upsampled intermediate image to obtain the third upsampled image.
[0120] As one possible implementation of this application, in this implementation, the step of performing edge detection on the feature map using the difference of Gaussian operator to determine the edge of the road surface arrow to be identified, and obtaining the type of the road surface arrow to be identified, includes:
[0121] The pixel difference between the target road arrow region and the non-target road arrow region in the feature map is calculated using the difference of Gaussian operators, and the edge of the target road arrow is determined based on the pixel difference.
[0122] The outer polygon of the road surface arrow to be identified is obtained by performing an outer polygon operation based on the edge of the arrow.
[0123] As one possible implementation of this application, in this implementation, the step of performing a circumscribed polygon operation based on the edge of the road surface arrow to be identified, to obtain the circumscribed polygon of the road surface arrow to be identified, includes...
[0124] A preset corner point recognition model is used to identify multiple corner points of the edge of the road surface arrow to be identified in the feature map;
[0125] Based on the multiple corner points, a circumscribed polygon operation is performed to obtain the circumscribed polygon of the road surface arrow to be identified.
[0126] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated further here.
[0127] This application embodiment preprocesses the road arrow image to obtain the road arrow image to be identified, and uses a pre-trained 4-layer V-shaped network model combined with the difference of Gaussian operator to identify the road arrow image. The identified feature map is then edge-detected using the difference of Gaussian operator to determine the edges of the road arrows to be identified, thus determining the type of the road arrow. By using the network model to identify the road arrow image, a large number of road elements can be identified in a short time without manual annotation, saving manpower. Furthermore, the pre-trained 4-layer V-shaped network model combined with the difference of Gaussian operator can accurately identify occluded road arrows, improving the accuracy of road arrow recognition and thus enhancing the precision of high-precision maps.
[0128] The following is for reference. Figure 9The diagram illustrates a structural schematic of an electronic device 1000 suitable for implementing embodiments of the present disclosure. The terminal devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 9 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.
[0129] The electronic device includes a memory and a processor, wherein the processor may be referred to as processing device 1001 as described below, and the memory may include at least one of read-only memory (ROM) 1002, random access memory (RAM) 1003, and storage device 1008 as described below, as follows:
[0130] like Figure 9 As shown, the electronic device 1000 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1008 into a random access memory (RAM) 1003. The RAM 1003 also stores various programs and data required for the operation of the electronic device 1000. The processing unit 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.
[0131] Typically, the following devices can be connected to the I / O interface 1005: input devices 1006 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 1007 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1008 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows electronic device 1000 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 9 An electronic device 1000 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.
[0132] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 1009, or installed from storage device 1008, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of embodiments of this disclosure.
[0133] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.
[0134] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.
[0135] The aforementioned computer-readable medium carries one or more programs. When the aforementioned one or more programs are executed by the electronic device, the electronic device causes the following actions: preprocessing a pre-acquired road arrow image to obtain a road arrow image to be recognized, which can be used by a road arrow recognition model; inputting the road arrow image to be recognized into a preset road arrow recognition model to obtain a feature map of the road arrow image to be recognized, wherein the preset road arrow recognition model is a pre-trained 4-layer V-shaped network model combined with a Gaussian difference operator; and using the Gaussian difference operator to perform edge detection on the feature map to determine the edges of the road arrow to be recognized, thereby obtaining the type of the road arrow to be recognized.
[0136] Computer program code for performing the operations of this disclosure can be written in one or more programming languages or a combination thereof, including but not limited to object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0137] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0138] The modules or units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules or units do not necessarily limit the specific unit itself.
[0139] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.
[0140] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0141] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.
[0142] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.
[0143] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.
Claims
1. A method for recognizing road arrows, characterized in that, include: The pre-acquired road arrow images are pre-processed to obtain road arrow images to be recognized that can be used by the road arrow recognition model. The road arrow image to be identified is input into a preset road arrow recognition model to obtain the feature map of the road arrow image to be identified. The preset road arrow recognition model is a pre-trained 4-layer V-shaped network model that incorporates the difference of Gaussian operator. The Gaussian difference operator is used to perform edge detection on the feature map to determine the edges of the road surface arrow to be identified, thereby obtaining the type of the road surface arrow to be identified; The preset road arrow recognition model is used for: The image of the road arrow to be identified is converted into matrix input data of a preset size; The matrix input data is subjected to a first feature extraction module to perform a first feature extraction, resulting in a first feature extraction image, wherein the first feature extraction module includes a convolution module; the first feature extraction image is subjected to a second feature extraction module to perform a second feature extraction, resulting in a second feature extraction image, wherein the second feature extraction module includes a convolution module; the second feature extraction image is subjected to a third feature extraction module to perform a third feature extraction, resulting in a third feature extraction image, wherein the third feature extraction module includes a convolution module. The third feature extraction image is upsampled for the first time to obtain a first upsampled intermediate image. The third feature extraction image is then passed through a Gaussian difference operator and concatenated with the first upsampled intermediate image to obtain a first upsampled image. The first upsampled image is then upsampled for the second time to obtain a second upsampled intermediate image. The second feature extraction image is then passed through a Gaussian difference operator and concatenated with the second upsampled intermediate image to obtain a second upsampled image. The second upsampled image is then upsampled for the third time to obtain a third upsampled intermediate image. The first feature extraction image is then passed through a Gaussian difference operator and concatenated with the third upsampled intermediate image to obtain a third upsampled image.
2. The road arrow recognition method according to claim 1, characterized in that, The preprocessing of the pre-acquired road arrow image to obtain a road arrow image to be recognized that can be used by the road arrow recognition model includes: The pre-acquired road arrow image is processed by image segmentation, and the outline of the road arrow to be identified in the pre-acquired road arrow image after image segmentation is marked to obtain the road arrow image to be identified.
3. The road arrow recognition method according to claim 2, characterized in that, The step of inputting the road arrow image to be identified into a preset road arrow recognition model to obtain the feature map of the road arrow image includes: The image of the road arrow to be identified is converted into matrix input data of a preset size; Feature extraction is performed on the matrix input data to obtain a feature-extracted image; The feature-extracted image is upsampled to obtain an upsampled image; The feature-extracted image is concatenated with the upsampled image to obtain the feature map of the road arrow image to be identified.
4. The road arrow recognition method according to claim 1, characterized in that, The step of using the Gaussian difference operator to perform edge detection on the feature map, determining the edges of the road surface arrow to be identified, and obtaining the type of the road surface arrow to be identified includes: The pixel difference between the target road arrow region and the non-target road arrow region in the feature map is calculated using the difference of Gaussian operators, and the edge of the target road arrow is determined based on the pixel difference. The outer polygon of the road surface arrow to be identified is obtained by performing an outer polygon operation based on the edge of the arrow.
5. The road arrow recognition method according to claim 4, characterized in that, The circumscribed polygon operation is performed based on the edges of the road surface arrow to be identified, resulting in the circumscribed polygon of the road surface arrow to be identified, including... A preset corner point recognition model is used to identify multiple corner points of the edge of the road surface arrow to be identified in the feature map; Based on the multiple corner points, a circumscribed polygon operation is performed to obtain the circumscribed polygon of the road surface arrow to be identified.
6. A road arrow recognition device, characterized in that, include: The image acquisition module is used to preprocess the pre-acquired road arrow image to obtain a road arrow image to be recognized that can be used by the road arrow recognition model. The image recognition module is used to input the road arrow image to be recognized into a preset road arrow recognition model to obtain the feature map of the road arrow image to be recognized. The preset road arrow recognition model is a pre-trained 4-layer V-shaped network model that incorporates the difference of Gaussian operator. The type determination module is used to perform edge detection on the feature map using the difference of Gaussians operator to determine the edge of the road surface arrow to be identified, and to obtain the type of the road surface arrow to be identified; The preset road arrow recognition model is used for: The image of the road arrow to be identified is converted into matrix input data of a preset size; The matrix input data is subjected to a first feature extraction module to perform a first feature extraction, resulting in a first feature extraction image, wherein the first feature extraction module includes a convolution module; the first feature extraction image is subjected to a second feature extraction module to perform a second feature extraction, resulting in a second feature extraction image, wherein the second feature extraction module includes a convolution module; the second feature extraction image is subjected to a third feature extraction module to perform a third feature extraction, resulting in a third feature extraction image, wherein the third feature extraction module includes a convolution module. The third feature extraction image is upsampled for the first time to obtain a first upsampled intermediate image. The third feature extraction image is then passed through a Gaussian difference operator and concatenated with the first upsampled intermediate image to obtain a first upsampled image. The first upsampled image is then upsampled for the second time to obtain a second upsampled intermediate image. The second feature extraction image is then passed through a Gaussian difference operator and concatenated with the second upsampled intermediate image to obtain a second upsampled image. The second upsampled image is then upsampled for the third time to obtain a third upsampled intermediate image. The first feature extraction image is then passed through a Gaussian difference operator and concatenated with the third upsampled intermediate image to obtain a third upsampled image.
7. An electronic device, characterized in that, include: processor; as well as A memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the method as described in any one of claims 1-5.
8. A computer-readable storage medium having executable code stored thereon, which, when executed by a processor of an electronic device, causes the processor to perform the method as described in any one of claims 1-5.