Traffic keyword recognition method and device, equipment, storage medium and program product
By extracting and identifying the text content in traffic signs on low-computing equipment, the difficulty of running complex deep learning models on low-computing equipment for traffic keyword recognition is solved, and efficient and accurate traffic keyword recognition is achieved.
Patent Information
- Application Number
- CN202411923301.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-24
- Publication Date
- 2025-04-25
AI Technical Summary
There are difficulties in running complex deep learning models on low-computing devices for traffic keyword identification, and in complex traffic environments, image occlusion, incompleteness and blurring increase the difficulty of recognition, resulting in a decrease in accuracy.
By extracting traffic sign images from the collected images, sign type recognition is performed, target traffic sign images containing text content are filtered out, and keyword recognition is used using a pre-trained traffic keyword recognition model. This method focuses on the text content in traffic signs, reducing the amount of data processed and computational complexity.
It realizes efficient traffic keyword recognition on edge devices or low-computing equipment, reduces complexity, improves recognition accuracy, and solves the problem of efficient text recognition in low-resource environments.
Smart Images
Figure CN119888706A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of traffic information processing, and in particular to a traffic keyword recognition method, device, equipment, storage medium and program product. Background Art
[0002] At present, the update of electronic maps is to collect data through crowdsourcing, and update the map through automated processing to maintain the accuracy and timeliness of map information. The data collected is mainly in the format of pictures and videos, and then automated processing is completed through artificial intelligence and big data technology to realize the recognition and extraction of the location, geometry and attributes of various elements in the map. In this process, the recognition of text and symbols is an important link. Text is not only used to identify basic information such as place names and road names in the map, but also to provide rich semantic information, such as traffic rules information such as driving direction, lane changes and speed limits.
[0003] The automatic recognition of traffic keywords can capture and identify key information in traffic scenes. The automatic recognition of traffic keywords is a deeper understanding and application of traffic scenes, which facilitates the judgment of complex scenes in various automated processing processes, thereby improving the automation rate of crowdsourcing updates and the speed of map updates.
[0004] Currently, deep learning-based text recognition models are used for traffic keyword recognition, but the limited processing power and memory of low-computing devices make it difficult to run complex deep learning models. Summary of the invention
[0005] The embodiments of the present application provide a traffic keyword recognition method, apparatus, device, storage medium, and program product to achieve the effect of traffic keyword recognition on edge devices or low-computing power devices.
[0006] In a first aspect, an embodiment of the present application provides a method for identifying traffic keywords, including:
[0007] Extracting traffic sign images from the collected images;
[0008] Performing sign type recognition on the traffic sign image to obtain a sign type recognition result;
[0009] Based on the sign type recognition result, extracting a target traffic sign image, wherein the target traffic sign image is a traffic sign image containing text content;
[0010] Traffic keywords in the target traffic sign image are identified to obtain keyword recognition results.
[0011] In a possible implementation manner, the identifying the traffic keyword in the target traffic sign image to obtain the keyword recognition result includes:
[0012] The target traffic sign image is input into a pre-trained traffic keyword recognition model for traffic keyword recognition to obtain the keyword recognition result. The traffic keyword recognition model is used to detect the position of each text line in the target traffic sign image and identify the text content of each detected text line to obtain the keyword recognition result. The keyword recognition result includes a target keyword category to which each text line in the target traffic sign image belongs. The target keyword category is a predefined keyword category among multiple predefined keyword categories that matches the text line of the target traffic sign image.
[0013] In a possible implementation manner, extracting a traffic sign image from a collected image includes:
[0014] Inputting the image into a pre-trained sign detection model to perform traffic sign detection, and obtaining a sign detection result, wherein the sign detection result includes position information of the detected traffic sign in the image;
[0015] The traffic sign image is cropped from the image based on the sign detection result.
[0016] In a possible implementation, performing sign type recognition on the traffic sign image to obtain a sign type recognition result includes:
[0017] The traffic sign image is input into a pre-trained sign classification model to perform sign type recognition to obtain a sign type recognition result, wherein the sign type recognition result includes a target sign category to which the traffic sign image belongs, and the target sign category is a predefined sign category among multiple predefined sign categories that matches the traffic sign image.
[0018] In a possible implementation, extracting a target traffic sign image based on the sign type recognition result includes:
[0019] Querying a predefined mapping relationship between a sign category and whether it contains text content, and determining whether the target sign category of the sign recognition result contains text content;
[0020] If the target sign category of the sign recognition result contains text content, the corresponding traffic sign image is determined as the target traffic sign image.
[0021] In a possible implementation, the method further includes:
[0022] The keyword recognition result is sent to a cloud, so that the cloud updates the attribute information of the map element based on the keyword recognition result.
[0023] In a possible implementation manner, the predefined keyword category is obtained in the following manner:
[0024] Performing text recognition on the acquired traffic sign image data to obtain text recognition results;
[0025] Performing clustering processing on the text recognition results, and generating preliminary keyword categories according to the clustering results;
[0026] The preliminary keyword category is adjusted to obtain the predefined keyword category.
[0027] In a second aspect, an embodiment of the present application provides a traffic keyword recognition device, comprising:
[0028] A first extraction module, used to extract a traffic sign image from the collected image;
[0029] A first recognition module is used to perform sign type recognition on the traffic sign image to obtain a sign type recognition result;
[0030] A second extraction module is used to extract a target traffic sign image based on the sign type recognition result, wherein the target traffic sign image is a traffic sign image containing text content;
[0031] The second recognition module is used to recognize traffic keywords in the target traffic sign image and obtain keyword recognition results.
[0032] In a possible implementation manner, the second identification module is specifically configured to:
[0033] The target traffic sign image is input into a pre-trained traffic keyword recognition model for traffic keyword recognition to obtain the keyword recognition result. The traffic keyword recognition model is used to detect the position of each text line in the target traffic sign image and identify the text content of each detected text line to obtain the keyword recognition result. The keyword recognition result includes a target keyword category to which each text line in the target traffic sign image belongs. The target keyword category is a predefined keyword category among multiple predefined keyword categories that matches the text line of the target traffic sign image.
[0034] In a possible implementation manner, the first extraction module is specifically configured to:
[0035] Inputting the image into a pre-trained sign detection model to perform traffic sign detection, and obtaining a sign detection result, wherein the sign detection result includes position information of the detected traffic sign in the image;
[0036] The traffic sign image is cropped from the image based on the sign detection result.
[0037] In a possible implementation manner, the first identification module is specifically configured to:
[0038] The traffic sign image is input into a pre-trained sign classification model to perform sign type recognition to obtain a sign type recognition result, wherein the sign type recognition result includes a target sign category to which the traffic sign image belongs, and the target sign category is a predefined sign category among multiple predefined sign categories that matches the traffic sign image.
[0039] In a possible implementation manner, the second extraction module is specifically configured to:
[0040] Querying a predefined mapping relationship between a sign category and whether it contains text content, and determining whether the target sign category of the sign recognition result contains text content;
[0041] If the target sign category of the sign recognition result contains text content, the corresponding traffic sign image is determined as the target traffic sign image.
[0042] In a possible implementation manner, the traffic keyword recognition device is further used for:
[0043] The keyword recognition result is sent to a cloud, so that the cloud updates the attribute information of the map element based on the keyword recognition result.
[0044] In a possible implementation manner, the predefined keyword category is obtained in the following manner:
[0045] Performing text recognition on the acquired traffic sign image data to obtain text recognition results;
[0046] Performing clustering processing on the text recognition results, and generating preliminary keyword categories according to the clustering results;
[0047] The preliminary keyword category is adjusted to obtain the predefined keyword category.
[0048] In a third aspect, an embodiment of the present application provides a traffic keyword recognition device, including: a memory, a processor;
[0049] The memory stores computer-executable instructions;
[0050] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.
[0051] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.
[0052] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.
[0053] The traffic keyword recognition method, device, equipment, storage medium and program product provided in the embodiments of the present application can extract traffic sign images from the collected images, and then perform sign type recognition on these traffic sign images to obtain sign type recognition results. Based on the sign type recognition results, target traffic sign images containing text content can be extracted from these traffic sign images, and traffic keyword recognition can be performed on the target traffic sign images. The recognition of traffic keywords in this application is only for traffic signs rather than for the entire scene. The recognition of traffic signs is only for traffic signs containing text content rather than for all traffic signs. Through this focusing method, the complexity of the traffic keyword recognition task is greatly reduced. The effect of realizing traffic keyword recognition on edge devices or low-computing-power devices is achieved. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0055] Figure 1 A schematic diagram of a scenario for the traffic keyword recognition method provided in this application;
[0056] Figure 2 Schematic diagram of the traffic keyword identification method provided in this application Figure 1 ;
[0057] Figure 3 Schematic diagram of the traffic keyword identification method provided in this application Figure 2 ;
[0058] Figure 4 A schematic diagram of the sign detection results provided in the embodiment of the present application;
[0059] Figure 5 A schematic diagram of an image coordinate system provided in an embodiment of the present application;
[0060] Figure 6 A schematic diagram of a keyword recognition result provided in an embodiment of the present application;
[0061] Figure 7 A process diagram of the traffic keyword identification method provided for this application;
[0062] Figure 8 A schematic diagram of the structure of the traffic keyword recognition device provided by this application;
[0063] Fig. 9 This is a schematic diagram of the structure of the traffic keyword recognition device provided in this application.
[0064] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION
[0065] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.
[0066] Scene Text Recognition (STR) refers to the recognition of text information in natural scene images. In the images taken by the vehicle dashcam, this technology can be used to recognize and extract the text content in the image, such as license plate numbers, text on traffic signs, etc.
[0067] With the rapid development of artificial intelligence technology, natural scene text recognition is usually implemented using a text recognition model based on deep learning. However, the following problems are encountered during the application process:
[0068] 1. The operating efficiency of the text recognition model on low-computing power devices is significantly limited, which is time-consuming and inefficient. Some model operators cannot run due to device compatibility issues, which directly hinders the rapid engineering application of the model and makes it a major problem to achieve efficient text recognition in a low-resource environment.
[0069] 2. In complex traffic environments, the collected images often face multiple challenges such as occlusion, incompleteness, and blur. These factors not only increase the difficulty of text detection, but also lead to a significant decrease in the accuracy of text content recognition.
[0070] The traffic keyword recognition method provided in the present application can extract traffic sign images from the collected images, and then perform sign type recognition on these traffic sign images to obtain sign type recognition results. Based on the sign type recognition results, target traffic sign images containing text content can be extracted from these traffic sign images, and traffic keyword recognition can be performed on the target traffic sign images. The recognition of traffic keywords in the present application is only for the recognition of traffic signs rather than the recognition of the entire scene. The recognition of traffic signs is only for the recognition of traffic signs containing text content rather than the recognition of all traffic signs. Through this focusing method, the complexity of the traffic keyword recognition task is greatly reduced, and the technical problem of difficulty in achieving efficient text recognition in a low-resource environment is solved.
[0071] Figure 1 A schematic diagram of a traffic keyword recognition method provided in this application, such as Figure 1 As shown, in this application scenario, the driving recorder in the vehicle 101 shoots the road and the surrounding environment to obtain corresponding images. For the collected images, the driving recorder can extract traffic sign images from them, and further identify the type of the extracted traffic sign images. Based on the sign type identification result, the target traffic sign image containing text content is extracted from the traffic sign image, and then the target traffic sign image is identified by traffic keywords to obtain keyword recognition results. The driving recorder can send the obtained keyword recognition results to the cloud 102. The cloud can use the obtained keyword recognition results to update the map.
[0072] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.
[0073] Figure 2 Schematic diagram of the traffic keyword identification method provided in this application Figure 1 ,like Figure 2 As shown, the method includes:
[0074] S201: extracting a traffic sign image from the collected image.
[0075] In the embodiment of the present application, a small image of a traffic sign can be cropped from the original image. The extracted traffic sign image can be preprocessed, such as denoising, adjusting brightness and contrast, to improve the accuracy of subsequent recognition.
[0076] Traffic signs, also known as traffic signs, are used to convey information about road traffic rules, warnings, and guidance to road users to ensure smooth and safe traffic. Traffic signs can be divided into types such as instruction signs, warning signs, and prohibition signs. Instruction signs are mainly used to indicate road information such as directions, place names, and distances to help drivers understand their driving routes. Warning signs are used to alert drivers to potential dangers or abnormal conditions ahead so that timely measures can be taken to avoid accidents. For example, sharp turn signs, continuous downhill signs, etc. Prohibition signs are mainly used to prohibit or restrict certain traffic behaviors to ensure traffic order and safety. For example, no parking signs, no honking signs, etc.
[0077] S202: Perform sign type recognition on the traffic sign image to obtain a sign type recognition result.
[0078] The sign type recognition result indicates the sign category to which the traffic sign image belongs.
[0079] S203: extracting a target traffic sign image based on the sign type recognition result, where the target traffic sign image is a traffic sign image containing text content.
[0080] According to the sign type recognition result, traffic sign images containing text content are screened out, and traffic sign images containing no text content are filtered out, that is, traffic sign images that do not require text recognition are filtered out.
[0081] S204: Identify traffic keywords in the target traffic sign image and obtain keyword recognition results.
[0082] In the related art, keyword recognition is directly performed on the collected original image, and all text information in the image will be recognized, such as the text on the billboard, the text on the license plate, etc. In the embodiment of the present application, traffic keyword recognition is not directly performed on the collected original image, but the processing scope is gradually narrowed, and traffic keyword recognition is only performed on the target traffic sign image after the sign category is identified. By focusing on some traffic signs for keyword recognition, the amount of data to be processed and the computational complexity can be significantly reduced.
[0083] The traffic keyword recognition method provided in the embodiment of the present application can extract traffic sign images from the collected images, and then perform sign type recognition on these traffic sign images to obtain sign type recognition results. Based on the sign type recognition results, target traffic sign images containing text content can be extracted from these traffic sign images, and traffic keyword recognition can be performed on the target traffic sign images. The recognition of traffic keywords in this application is only for traffic signs rather than for the entire scene. The recognition of traffic signs is only for traffic signs containing text content rather than for all traffic signs. Through this focusing method, the complexity of the traffic keyword recognition task is greatly reduced. The effect of realizing traffic keyword recognition on edge devices or low-computing-power devices is achieved.
[0084] Figure 3 Schematic diagram of the traffic keyword identification method provided in this application Figure 2 ,like Figure 3 As shown, in this embodiment Figure 2 Based on the embodiment, a traffic keyword recognition method is described in detail, and the method includes:
[0085] S301: extracting a traffic sign image from the collected image.
[0086] In a possible implementation, extracting a traffic sign image from a captured image may specifically include:
[0087] Input the image into a pre-trained sign detection model to detect traffic signs and obtain a sign detection result, which includes the position information of the detected traffic signs in the image;
[0088] The traffic sign image is cropped from the image based on the sign detection result.
[0089] The sign detection model can be a lightweight deep learning target detection model with low computing power requirements, which can be a lightweight YOLO model, NanoDet model, MobileDet model, PP-PicoDet model, etc. By adopting a lightweight deep learning target detection model, efficient target detection can be achieved on low computing power devices to meet the requirements of timeliness and accuracy.
[0090] In this embodiment, the collected complete original image can be input into a pre-trained sign detection model. The sign detection model is trained to recognize traffic signs and output their position information in the original image. Then, each detected traffic sign image can be cropped from the original image using the position information in the sign detection result.
[0091] Reference Figure 4As shown, it is a schematic diagram of the sign detection result provided in an embodiment of the present application. The sign detection result can be represented in the form of a detection box. The green box in the figure is the visualization result of drawing the sign detection result into the original image.
[0092] Reference Figure 5 As shown, it is a schematic diagram of the image coordinate system provided by the embodiment of the present application. In the image coordinate system (with the upper left corner of the image as the origin of the coordinate system, the horizontal right direction is the positive direction of the x-axis, and the vertical downward direction is the positive direction of the y-axis), each detection frame can be described by an array in terms of its position and size in the image. The array format can be bbox(x, y, w, h), where x represents the horizontal coordinate of the upper left corner of the detection frame, which represents the distance from the left edge of the image to the left edge of the detection frame. y represents the vertical coordinate of the upper left corner of the detection frame, which represents the distance from the upper edge of the image to the upper edge of the detection frame. w represents the width of the detection frame, which represents the length of the detection frame in the x-axis direction. h represents the height of the detection frame, which represents the length of the detection frame in the y-axis direction.
[0093] In addition, template matching methods can also be used to extract traffic sign images from the collected images. Traffic signs have specific shapes, such as circles, triangles, rectangles, etc. Therefore, shape features can be used to extract traffic signs. For example, edge detection algorithms can be used to extract edge information in images, and the extracted edge information can be matched with a preset traffic sign shape template to extract traffic sign images.
[0094] S302: Perform sign type recognition on the traffic sign image to obtain a sign type recognition result.
[0095] In a possible implementation, performing sign type recognition on a traffic sign image to obtain a sign type recognition result may specifically include:
[0096] The traffic sign image is input into a pre-trained sign classification model to perform sign type recognition to obtain a sign type recognition result. The sign type recognition result includes a target sign category to which the traffic sign image belongs. The target sign category is a predefined sign category that matches the traffic sign image among multiple predefined sign categories.
[0097] The sign classification model can be a lightweight deep learning image classification model with low computing power requirements, which can be a lightweight ResNet model, a MobileNet model, an EfficientNet model, etc. By adopting a lightweight deep learning image classification model, efficient traffic sign classification can be achieved on low computing power devices to meet the requirements of timeliness and accuracy.
[0098] In this embodiment, the traffic sign image can be input into a pre-trained sign classification model. The sign classification model is trained to identify the target sign category to which the traffic sign image belongs.
[0099] During the model training phase, a set of sign categories can be defined. These categories can cover various sign types, such as speed limit signs, straight ahead signs, parking lot signs, no parking signs, etc. Signs can be classified according to color or shape or the traffic meaning they express. Figure 4 As shown in the figure, 0A0001 is the category code of the sign category to which the blue sign belongs. The category code is a unique identifier used to identify different sign categories.
[0100] Since there are many types of signs in the real world, we first use a target detection model to distinguish traffic signs from other geographical elements (such as traffic lights), and then train a classification model separately to complete the distinction between specific categories of signs, thereby decoupling the target detection task and the classification task. This can effectively cope with complex sign recognition tasks and ensure recognition results.
[0101] In addition, the traffic sign image is subjected to sign type recognition to obtain a sign type recognition result, and a template matching method can also be used. The traffic sign image can be compared with pre-stored traffic sign templates corresponding to each sign type, and the sign type to which the traffic sign image belongs can be determined by calculating the similarity.
[0102] S303: extracting a target traffic sign image based on the sign type recognition result, where the target traffic sign image is a traffic sign image containing text content.
[0103] In a possible implementation, based on the sign type recognition result, extracting the target traffic sign image may specifically include:
[0104] Query the mapping relationship between the predefined sign category and whether it contains text content, and determine whether the target sign category of the sign recognition result contains text content;
[0105] If the target sign category of the sign recognition result contains text content, the corresponding traffic sign image is determined as the target traffic sign image.
[0106] In this embodiment, a database or data structure may be created, which includes all predefined sign categories and their attributes, including an identification of whether text content is included. According to the target sign category in the sign recognition result, a mapping relationship between the predefined sign category and whether it contains text content is queried. If the query result shows that the target sign category contains text content, the corresponding traffic sign image is determined as the target traffic sign image.
[0107] S304. Input the target traffic sign image into a pre-trained traffic keyword recognition model to perform traffic keyword recognition and obtain a keyword recognition result. The traffic keyword recognition model is used to detect the position of each text line in the target traffic sign image and recognize the text content of each detected text line to obtain a keyword recognition result. The keyword recognition result includes a target keyword category to which each text line in the target traffic sign image belongs. The target keyword category is a predefined keyword category among multiple predefined keyword categories that matches the text line of the target traffic sign image.
[0108] In traffic keyword recognition, it is crucial to understand and interpret traffic keywords. These keywords often appear on signs and convey specific traffic status or information. Here are some common traffic keywords and their explanations:
[0109] Emergency lane: An area on the right side of the road adjacent to the driving lane for driving or parking in emergency situations.
[0110] Construction Ahead: Indicates that construction is ongoing on the road ahead.
[0111] School intersection: There is a school ahead, please slow down and pay attention to the safety of pedestrians, especially students.
[0112] Auxiliary road: Indicates that the road you are currently traveling on is an auxiliary road rather than a main road. It is usually used to divert traffic or provide an additional passage.
[0113] Emergency parking strip: Indicates that there is a special emergency parking area for vehicles to use in emergency situations.
[0114] Small passenger car lane: This lane is marked as being reserved for small passenger cars and may have specific speed requirements.
[0115] In this embodiment, the extracted target traffic sign image can be input into a pre-trained traffic keyword recognition model. The traffic keyword recognition model is trained to detect text lines and recognize traffic keywords for each detected text line. The traffic keyword recognition model can output a target keyword category to which each text line belongs, where the target keyword category is selected from a plurality of predefined keyword categories and matches the content of the text line.
[0116] The design of traffic signs usually follows the principle that each text line corresponds to one type of traffic information. This design is to ensure that drivers can quickly and accurately obtain and understand the key information on the sign while driving. In this embodiment, by detecting the position of the text line and locating the text line, it is possible to effectively avoid confusing the characters of adjacent text lines during the recognition process, thereby improving the recognition accuracy. The position information of the text line is not only used in the current recognition process, but also in subsequent applications. For example, by combining the recognized text content with the position information, the map data can be automatically updated.
[0117] The keyword recognition result may also include the confidence that the text line belongs to the target keyword category and the position information of the text line in the input target traffic sign image.
[0118] Reference Figure 6 As shown, it is a schematic diagram of the keyword recognition result provided in the embodiment of the present application. The colored box in the figure is the visualization result of drawing the keyword recognition result into the target traffic sign image. A bounding box can be constructed for each detected text line. The color of the box can be encoded according to the keyword category to which the text line content belongs, so as to distinguish different categories. The text format above the bounding box is "Keyword category number: confidence of traffic keyword recognition model". Among them, the keyword category number represents the number of the predefined keyword category to which the text line belongs. The confidence of the traffic keyword recognition model represents the confidence of the model in the keyword recognition result of the text line, which is usually a value between 0 and 1. The closer to 1, the more reliable the model result. The position of the bounding box can be defined using the image coordinate system. The specific definition method can be as follows. Figure 5 shown.
[0119] In addition, in order to identify traffic keywords in the target traffic sign image and obtain keyword recognition results, a template matching method can be used. By utilizing image processing technology, the area containing characters can be segmented from the traffic sign image, and the detected character area can be matched with a preset character template. By calculating the similarity, the character template that best matches the character area can be determined, thereby obtaining keyword recognition results.
[0120] In a possible implementation, the predefined keyword categories may be obtained in the following manner:
[0121] Performing text recognition on the acquired traffic sign image data to obtain text recognition results;
[0122] Perform clustering on the text recognition results, and generate preliminary keyword categories based on the clustering results;
[0123] The preliminary keyword categories are adjusted to obtain predefined keyword categories.
[0124] In this embodiment, a large amount of traffic sign image data can be obtained, and text recognition can be performed on the traffic sign images in the traffic sign image data. Then, the recognized text content can be clustered using a text clustering algorithm (such as K-Means). The purpose of clustering is to classify similar text content as belonging to the same category, thereby generating preliminary keyword categories. For example, small cars / small lanes / small passenger cars / small passenger cars / small passenger lanes / special for small passenger cars / special for small cars can be classified as belonging to the first category; slow down / slow down / please slow down / vehicle slow down / slow driving / please slow down to avoid can be classified as the second category; road construction / front construction / construction area / subway construction / auxiliary road construction / construction period / closed construction / road station construction / bridge construction / construction congestion can also be classified as the third category. The preliminary keyword categories are adjusted, specifically, texts with the same meaning but divided into different categories (such as "school intersection" and "school ahead") are merged, and texts with different meanings but mistakenly classified into the same category (such as "no parking" and "no entry") are separated, thereby generating predefined keyword categories.
[0125] Among them, the preliminary keyword categories can also be adjusted according to the user demand data and the map production data to generate predefined keyword categories. The user demand data is the user's recognition demand data for traffic signs, such as which traffic sign information is more important to the user. The first keyword demand information can be extracted from the user demand data, and the preliminary keyword categories can be adjusted according to the first keyword demand information. Map production data refers to the keyword data in the map production process. The map production data records the keywords identified in the map production process and the frequency of occurrence of each keyword. The second keyword demand information can be determined based on the map production data. The second keyword demand information can include keywords with high frequency of occurrence and high contribution to map updates. According to the second keyword demand information, the preliminary keyword categories are adjusted. Through the above method, it can be ensured that the predefined keyword categories can cover user needs and high-frequency keywords in the production process, and the keyword recognition effect is guaranteed.
[0126] By performing statistical analysis on traffic keywords, we can effectively define the predefined keyword categories for the traffic keyword recognition model. We then use the labeled dataset to train the traffic keyword recognition model. This ensures that the model can recognize and classify the keywords in the input image into the predefined keyword categories.
[0127] In a possible implementation manner, the traffic keyword identification method of the embodiment of the present application may further include:
[0128] The keyword recognition results are sent to the cloud, so that the cloud updates the attribute information of the map features based on the keyword recognition results.
[0129] The traffic keyword recognition method of the embodiment of the present application can be specifically executed on a vehicle driving recorder. After the vehicle driving recorder generates a keyword recognition result, it can send the keyword recognition result to a cloud server through a secure network protocol to update the relevant map element attributes in the cloud map database. The traffic keyword recognition method of the embodiment of the present application can be applied to crowdsourcing mapping scenarios.
[0130] Reference Figure 7 As shown, it is a process schematic diagram of the traffic keyword recognition method provided by the present application, in which sign detection is performed on the collected image to extract the traffic sign image, and the extracted traffic sign image is further identified by the sign type, and the target traffic sign image containing text content is extracted from the traffic sign image based on the sign type recognition result, and then text line detection is performed on the target traffic sign image and traffic keyword recognition is performed on the detected text line to obtain the keyword recognition result.
[0131] The traffic keyword recognition method provided in the embodiment of the present application uses deep learning target detection technology to ensure the automatic extraction and matching of text content, and adopts a basic target detection model to process complex text recognition tasks, thereby achieving lightweight calculations and ensuring efficient operation on low-computing power devices.
[0132] The traffic keyword recognition method provided in the embodiment of the present application can extract traffic sign images from the collected images, and then perform sign type recognition on these traffic sign images to obtain sign type recognition results. Based on the sign type recognition results, target traffic sign images containing text content can be extracted from these traffic sign images, and traffic keyword recognition can be performed on the target traffic sign images. The recognition of traffic keywords in this application is only for traffic signs rather than for the entire scene. The recognition of traffic signs is only for traffic signs containing text content rather than for all traffic signs. Through this focusing method, the complexity of the traffic keyword recognition task is greatly reduced. The effect of realizing traffic keyword recognition on edge devices or low-computing-power devices is achieved.
[0133] Figure 8 A schematic diagram of the structure of the traffic keyword recognition device provided by this application, such as Figure 8 As shown, the traffic keyword recognition device 80 provided in this embodiment includes:
[0134] The first extraction module 801 is used to extract the traffic sign image from the collected image;
[0135] The first recognition module 802 is used to perform sign type recognition on the traffic sign image to obtain a sign type recognition result;
[0136] The second extraction module 803 is used to extract a target traffic sign image based on the sign type recognition result, where the target traffic sign image is a traffic sign image containing text content;
[0137] The second recognition module 804 is used to recognize traffic keywords in the target traffic sign image and obtain keyword recognition results.
[0138] In a possible implementation manner, the second identification module is specifically used to:
[0139] The target traffic sign image is input into a pre-trained traffic keyword recognition model for traffic keyword recognition to obtain a keyword recognition result. The traffic keyword recognition model is used to detect the position of each text line in the target traffic sign image and identify the text content of each detected text line to obtain a keyword recognition result. The keyword recognition result includes a target keyword category to which each text line in the target traffic sign image belongs. The target keyword category is a predefined keyword category among multiple predefined keyword categories that matches the text line of the target traffic sign image.
[0140] In a possible implementation, the first extraction module is specifically used to:
[0141] Input the image into a pre-trained sign detection model to detect traffic signs and obtain a sign detection result, which includes the position information of the detected traffic signs in the image;
[0142] The traffic sign image is cropped from the image based on the sign detection result.
[0143] In a possible implementation manner, the first identification module is specifically used to:
[0144] The traffic sign image is input into a pre-trained sign classification model to perform sign type recognition to obtain a sign type recognition result. The sign type recognition result includes a target sign category to which the traffic sign image belongs. The target sign category is a predefined sign category that matches the traffic sign image among multiple predefined sign categories.
[0145] In a possible implementation, the second extraction module is specifically used to:
[0146] Query the mapping relationship between the predefined sign category and whether it contains text content, and determine whether the target sign category of the sign recognition result contains text content;
[0147] If the target sign category of the sign recognition result contains text content, the corresponding traffic sign image is determined as the target traffic sign image.
[0148] In a possible implementation, the traffic keyword recognition device is further used to:
[0149] The keyword recognition results are sent to the cloud, so that the cloud updates the attribute information of the map features based on the keyword recognition results.
[0150] In a possible implementation, the predefined keyword category is obtained in the following manner:
[0151] Performing text recognition on the acquired traffic sign image data to obtain text recognition results;
[0152] Perform clustering on the text recognition results, and generate preliminary keyword categories based on the clustering results;
[0153] The preliminary keyword categories are adjusted to obtain predefined keyword categories.
[0154] The traffic keyword recognition device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be described in detail here.
[0155] Fig. 9 This is a schematic diagram of the structure of the traffic keyword recognition device provided by this application. Fig. 9 As shown, the traffic keyword recognition device 90 provided in this embodiment includes: at least one processor 901 and a memory 902. Optionally, the device 90 also includes a communication component 903. The processor 901, the memory 902 and the communication component 903 are connected via a bus.
[0156] In a specific implementation process, at least one processor 901 executes the computer execution instructions stored in the memory 902, so that at least one processor 901 executes the above method.
[0157] The specific implementation process of the processor 901 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.
[0158] In the above embodiments, it should be understood that the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the invention can be directly implemented as a hardware processor, or can be implemented by a combination of hardware and software modules in the processor.
[0159] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.
[0160] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.
[0161] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.
[0162] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.
[0163] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.
[0164] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.
[0165] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.
[0166] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0167] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0168] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.
[0169] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.
[0170] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.
Claims
1. A traffic keyword recognition method, characterized in that: include: Extracting traffic sign images from the collected images; Performing sign type recognition on the traffic sign image to obtain a sign type recognition result; Based on the sign type recognition result, extracting a target traffic sign image, wherein the target traffic sign image is a traffic sign image containing text content; Traffic keywords in the target traffic sign image are identified to obtain keyword recognition results.
2. The method according to claim 1, characterized in that The step of identifying the traffic keywords in the target traffic sign image and obtaining the keyword recognition result includes: The target traffic sign image is input into a pre-trained traffic keyword recognition model for traffic keyword recognition to obtain the keyword recognition result. The traffic keyword recognition model is used to detect the position of each text line in the target traffic sign image and identify the text content of each detected text line to obtain the keyword recognition result. The keyword recognition result includes a target keyword category to which each text line in the target traffic sign image belongs. The target keyword category is a predefined keyword category among multiple predefined keyword categories that matches the text line of the target traffic sign image.
3. The method according to claim 1, characterized in that The step of extracting a traffic sign image from the collected image includes: Inputting the image into a pre-trained sign detection model to perform traffic sign detection, and obtaining a sign detection result, wherein the sign detection result includes position information of the detected traffic sign in the image; The traffic sign image is cropped from the image based on the sign detection result.
4. The method according to claim 1, characterized in that: The step of performing sign type recognition on the traffic sign image to obtain a sign type recognition result includes: The traffic sign image is input into a pre-trained sign classification model to perform sign type recognition to obtain a sign type recognition result, wherein the sign type recognition result includes a target sign category to which the traffic sign image belongs, and the target sign category is a predefined sign category among multiple predefined sign categories that matches the traffic sign image.
5. The method according to claim 1, characterized in that The step of extracting a target traffic sign image based on the sign type recognition result includes: Querying a predefined mapping relationship between a sign category and whether it contains text content, and determining whether the target sign category of the sign recognition result contains text content; If the target sign category of the sign recognition result contains text content, the corresponding traffic sign image is determined as the target traffic sign image.
6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: The keyword recognition result is sent to a cloud, so that the cloud updates the attribute information of the map element based on the keyword recognition result.
7. The method according to any one of claims 2 to 5, characterized in that: The predefined keyword categories are obtained in the following manner: Performing text recognition on the acquired traffic sign image data to obtain text recognition results; Performing clustering processing on the text recognition results, and generating preliminary keyword categories according to the clustering results; The preliminary keyword category is adjusted to obtain the predefined keyword category.
8. A traffic keyword recognition device, characterized in that: include: A first extraction module, used to extract a traffic sign image from the collected image; A first recognition module is used to perform sign type recognition on the traffic sign image to obtain a sign type recognition result; A second extraction module is used to extract a target traffic sign image based on the sign type recognition result, wherein the target traffic sign image is a traffic sign image containing text content; The second recognition module is used to recognize traffic keywords in the target traffic sign image and obtain keyword recognition results.
9. A traffic keyword recognition device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 7.
10. A computer-readable storage medium / computer program product, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor; and / or, The computer program product comprises a computer program, which implements the method according to any one of claims 1 to 7 when executed by a processor.