Traffic element identification method, device and equipment and computer readable storage medium

Abstract

The application provides a traffic element identification method and device, equipment and a computer readable storage medium; it relates to the field of map and the field of artificial intelligence; the method comprises: detecting the traffic element of the obtained road image to obtain the traffic element to be classified; from the traffic element to be classified and the obtained multiple traffic element templates, the traffic element to be classified is predicted, and multiple space field characteristics corresponding to the multiple traffic element templates; wherein the space field characteristic characterizes the attraction or repulsion degree between the traffic element to be classified and the traffic element template; according to the multiple space field characteristics and the traffic element to be classified, the matching element template corresponding to the traffic element to be classified is found from the multiple traffic element templates; the category corresponding to the matching element template is determined as the target category of the traffic element to be classified. Through the application, the accuracy of traffic element identification can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of maps and the field of artificial intelligence, and in particular to a traffic sign recognition method, device, equipment and computer readable storage medium. BACKGROUND

[0002] Traffic elements (such as traffic signs, monitoring facilities, etc.) can indicate which roads have traffic restrictions, for example, which sections need to be speed limited, which sections have illegal photographing, etc. Traffic element recognition refers to identifying traffic elements in collected road images with the aid of artificial intelligence technology, so as to update map data using the identified traffic elements, or to perform automatic driving, etc.

[0003] In related technologies, a recognition model is trained using labeled data, and the recognition model is used to identify traffic elements in collected road images. However, due to problems such as uneven distribution of traffic element samples of different categories in the labeled data, and low quality of the collected road images, the recognition model may have false positives and false negatives when identifying traffic elements, thereby reducing the accuracy of traffic element recognition. SUMMARY

[0004] The embodiments of the present application provide a traffic element recognition method, device, equipment and computer readable storage medium, which can improve the accuracy of traffic element recognition.

[0005] The technical solutions of the embodiments of the present application are as follows:

[0006] The embodiments of the present application provide a traffic element recognition method, comprising:

[0007] Detecting traffic elements in the acquired road images to obtain traffic elements to be classified; wherein the traffic elements include traffic signs or monitoring facilities in the road;

[0008] From the traffic elements to be classified and a plurality of traffic element templates acquired, predicting a classification feature to be classified and a plurality of spatial potential field features corresponding to the plurality of traffic element templates; wherein the spatial potential field features represent the attraction or repulsion degree between the traffic elements to be classified and the traffic element templates;

[0009] According to the plurality of spatial potential field features and the classification feature to be classified, searching for a matching element template corresponding to the traffic elements to be classified from the plurality of traffic element templates;

[0010] Determining the category corresponding to the matching element template as the target category of the traffic elements to be classified.

[0011] The embodiments of the present application provide a traffic element recognition device, comprising:

[0012] an image detection module, configured to detect a traffic element from a road image acquired by the image acquisition module, to obtain a traffic element to be classified; wherein the traffic element comprises a traffic sign or a monitoring facility in the road;

[0013] a feature prediction module, configured to predict a feature to be classified and a plurality of spatial potential field features corresponding to a plurality of traffic element templates from the traffic element to be classified and the plurality of traffic element templates; wherein the spatial potential field feature represents an attraction or repulsion degree between the traffic element to be classified and the traffic element template;

[0014] a template matching module, configured to find a matching element template corresponding to the traffic element to be classified from the plurality of traffic element templates according to the plurality of spatial potential field features and the feature to be classified;

[0015] a category determination module, configured to determine a category corresponding to the matching element template as a target category of the traffic element to be classified.

[0016] In some embodiments of the present application, the template matching module is further configured to perform spatial correction on the feature to be classified based on the plurality of spatial potential field features to obtain a plurality of corrected features corresponding to the plurality of spatial potential field features; and determine the matching element template corresponding to the traffic element to be classified from the plurality of traffic element templates according to a similarity between the plurality of corrected features and features of the plurality of traffic element templates.

[0017] In some embodiments of the present application, the template matching module is further configured to perform spatial correction on the feature to be classified by using each spatial potential field feature in the plurality of spatial potential field features to obtain a corrected feature corresponding to each spatial potential field feature; and obtain the plurality of corrected features corresponding to the plurality of spatial potential field features when the spatial correction on the feature to be classified is completed by using all the plurality of spatial potential field features.

[0018] In some embodiments of the present application, the template matching module is further configured to select a matching feature dimension corresponding to each feature dimension of the feature to be classified from a plurality of feature dimensions of the corrected feature corresponding to each spatial potential field feature; calculate a correction component for each feature dimension of the feature to be classified by using a feature value and a direction corresponding to the matching feature dimension; and fuse the feature component of each feature dimension of the feature to be classified and the correction component to obtain the corrected feature corresponding to each spatial potential field feature.

[0019] In some embodiments of the present application, the template matching module is further configured to perform similarity calculation on each of the plurality of rectified features and features of the plurality of traffic element templates to obtain a plurality of feature similarities; and select the matching traffic element template from the plurality of traffic element templates, wherein the matching traffic element template corresponds to the maximum feature similarity.

[0020] In some embodiments of the present application, the template matching module is further configured to perform similarity calculation on feature components of a plurality of feature dimensions in each of the rectified features and feature components of a plurality of feature dimensions in each of the traffic element templates to obtain a feature similarity corresponding to each of the traffic element templates; and obtain the plurality of feature similarities when the feature similarities corresponding to the features of the plurality of traffic element templates are calculated.

[0021] In some embodiments of the present application, the template matching module is further configured to determine, for a plurality of feature dimensions in each of the rectified features, a target feature dimension from a plurality of feature dimensions in the feature of each of the traffic element templates; perform distance calculation on feature components of the plurality of feature dimensions in each of the rectified features and feature components corresponding to the target feature dimension to obtain feature distances corresponding to the plurality of feature dimensions of each of the rectified features; and accumulate the feature distances corresponding to the plurality of feature dimensions of each of the rectified features to obtain the feature similarity corresponding to each of the traffic element templates.

[0022] In some embodiments of the present application, the feature prediction module is further configured to perform image feature extraction on the traffic element to be classified to obtain an element image feature; perform image feature extraction on the plurality of traffic element templates respectively to obtain a plurality of template image features corresponding to the plurality of traffic element templates; fuse the element image feature with the plurality of template image features respectively to obtain a plurality of fused image features corresponding to the plurality of traffic element templates; and predict the plurality of spatial potential field features corresponding to the traffic element to be classified and the plurality of traffic element templates from the plurality of fused image features.

[0023] In some embodiments of the present application, the element image feature includes an element image feature vector, and the template image feature includes a template image feature vector.

[0024] The feature prediction module is further configured to splice the element image feature vector with a plurality of template image feature vectors respectively to obtain the plurality of fused image features corresponding to the plurality of traffic element templates; or sum the element image feature vector with the plurality of template image feature vectors respectively to obtain the plurality of fused image features corresponding to the plurality of traffic element templates.

[0025] In some embodiments of the present application, the image detection module is further configured to perform feature extraction on the acquired road image to obtain a to-be-detected feature map; for each feature point of the to-be-detected feature map, a plurality of candidate regions are generated; image features in the plurality of candidate regions are detected for a traffic element respectively to obtain a detection result and a confidence level corresponding to each of the plurality of candidate regions; and image content in a candidate region corresponding to the highest confidence level is determined as the to-be-classified traffic element.

[0026] In some embodiments of the present application, the traffic element identification apparatus further comprises a template generation module.

[0027] The template generation module is configured to acquire a plurality of traffic element images; perform category division on the plurality of traffic element images according to a plurality of preset categories of traffic elements to obtain traffic element images corresponding to each of the plurality of preset categories; and generate the plurality of traffic element templates by using the traffic element images corresponding to each of the plurality of preset categories.

[0028] In some embodiments of the present application, the template generation module is further configured to fuse traffic element images corresponding to each of the plurality of preset categories to obtain a traffic element template corresponding to each of the plurality of preset categories; and obtain the plurality of traffic element templates when traffic element templates corresponding to all of the plurality of preset categories are generated.

[0029] An embodiment of the present application provides a traffic element identification device, comprising:

[0030] A memory configured to store executable traffic element identification instructions;

[0031] A processor configured to execute the executable traffic element identification instructions stored in the memory to implement the traffic element identification method provided in an embodiment of the present application.

[0032] An embodiment of the present application provides a computer readable storage medium storing executable traffic element identification instructions, which are used to cause a processor to execute to implement the traffic element identification method provided in an embodiment of the present application.

[0033] The embodiments of the present application have the following beneficial effects: the traffic element recognition device detects the traffic element to be classified from the acquired road image first, then performs prediction on the traffic element to be classified and the acquired plurality of traffic element templates to obtain the traffic element to be classified feature and a plurality of space field features capable of representing the attraction or repulsion degree between the traffic element to be classified and the plurality of traffic element templates, and then based on the attraction or repulsion represented by the plurality of space field features, the process of determining the matching element template is regulated, the matching element template is searched from the plurality of traffic element templates, and the category of the matching element template is determined as the target category. In this way, by introducing prior knowledge to the space field feature to regulate the process of determining the matching element template, the determined matching element template is more accurate, thereby reducing the influence of the training sample or the image quality on the traffic element recognition process, and finally improving the accuracy of traffic element recognition. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 is an optional architecture schematic diagram of a traffic element recognition system 100 provided by the embodiments of the present application;

[0035] Figure 2 is a structural schematic diagram of a traffic element recognition device provided by the embodiments of the present application;

[0036] Figure 3 is an optional flow schematic diagram of a traffic element recognition method provided by the embodiments of the present application;

[0037] Figure 4 is a schematic diagram of a traffic element template provided by the embodiments of the present application;

[0038] Figure 5 is an optional flow schematic diagram of a traffic element recognition method provided by the embodiments of the present application;

[0039] Figure 6 is a process schematic diagram of predicting the traffic element to be classified feature and the space field feature provided by the embodiments of the present application;

[0040] Figure 7 is a schematic diagram of fusing the element image feature vector and the template image feature vector provided by the embodiments of the present application;

[0041] Figure 8 is a schematic diagram of a preset number of candidate regions provided by the embodiments of the present application;

[0042] Figure 9 is a process schematic diagram of detecting the traffic element of the road image provided by the embodiments of the present application;

[0043] Figure 10 is a schematic diagram of the detected traffic element to be classified provided by the embodiments of the present application;

[0044] Figure 11 is a schematic diagram of a traffic sign recognition process in a road provided by an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings, and the described embodiments should not be regarded as limiting the present application. All other embodiments obtained by those of ordinary skill in the art without making any creative labor fall within the scope of protection of the present application.

[0046] In the following description, "some embodiments" are related to a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0047] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0048] The related data collection and processing in the embodiments of the present application should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and within the scope of authorization of laws and regulations and the personal information subject, carry out subsequent data use and processing.

[0049] Before the embodiments of the present application are further described in detail, the terms and terms involved in the embodiments of the present application are explained, and the terms and terms involved in the embodiments of the present application are applicable to the following explanations.

[0050] 1) Artificial intelligence (Artificial Intelligence, AI) is to use digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which tries to understand the essence of intelligence and generate a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0051] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software level technology. Artificial intelligence basic technology generally includes, such as sensors, special artificial chip, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer actual technology, speech processing technology, natural language processing technology and machine learning / deep learning, automatic driving, intelligent transportation and other several major directions.

[0052] 2) Computer vision (CV) is a science that studies how to make machines "see". Further, it refers to using cameras and computers to replace human eyes to identify, track and measure targets and further process graphics, so that the computer processing becomes more suitable for human eye observation or image transmission to instrument detection. As a scientific discipline, computer vision researches related theories and technologies, trying to establish artificial intelligence systems that can obtain information from images or multidimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, autonomous driving, intelligent transportation and other technologies, as well as common face recognition, fingerprint recognition and other biometric identification technologies.

[0053] 3) Machine learning (ML) is a multi-disciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It is a subject that studies how computers simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge structure to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental approach to making computers intelligent, and its applications are widespread in various fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning and other technologies.

[0054] 4) Convolutional neural network (CNN) is a class of feedforward neural network containing convolution calculation and having a deep structure, which is one of the representative algorithms of deep learning. Convolutional neural network has the ability of feature learning and can perform shift-invariant classification on input information according to its hierarchical structure.

[0055] 5) Classification network refers to a neural network that identifies the objects contained in an image. The input of the classification network is image data, and the output is the category of the object contained in the image.

[0056] 6) Traffic element, a useful physical point information in an electronic map, including traffic signs or monitoring facilities in a road, for example, speed limit signs, electronic eyes, traffic restriction signs, etc. in a road.

[0057] 7) Feature similarity, a measure for evaluating the similarity between different spatial features. Feature similarity can be measured by distance, angle, etc.

[0058] 8) Spatial potential field, used to describe the characteristics of mutual attraction or repulsion between two features or objects. For example, when two features attract each other, there is an attractive force between the two features; when two features repel each other, there is a repulsive force between the two features.

[0059] With the research and progress of artificial intelligence technology, artificial intelligence technology is researched and applied in many fields, such as smart home, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned vehicles, autonomous vehicles, drones, robots, smart medical care, smart customer service, Internet of Vehicles, autonomous driving, and intelligent transportation. With the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0060] Traffic elements (such as traffic signs, monitoring settings, etc.) can indicate which roads have traffic restrictions, for example, which sections need to be speed limited, which sections have illegal photographing, etc. Traffic element recognition refers to identifying traffic elements in collected road images with the help of artificial intelligence technology, so as to update map data using the identified traffic elements, or for autonomous driving, etc.

[0061] In related technologies, a recognition model is trained using labeled data, and the recognition model is used to identify traffic elements in collected road images. However, the number of samples of different categories of traffic elements in the labeled data may be different, for example, speed limit signs are more common in urban areas, mountainous roads, etc. and the number of samples is large, while rockfall signs only appear in a small number of sections of mountainous roads and the number of samples is small. The different number of samples of different categories of traffic elements will lead to unbalanced labeled data samples. The recognition model trained using unbalanced labeled data samples is likely to have false positives and false negatives when identifying traffic elements in road images.

[0062] Moreover, the road images are likely to be collected during vehicle driving rather than in a stable environment, so the quality of the collected road images is likely to be low. When using the recognition model to identify road images with low quality, false positives and false negatives are also likely to occur.

[0063] From the above, in the related art, due to the problems such as the unbalanced samples of different categories of traffic elements in the labeled data, the low quality of the collected road images, and the like, when the recognition model is used to recognize the traffic elements, misrecognition and false recognition may occur, so that the accuracy of traffic element recognition is low.

[0064] The embodiment of the present application provides a traffic element recognition method, device, equipment and computer readable storage medium, which can improve the recognition efficiency of traffic elements. The following describes an exemplary application of the traffic element recognition device provided by the embodiment of the present application. The device provided by the embodiment of the present application can be implemented as a notebook computer, a tablet computer, a desktop computer, a vehicle-mounted terminal, a mobile device (for example, a mobile phone, a portable music player, a personal digital assistant, a dedicated message device, a portable game device) and various types of terminals. It can also be implemented as a server, and can also be implemented as a device cluster composed of a terminal and a server. The following describes an exemplary application when the traffic element recognition device is implemented as or a server.

[0065] Referring to Figure 1 , Figure 1 is an optional architecture schematic diagram of the traffic element recognition system 100 provided by the embodiment of the present application, for realizing a traffic element recognition application. The terminal (exemplarily shows the terminal 400-1 and the terminal 400-2) connects the server 200 through the network 300, and the network 300 can be a wide area network or a local area network, or a combination of the two.

[0066] The server 200 is configured to detect traffic elements in the acquired road image to obtain a traffic element to be classified, wherein the traffic element includes a traffic sign or a monitoring facility in the road; predict a classification feature to be classified corresponding to the traffic element to be classified and a plurality of spatial potential field features corresponding to a plurality of traffic element templates from the traffic element to be classified and the plurality of traffic element templates; wherein the spatial potential field feature represents the attraction or repulsion degree between the traffic element to be classified and the traffic element template; based on the plurality of spatial potential field features and the classification feature to be classified, find out a matching element template corresponding to the traffic element to be classified from the plurality of traffic element templates; and determine the class corresponding to the matching element template as a target class of the traffic element to be classified. In this way, the server 200 realizes traffic element recognition.

[0067] In some embodiments, the server 200 also updates the map data by using the traffic element to be classified and the target class, and sends the updated map data to the terminal 400-1 and the terminal 400-2 through the network 300, so as to realize the update of the map data of the terminal 400-1 and the terminal 400-2.

[0068] In some embodiments, the server 200 can be a standalone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and artificial intelligence platforms. The terminal 400 can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a vehicle-mounted terminal, and the like, but is not limited thereto. The terminal and the server can be connected directly or indirectly through wired or wireless communication, which is not limited in the embodiments of the present application.

[0069] Referring to Figure 2 , Figure 2 is a structural schematic diagram of a traffic element recognition device provided by an embodiment of the present application, Figure 2 The traffic element recognition device 500 shown in the figure includes at least one processor 510, a memory 550, at least one network interface 520, and a user interface 530. The various components in the traffic element recognition device 500 are coupled together through a bus system 540. It can be understood that the bus system 540 is used to realize the connection communication between the components. In addition to including a data bus, the bus system 540 also includes a power bus, a control bus, and a status signal bus. However, for the purpose of clear illustration, all kinds of buses are marked as the bus system 540 in Figure 2 .

[0070] The processor 510 can be an integrated circuit chip with signal processing capability, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0071] The user interface 530 includes one or more output devices 531 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 530 also includes one or more input devices 532 that facilitate user input, such as a keyboard, a mouse, a microphone, a touch screen display, a camera, other input buttons and controls.

[0072] The memory 550 can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, etc. The memory 550 optionally includes one or more storage devices physically located in proximity to the processor 510.

[0073] The memory 550 includes volatile memory or nonvolatile memory, and can include both volatile and nonvolatile memory. The nonvolatile memory can be read only memory (ROM), and the volatile memory can be random access memory (RAM). The memory 550 described in the embodiments of the present application is intended to include any suitable type of memory.

[0074] In some embodiments, the memory 550 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, which are exemplarily illustrated below.

[0075] The operating system 551 includes system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks.

[0076] The network communication module 552 is used to reach other computing devices via one or more (wired or wireless) network interfaces 520, exemplary network interfaces 520 including Bluetooth, wireless compatibility certification (Wi-Fi), and universal serial bus (USB), etc.

[0077] The presentation module 553 is used to enable the presentation of information via one or more output devices 531 associated with the user interface 530 (e.g., a display screen, a speaker, etc.), such as a user interface for operating peripheral devices and displaying content and information.

[0078] The input processing module 554 is used to detect and interpret one or more user inputs or interactions from one or more input devices 532.

[0079] In some embodiments, the apparatus provided by the embodiments of the present application can be implemented in software, Figure 2 An element recognition apparatus 555 stored in the memory 550 is shown, which can be software in the form of programs and plug-ins, including the following software modules: an image detection module 5551, a feature prediction module 5552, a template matching module 5553, a category determination module 5554, and a template generation module 5555. These modules are logical, and thus can be combined or further split according to the implemented functions.

[0080] The functions of the various modules will be described below.

[0081] In some embodiments, the traffic element identification apparatus provided by the embodiments of the present application can be implemented in a hardware manner. For example, the traffic element identification apparatus provided by the embodiments of the present application can be a hardware decoding processor programmed to execute the traffic element identification method provided by the embodiments of the present application. For example, the hardware decoding processor can be one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic elements.

[0082] For example, the embodiments of the present application provide a traffic element identification device, which comprises:

[0083] a memory configured to store executable traffic element identification instructions;

[0084] a processor configured to execute the executable traffic element identification instructions stored in the memory to implement the traffic element identification method provided by the embodiments of the present application.

[0085] In the following, the traffic element identification method provided by the embodiments of the present application will be described in conjunction with the exemplary application and implementation of the traffic element identification device provided by the embodiments of the present application.

[0086] For example, the embodiments of the present application provide a traffic element identification method, which comprises: Figure 3 Figure 3 is an optional flowchart of the traffic element identification method provided by the embodiments of the present application, which will be described in conjunction with the steps shown in Figure 3 .

[0087] S101, detecting traffic elements in the acquired road image to obtain traffic elements to be classified.

[0088] ​The embodiments of the present application are implemented in the scenario of identifying traffic elements in a road image collected, for example, identifying a deceleration sign in a road to automatically control a vehicle to decelerate, or identifying an electronic eye in a road to update map data. When starting a traffic element identification process, a traffic element identification device acquires a road image, and then detects the road image to cut out traffic elements detected from the road image as traffic elements to be classified. For example, the traffic element identification device detects a triangular traffic sign or a circular sign in the road image as the traffic elements to be classified. The traffic elements include traffic signs or monitoring facilities in a road.

[0089] In the embodiments of the present application, when acquiring a road image, the traffic element identification device can call an image acquisition device to acquire a road image by image acquisition of a road being traveled, or acquire a road image uploaded by another user from a network or a database.

[0090] In some embodiments, the image acquisition device can be a camera, so that the acquired road image is a photo of the road. In other embodiments, the image acquisition device can be a video camera, so that the acquired road image is a video of the road.

[0091] When the image acquisition device is a camera, the traffic element identification device can call the camera to take a photo of the road being traveled at a certain time interval, for example, every 10 seconds or every 1 minute, to obtain a photo of the road; or the traffic element identification device can acquire a current geographic position through a positioning system, and call the camera to take a photo of the road being traveled when the geographic position is within a preset range of a road waiting for map data update, to obtain a photo of the road.

[0092] When the image acquisition device is a video camera, the traffic element identification device can acquire a road image continuously after starting a traffic element identification process, to obtain a video of the road; or the traffic element identification device can call the video camera to record a video of the road when receiving an instruction of starting recording triggered by a staff.

[0093] In some embodiments, the traffic element identification device uses a trained traffic element detection model to first extract features of a road image to obtain a feature map of the road image, then selects a candidate frame for the feature map, and classifies image content in the candidate frame to detect traffic elements to be classified.

[0094] In some other embodiments, since the shape of the traffic element is regular, the traffic element recognition device can match the traffic element from the road image through a contour template of the traffic element, and the matched traffic element is the traffic element to be classified.

[0095] S102, from the traffic element to be classified and the obtained plurality of traffic element templates, a plurality of spatial potential field features corresponding to the plurality of traffic element templates are predicted.

[0096] After the traffic element to be classified is obtained by the traffic element recognition device, a plurality of traffic element templates prepared by the traffic element recognition device are obtained from a storage space or a database of the traffic element recognition device, and then the traffic element to be classified is combined with the plurality of traffic element templates respectively to predict features, so as to obtain a plurality of spatial potential field features of the plurality of traffic element template objects and the traffic element to be classified.

[0097] It can be understood that the traffic element template can include various traffic signs, and can also include electronic eyes, traffic lights and other traffic facilities, which are not limited in the present application. Further, in the obtained plurality of traffic element templates, one category can have one traffic element template, or one type can have a plurality of traffic element templates, which are not limited in the present application.

[0098] Exemplarily, Figure 4 is a schematic diagram of a traffic element template provided by an embodiment of the present application, wherein Figure 4 It can be seen that in the plurality of traffic element templates, various warning signs 4-1, various prohibition signs 4-2 and various information signs 4-3 can be included.

[0099] It should be noted that the spatial potential field feature represents the attraction or repulsion degree between the traffic element to be classified and the traffic element template. The attraction or repulsion can be understood as the similarity between the traffic element to be classified and the traffic element template. That is, if the traffic element to be classified and the traffic element template are similar, the traffic element to be classified and the traffic element template attract each other, and if the traffic element to be classified and the traffic element template are different, the traffic element to be classified and the traffic element template repel each other.

[0100] In the embodiment of the present application, the spatial potential field feature can be a spatial potential field feature vector or a spatial potential field feature parameter. The spatial potential field feature has a direction, which can be determined by the direction of each dimension component of the spatial potential field vector, or can be determined by the positive or negative sign of the spatial potential field feature parameter, which is not limited in the present application.

[0101] In some embodiments, the traffic element recognition device can perform feature extraction on the traffic element to be classified and on the plurality of traffic element templates, fuse the features of the plurality of traffic element templates with the features of the traffic element to be classified one by one, obtain the fused features corresponding to the plurality of traffic element templates respectively, and then perform prediction on the fused features to obtain the plurality of spatial potential field features and the feature of the traffic element to be classified.

[0102] In other embodiments, the traffic element recognition device can further group the traffic element to be classified with the plurality of traffic element templates respectively to obtain a plurality of element pairs corresponding to the plurality of traffic element templates, and then input the traffic element template and the traffic element to be classified in each element pair into the feature prediction model to predict the plurality of spatial potential field features and the feature of the traffic element to be classified.

[0103] S103, find the matching traffic element template corresponding to the traffic element to be classified from the plurality of traffic element templates according to the plurality of spatial potential field features and the feature of the traffic element to be classified.

[0104] After obtaining the plurality of spatial potential field features corresponding to the plurality of traffic element templates, the traffic element recognition device can combine the plurality of spatial potential field features and the feature of the traffic element to be classified to find the traffic element template closest to the traffic element to be classified from the plurality of traffic element templates as the matching traffic element template.

[0105] Since the traffic element templates are prepared in advance, the traffic element recognition device predicts the plurality of spatial potential field features from the traffic element to be classified and the plurality of traffic element templates, and then finds the matching traffic element template based on the spatial potential field features and the feature of the traffic element to be classified. The process is essentially adding prior knowledge to the process of determining the matching traffic element template to preliminarily determine which traffic element template the traffic element to be classified is attracted to or repelled by, so as to regulate the process of determining the matching traffic element template through the attraction or repulsion, so that a more accurate matching traffic element template can be accurately determined for the traffic element that cannot be classified due to insufficient training samples or poor image quality.

[0106] In some embodiments, the traffic element recognition device can use the plurality of spatial potential field features to respectively perform spatial correction on the feature of the traffic element to be classified, and then use the similarity between the corrected features and the features of the plurality of traffic element templates to select the matching traffic element feature from the plurality of traffic element templates.

[0107] In other embodiments, the traffic element recognition device can further fuse the feature of the traffic element to be classified with each spatial potential field feature, and then cluster the fused features corresponding to each spatial potential field feature with the features of the plurality of traffic element templates to select the matching traffic element feature.

[0108] S104, determine the category corresponding to the matched element template as the target category of the traffic element to be classified.

[0109] Finally, the traffic element recognition device determines the category of the matched element template, and then takes the category as the target category, thereby completing the classification process of the traffic element to be classified. Since the accuracy of the matched element template is improved in the above steps, the accuracy of the target category determined in this step is also improved accordingly.

[0110] In the embodiments of the present application, the traffic element recognition device first detects the traffic element to be classified from the obtained road image, then predicts the traffic element to be classified and the obtained multiple traffic element templates to obtain the traffic element to be classified and multiple spatial potential field features capable of representing the attraction or repulsion degree between the traffic element to be classified and the multiple traffic element templates, and then based on the attraction or repulsion represented by the multiple spatial potential field features, the process of determining the matched element template is regulated, the matched element template is searched from the multiple traffic element templates, and the category of the matched element template is determined as the target category. In this way, by introducing prior knowledge to the spatial potential field features, the process of determining the matched element template is regulated, so that the determined matched element template is more accurate, thereby reducing the influence of training samples or image quality on the traffic element recognition process, and finally improving the accuracy of traffic element recognition.

[0111] Referring to Figure 5 , Figure 5 is an optional flowchart of the traffic element recognition method provided by the embodiments of the present application. In some embodiments of the present application, the matched element template corresponding to the traffic element to be classified is searched from the multiple traffic element templates according to the multiple spatial potential field features and the traffic element to be classified, that is, the specific implementation process of S103 can include S1031-S1032, as follows:

[0112] S1031, based on the multiple spatial potential field features, spatially correct the traffic element to be classified to obtain multiple correction features corresponding to the multiple spatial potential field features.

[0113] When the traffic element recognition device determines the matched element template, the multiple spatial potential field features are first used to respectively spatially correct the traffic element to be classified, so as to adjust the spatial relationship between the traffic element to be classified and the features of each traffic element template, so that the spatial relationship between the traffic element to be classified and the features of the multiple traffic element templates is more clear. The multiple correction features corresponding to the multiple spatial potential field features are one-to-one corresponding to the multiple traffic element templates.

[0114] It should be noted that the influence of the training samples and the image quality on the traffic element recognition process is essentially that the position of the feature extracted from the image in the feature space is inaccurate. The embodiments of the present application introduce prior knowledge by spatially correcting the to-be-classified feature with the plurality of spatial potential field features to correct the position of the to-be-classified feature extracted from the image in the feature space.

[0115] In some embodiments, the traffic element recognition device can weight the to-be-classified feature and the plurality of spatial potential field features respectively to correct the to-be-classified feature in the feature space to obtain a plurality of corrected features.

[0116] In some other embodiments, the traffic element recognition device can also multiply the to-be-classified feature and the plurality of spatial potential field features respectively to correct the to-be-classified feature in the feature space to obtain a plurality of corrected features.

[0117] S1032, determining the matching element template corresponding to the to-be-classified traffic element from the plurality of traffic element templates according to the similarity between the plurality of corrected features and the features of the plurality of traffic element templates.

[0118] After obtaining the plurality of corrected features, the traffic element recognition device calculates the similarity between each corrected feature and the features of the plurality of traffic element templates, so that the similarity between the plurality of corrected features and the plurality of traffic element templates is calculated, and then the highest similarity is selected from all the similarities, and the traffic element template corresponding to the highest similarity is selected as the matching element template.

[0119] In the embodiments of the present application, the traffic element recognition device first corrects the to-be-classified feature in space by using the plurality of spatial potential field features, so as to obtain the plurality of corrected features with a more clear spatial relationship with the traffic element templates, and then selects a more accurate matching element template for the to-be-classified traffic element based on the similarity between the plurality of corrected features and the features of the plurality of traffic element templates.

[0120] In some embodiments of the present application, the to-be-classified feature is corrected in space based on the plurality of spatial potential field features to obtain the plurality of corrected features corresponding to the plurality of spatial potential field features, that is, the specific implementation process of S1031 can include S1031a-S1031b, as follows:

[0121] S1031a, correcting the to-be-classified feature in space by using each spatial potential field feature in the plurality of spatial potential field features to obtain a corrected feature corresponding to each spatial potential field feature.

[0122] The traffic element recognition device corrects the to-be-classified feature in space by each spatial potential field feature, so that the spatial relationship between the to-be-classified feature and the feature of each traffic element template is adjusted, for example, whether the to-be-classified feature is attracted to the feature of each traffic element template (attractive force) or the to-be-classified feature is repelled from the feature of the current traffic element template (repulsive force), so as to obtain the corrected feature corresponding to each spatial potential field feature.

[0123] S1031b, when the spatial correction of the to-be-classified feature is completed by using multiple spatial potential field features, multiple corrected features corresponding to the multiple spatial potential field features are obtained

[0124] The traffic element recognition device determines the corresponding corrected feature for each traffic element template, so that the traffic element recognition device obtains multiple corrected features after the spatial correction of the to-be-classified feature is completed by using multiple spatial potential field features, and the number of corrected features is equal to the number of traffic element templates.

[0125] For example, when there are n traffic element templates, there are n spatial potential field features, so that the traffic element recognition device obtains n corrected features.

[0126] In the embodiment of the application, the traffic element recognition device first corrects the to-be-classified feature in space by using each of the multiple spatial potential field features, and repeatedly in this way, after the spatial correction of the to-be-classified feature is completed by using all the spatial potential field features, multiple corrected features corresponding to the multiple traffic element templates are obtained, so as to calculate the similarity subsequently and determine the matching element template.

[0127] In some embodiments of the application, the spatial correction of the to-be-classified feature is performed by using each spatial potential field feature in the multiple spatial potential field features, and the corrected feature corresponding to each spatial potential field feature is obtained, that is, the specific implementation process of S1031a can include: S201-S203, as follows:

[0128] S201, from multiple feature dimensions of the corrected feature corresponding to each spatial potential field feature, a matching feature dimension corresponding to each feature dimension of the to-be-classified feature is selected.

[0129] The feature space is a multi-dimensional space, the to-be-classified feature is composed of feature components of multiple feature dimensions, and each spatial potential field feature is also composed of feature components of multiple feature dimensions. The traffic element recognition device first corresponds the multiple feature dimensions of each spatial potential field feature to the multiple feature dimensions of the to-be-classified feature, so that for each feature dimension of the to-be-classified feature, a corresponding matching feature dimension can be selected from the multiple feature dimensions of each spatial potential field feature.

[0130] It should be noted that the number of feature dimensions of each spatial potential field feature can be different from the number of feature dimensions of the feature to be classified. When the number of feature dimensions of each spatial potential field feature is the same as the number of feature dimensions of the feature to be classified, the traffic element recognition device first determines that each feature dimension of the feature to be classified is the dimension in the feature space, and then finds the corresponding dimension from each spatial potential field feature to determine the matching feature dimension corresponding to each feature dimension of the feature to be classified; when the number of feature dimensions of each spatial potential field feature is different from the number of feature dimensions of the feature to be classified, the traffic element recognition device selects a plurality of corresponding feature dimensions for each feature dimension of the feature to be classified, for example, groups the plurality of feature dimensions of each spatial potential field feature to obtain a feature dimension group corresponding to the number of feature dimensions of the feature to be classified, and then selects the feature dimension group corresponding to each feature dimension of the feature to be classified in order, and takes the feature dimension in the feature dimension group as the matching feature dimension.

[0131] S202, using the feature value and direction corresponding to the matching feature dimension, calculating a correction component for each feature dimension of the feature to be classified.

[0132] The traffic element recognition device can directly take the product of the feature value and direction of the matching feature dimension as the correction component corresponding to each feature dimension of the feature to be classified, or take the opposite number or multiple of the product of the feature value and direction of the matching feature dimension as the correction component corresponding to each feature dimension of the feature to be classified, which is not limited in the present application.

[0133] For example, when the feature value of the matching feature dimension is , and the direction is (representing the sign), then the correction component of each feature dimension of the feature to be classified can be .

[0134] In some embodiments, when the matching feature dimension of each feature dimension of the feature to be classified includes a plurality of feature dimensions, the traffic element recognition device can first calculate the dimension component of each feature dimension using the value and direction of each dimension in the plurality of feature dimensions, and then fuse the dimension components of the feature dimensions to obtain the correction component of each feature dimension of the feature to be classified.

[0135] S203, fusing the feature component and the correction component of each feature dimension of the feature to be classified to obtain a correction feature corresponding to each spatial potential field feature.

[0136] The traffic element recognition device directly superimposes the feature component of each feature dimension of the feature to be classified and the corresponding correction component, or superimposes the feature component of each feature dimension and the product of the corresponding correction component and the adjustment coefficient, and the superimposed feature is the corrected feature.

[0137] For example, the embodiment of the present application provides a formula for calculating the corrected feature, see formula (1):

[0138] (1)

[0139] wherein, is the feature component of each feature dimension of the feature to be classified, is the correction component of each feature dimension of the feature to be classified, n is the feature dimension of the feature to be classified, m is the feature dimension of the current spatial potential field feature, and Z is the dimension of the feature space, is the corrected feature.

[0140] In the embodiment of the present application, the traffic element recognition device will first determine the corresponding matching feature dimension for each feature dimension of the feature to be classified, then calculate the correction component by using the feature value and direction corresponding to the matching feature dimension, and finally superimpose the correction component on the feature component of each feature dimension of the feature to be classified, so as to realize the spatial correction of the feature to be classified.

[0141] In some embodiments of the present application, the matching element template corresponding to the traffic element to be classified is determined from the plurality of traffic element templates according to the similarity between the plurality of corrected features and the features of the plurality of traffic element templates, that is, the specific implementation process of S1032 can include: S1032a-S1032b, as follows:

[0142] S1032a, similarity calculation is performed between each of the plurality of corrected features and the features of the plurality of traffic element templates, to obtain a plurality of feature similarities.

[0143] The traffic element recognition device respectively performs similarity calculation between each corrected feature and the features of the plurality of traffic element templates, and the obtained feature similarity corresponds to each traffic element template. When the traffic element recognition device has performed similarity calculation between all the corrected features in the plurality of corrected features and the features of the plurality of traffic element templates, a plurality of feature similarities are obtained. At this time, the number of feature similarities should be the square of the number of traffic element templates.

[0144] That is, when the traffic element template has n, the corrected feature also has n, the traffic element recognition device can calculate n feature similarities for each corrected feature, and the traffic element recognition device will obtain feature similarities in total.

[0145] S1032b, select a matching traffic element template from the plurality of traffic element templates.

[0146] The traffic element recognition device compares the plurality of feature similarities in size, and determines the maximum feature similarity from the plurality of feature similarities. Since each feature similarity has a corresponding traffic element template, the traffic element recognition device determines the traffic element template corresponding to the maximum feature similarity by using the correspondence between the feature similarity and the traffic element template. The determined traffic element template is the matching traffic element template. That is, the matching traffic element template corresponds to the maximum feature similarity.

[0147] In the embodiments of the present application, the traffic element recognition device performs similarity calculation on the plurality of corrected features and the features of the plurality of traffic element templates respectively, thereby obtaining a plurality of feature similarities, and selects the matching element template with the maximum feature similarity. In this way, the traffic element recognition device can determine a more accurate matching element template for the traffic element to be classified, so as to subsequently determine a more accurate target category.

[0148] In some embodiments of the present application, the similarity calculation is performed on each of the plurality of corrected features and the features of the plurality of traffic element templates respectively to obtain a plurality of feature similarities, that is, the specific implementation process of S1032a can include: S301-S302, as follows:

[0149] S301, by performing similarity calculation on the feature components of the plurality of feature dimensions in each corrected feature and the feature components of the plurality of feature dimensions in each traffic element template, the feature similarity corresponding to each traffic element template is obtained.

[0150] S302, when the corresponding feature similarities of the features of the plurality of traffic element templates are calculated, a plurality of feature similarities are obtained.

[0151] The traffic element recognition device first selects one traffic element template from the plurality of traffic element templates as a current traffic element template. Then, when the feature components of each feature dimension of the current traffic element template and the feature components of each feature dimension in each corrected feature are all calculated for feature similarity, the feature similarity corresponding to the current traffic element template is obtained. When the traffic element recognition device completes the calculation of the feature similarity for all traffic element templates, a plurality of feature similarities corresponding to the plurality of traffic element templates are obtained.

[0152] In the embodiments of the present application, when the traffic element recognition device performs similarity calculation on each rectified feature and the plurality of traffic element templates, the traffic element recognition device performs similarity calculation on the feature components of each feature dimension of each rectified feature and the feature components of each feature dimension of each traffic element template in the plurality of traffic element templates to obtain a plurality of feature similarities corresponding to the plurality of traffic element templates, so as to subsequently select a matching traffic element template according to the plurality of feature similarities.

[0153] In some embodiments of the present application, the feature similarity corresponding to each traffic element template is obtained by performing similarity calculation on the feature components of the plurality of feature dimensions in each rectified feature and the feature components of the plurality of feature dimensions in the feature of each traffic element template, and the specific implementation process of S301 can include S3011-S3013 as follows.

[0154] S3011, for the plurality of feature dimensions in each rectified feature, determine the target feature dimension from the plurality of feature dimensions of the feature of each traffic element template.

[0155] Each traffic element template is composed of feature components of a plurality of feature dimensions, and the traffic element recognition device selects the target feature dimension corresponding to each feature dimension of each rectified feature from all feature dimensions of each traffic element template.

[0156] It can be understood that the specific implementation process of determining the target feature dimension for each feature dimension of each rectified feature by the traffic element recognition device is similar to the process of determining the matching feature dimension corresponding to each feature dimension of the to-be-classified feature from the plurality of feature dimensions of each spatial potential field feature in S201, and details are not repeated here.

[0157] S3012, perform distance calculation on the feature components of the plurality of feature dimensions in each rectified feature and the feature components corresponding to the target feature dimension to obtain the feature distance corresponding to the plurality of feature dimensions of each rectified feature.

[0158] After the traffic element recognition device determines the corresponding target feature dimension for each feature dimension of each rectified feature, the traffic element recognition device performs difference on the feature components of each feature dimension of each rectified feature and the feature components of the target feature dimension, and squares the obtained difference value as the feature distance, or takes the square root of the squared result to obtain the feature distance.

[0159] For example, when each feature dimension in the current rectified feature is represented as and the corresponding target feature dimension is represented as (wherein, If the feature distance of each feature dimension of the current traffic element template is represented as (i.e., the feature distance of each feature dimension of the current traffic element template is represented as a feature distance of a feature dimension of the current traffic element template), then the feature distance of each feature dimension in the current correction feature is .

[0160] S3013, accumulate the feature distances corresponding to the multiple feature dimensions of each correction feature to obtain a feature similarity corresponding to each traffic element template.

[0161] The traffic element recognition device accumulates the feature distances corresponding to the multiple feature dimensions of each correction feature together, and the accumulated result is the feature similarity corresponding to the feature of each traffic element template, which represents the similarity between each correction feature and the feature of each traffic element template.

[0162] For example, if the feature distance of each feature dimension of each correction feature is , then the feature similarity corresponding to the feature of each traffic element template is .

[0163] In the embodiments of the present application, when calculating the feature similarity corresponding to the feature of the current traffic element template, the traffic element recognition device first matches the multiple feature dimensions of the feature of each traffic element template with each feature dimension in each correction feature, then calculates the feature distance for each feature dimension in each correction feature and the target feature dimension corresponding thereto, and then accumulates the feature distances to obtain the feature similarity between each correction feature and each traffic element template, so as to subsequently obtain the feature similarities corresponding to the multiple traffic element templates.

[0164] In some embodiments of the present application, the to-be-classified feature and the multiple spatial potential field features corresponding to the multiple traffic element templates are predicted from the to-be-classified traffic element and the obtained multiple traffic element templates, i.e., the specific implementation process of S102 can include S1021-S1024, as follows:

[0165] S1021, perform image feature extraction on the to-be-classified traffic element to obtain an element image feature.

[0166] The traffic element recognition device can obtain a trained image feature extraction model, and then input the to-be-classified traffic element into the image feature extraction model to extract the element image feature from the to-be-classified traffic element through the image feature extraction model.

[0167] It can be understood that the image feature extraction model can be a convolutional neural network (CNN) model, an artificial neural network (ANN) model, or the like, which is not limited in the present application.

[0168] In some embodiments, the traffic element recognition device can further extract the corner point features of the traffic element to be classified through a corner point feature extraction algorithm, and take the corner point features as the element image features.

[0169] S1022, image feature extraction is performed on the plurality of traffic element templates respectively to obtain a plurality of template image features corresponding to the plurality of traffic element templates.

[0170] The traffic element recognition device can input the plurality of traffic element templates into the image feature extraction model respectively to obtain the template image features of the plurality of traffic element templates through the image feature extraction model.

[0171] It can be understood that the execution order of S1021 and S1022 does not affect the classified image features and the plurality of template image features. In other embodiments, the traffic element recognition device can execute S1022 first and then execute S1021, or execute S1021 and S1022 simultaneously.

[0172] S1023, the element image features are fused with the plurality of template image features respectively to obtain a plurality of fused image features corresponding to the plurality of traffic element templates.

[0173] The traffic element recognition device extracts one of the plurality of template image features as a current template image feature, and then fuses the element image features with the current template image feature to obtain a fused image feature corresponding to the current template image feature. Since there is a corresponding relationship between the plurality of template image features and the plurality of traffic element templates, the fused image feature corresponding to the current template image feature is essentially the fused image feature corresponding to the current traffic element template. When the traffic element recognition device fuses the element image features with the plurality of template image features respectively, the plurality of fused image features is obtained.

[0174] S1024, from the plurality of fused image features, a plurality of spatial potential field features corresponding to the classified features and the plurality of traffic element templates are predicted.

[0175] The traffic element recognition device inputs each fused image feature into the feature prediction model, so as to predict the spatial potential field features corresponding to each traffic element template and the classified features through the feature prediction model. Although the traffic element recognition device predicts the classified features from each fused image feature, these classified features are the same and can be regarded as one classified feature.

[0176] It can be understood that the feature prediction model can be a CNN model or other models such as an ANN model, which is not limited in the present application.

[0177] For example, Figure 6 is a process schematic diagram for predicting the to-be-classified feature and the spatial potential field feature. The traffic element recognition device inputs the to-be-classified traffic element 6-1 into the feature prediction model 6-2 to obtain the element image feature 6-3, and inputs the traffic element template 6-4 selected from the plurality of traffic element templates into the feature prediction model 6-5 to obtain the corresponding template image feature 6-6 (the structures of the feature prediction model 6-2 and the feature prediction model 6-5 can be the same or different). Then, the traffic element recognition device fuses the to-be-classified image feature 6-3 and the template image feature 6-6 into one feature 6-7, which is the fused image feature, and then inputs the feature 6-7 into the feature prediction model 6-8 to obtain the to-be-classified feature 6-9 and the spatial potential field feature 6-10 corresponding to the traffic element target 6-4. After the traffic element recognition device completes the process shown in the figure for each traffic element target Figure 6 , the to-be-classified feature and the plurality of spatial potential field features are obtained.

[0178] In the embodiments of the present application, the traffic element recognition device extracts the element image feature and the plurality of template image features from the to-be-classified traffic element and the plurality of traffic element templates, respectively, fuses the element image feature and the plurality of template image features, and performs feature prediction on the plurality of fused image features, so as to obtain the to-be-classified feature and the plurality of spatial potential field features, so as to facilitate subsequent use of the plurality of spatial potential field features and the to-be-classified image feature to find a more accurate matching element template and improve the accuracy of traffic element recognition.

[0179] In some embodiments of the present application, the element image feature includes an element image feature vector, and the template image feature includes a template image feature vector. At this time, the element image feature is fused with the plurality of template image features respectively to obtain the plurality of fused image features corresponding to the plurality of traffic element templates, that is, the specific implementation process of S1023 can include S1023a or S1023b, as follows:

[0180] S1023a, the element image feature vector is spliced with the plurality of template image feature vectors respectively to obtain the plurality of fused image features corresponding to the plurality of traffic element templates.

[0181] The traffic element recognition device splices the element image feature vector with any one of the plurality of template image feature vectors to obtain the fused image feature of the traffic element template corresponding to the template image feature vector. After splicing each template image feature vector with the element image feature vector respectively, the plurality of fused image features are obtained.

[0182] For example, Figure 7is a schematic diagram of fusing the element image feature vector and the template image feature vector provided in the embodiments of the present application. Referring to Figure 7 The traffic element recognition device splices the element image feature vector 7-1 and the template image feature vector 7-2 selected from the plurality of template image feature vectors, to obtain the fusion image feature 7-3 corresponding to the template image feature vector 7-2.

[0183] S1023b, summing the element image feature vector and the plurality of template image feature vectors respectively, to obtain a plurality of fusion image features corresponding to a plurality of traffic element templates.

[0184] The traffic element recognition device selects a template image feature vector from the plurality of spatial potential field feature vectors, and then adds the template image feature vector and the element image feature vector, to obtain the fusion image feature of the traffic element template corresponding to the template image feature vector. After the traffic element recognition device completes the weighting of the element image feature vector for a plurality of traffic element targets, a plurality of fusion image features corresponding to the plurality of traffic element targets are obtained.

[0185] In the embodiments of the present application, the traffic element recognition device can fuse the element image feature and the plurality of template image features by splicing or summing, to obtain a plurality of fusion image features corresponding to a plurality of traffic element templates, so as to facilitate subsequent prediction of the to-be-classified feature and the spatial potential field feature.

[0186] In some embodiments of the present application, the detection of the traffic element on the obtained road image to obtain the to-be-classified traffic element, that is, the specific implementation process of S101, can include: S1011-S1014, as follows:

[0187] S1011, performing feature extraction on the obtained road image to obtain a to-be-detected feature map.

[0188] S1012, generating a plurality of candidate regions for each feature point of the to-be-detected feature map.

[0189] The traffic element recognition device performs feature extraction on the obtained road image through the feature extraction model, and the obtained feature map is the to-be-detected feature map. Then, the traffic recognition device generates a plurality of candidate regions for each feature point in the to-be-detected feature map, and each candidate region contains the corresponding feature point.

[0190] In some embodiments, the traffic element recognition device can use each feature point of the feature map to be detected as the center to randomly generate multiple candidate regions (random scale, random aspect ratio). In other embodiments, the traffic element recognition device can generate a preset number of candidate regions for each feature point of the feature map to be detected according to certain candidate region generation rules. For example, it can use each feature point as the center to generate a preset number of candidate regions at a specific scale, or combine a specific aspect ratio at a specific scale to generate a preset number of candidate regions.

[0191] For example, Figure 8 This is a schematic diagram of a preset number of candidate regions provided in an embodiment of this application. The traffic element recognition device can set the scale of the candidate regions to 1 pixel, 2 pixels, and 3 pixels respectively, and set 3 aspect ratios at these scales: 1:1, 1:2, and 1:3 respectively. For each scale, the traffic element recognition device generates 3 candidate regions. Thus, for a certain feature point 8-1 in the feature map to be detected, 9 candidate regions will be generated.

[0192] Understandably, the feature extraction model includes convolutional layers, normalization layers, and activation layers. Convolutional layers are used to extract basic features such as edge textures; normalization layers normalize the features extracted by the convolutional layer, filtering out noise; and activation layers perform non-linear mapping on the features extracted by the convolutional layer, enhancing generalization ability.

[0193] S1013. Traffic elements are detected for the image features in multiple candidate regions respectively, and the detection results and confidence scores corresponding to each candidate region are obtained.

[0194] S1014. The image content in the candidate region corresponding to the highest confidence level is determined as the traffic element to be classified.

[0195] The traffic element recognition device uses a traffic element detection model to detect traffic elements in the image content of multiple candidate regions, thereby obtaining the detection result and confidence score for each candidate region. The detection result indicates whether the candidate region image contains traffic elements. Next, the traffic recognition device selects the region with the highest confidence score from the multiple candidate regions, and uses the image content of the candidate region corresponding to the highest confidence score as the traffic element to be classified.

[0196] For example, Figure 9 This is a schematic diagram illustrating the process of detecting traffic elements in a road image according to an embodiment of this application. See also... Figure 9 The traffic element recognition device first inputs the road image 9-1 into the feature extraction model 9-2 to obtain the features to be detected. Figure 9-3 Then from the features to be detectedFigure 9-3 The multiple candidate regions 9-4 are determined, and then the image features in the multiple candidate regions 9-4 (i.e., obtained from the to-be-detected feature map after the candidate regions are determined) are classified and detected 9-5 to determine whether the image part in the candidate region contains the traffic element, so that the image content in the candidate region with the highest confidence is taken as the to-be-classified traffic element.

[0197] Figure 10 is a schematic diagram of the detected to-be-classified traffic element provided by the embodiment of the present application. As shown in Figure 10 The traffic element recognition device detects 10-2 the traffic elements in the road image 10-1, and can determine the traffic elements 10-3 contained in the road image 10-1, i.e., the to-be-classified traffic elements.

[0198] In the embodiment of the present application, the traffic element recognition device will first extract features from the road image, and then generate multiple candidate regions for each feature point in the extracted to-be-detected feature map, detect the traffic elements in the image features in the multiple candidate regions, determine the detection result and the corresponding confidence for each candidate region, and determine the image content in the candidate region corresponding to the highest confidence as the to-be-classified traffic element, so as to subsequently identify the to-be-classified traffic element.

[0199] In some embodiments of the present application, before the traffic elements in the obtained road image are detected to obtain the to-be-classified traffic element, i.e., before S101, the method can further include: S105-S107, as follows:

[0200] S105, obtaining multiple traffic element images.

[0201] The traffic element recognition device can obtain multiple traffic element images by image acquisition equipment for image acquisition of traffic elements on the road, or download multiple traffic element images from the network, which is not limited in the present application.

[0202] S106, classifying the multiple traffic element images according to multiple preset categories of traffic elements to obtain traffic element images corresponding to each of the multiple preset categories.

[0203] S107, generating multiple traffic element templates using the traffic element images corresponding to each of the multiple preset categories.

[0204] The traffic element recognition device divides the plurality of traffic elements by image according to a plurality of preset categories of traffic elements, integrates the traffic element images of each preset category together, and obtains traffic element images corresponding to each preset category. Then, the traffic element recognition device directly uses the traffic element images corresponding to each of the plurality of preset categories as a plurality of traffic element templates, or regenerates a new image using the traffic element images corresponding to each of the plurality of preset categories as a traffic element template, to obtain a plurality of traffic element templates corresponding to the plurality of preset categories.

[0205] In some embodiments, the traffic element recognition device can classify the plurality of traffic element images by using an image classification model to realize the classification of the plurality of traffic element images. In other embodiments, the traffic element recognition device can also realize the classification of the plurality of traffic element images by matching the plurality of traffic element images with template images of each preset category.

[0206] It can be understood that the preset categories can be set according to actual conditions, for example, the preset categories are set as no left turn or speed limit 40, and the present application is not limited herein.

[0207] In the embodiments of the present application, the traffic element recognition device will first obtain a plurality of traffic element images before detecting traffic elements in the road image, to generate a plurality of traffic element templates from the plurality of traffic element images, so as to facilitate subsequent analysis of the traffic elements to be classified by using the traffic element templates.

[0208] In some embodiments of the present application, the plurality of traffic element templates are generated by using the traffic element images corresponding to each of the plurality of preset categories, that is, the specific implementation process of S107 can include S1071-S1072, as follows:

[0209] S1071, fusing the traffic element images corresponding to each of the plurality of preset categories to obtain a traffic element template corresponding to each of the plurality of preset categories.

[0210] S1072, obtaining a plurality of traffic element templates when the traffic element templates corresponding to the plurality of preset categories are generated.

[0211] The traffic element recognition device fuses the traffic element images corresponding to each of the preset categories into an image, and uses the fused image as a traffic element template corresponding to each of the preset categories. When the traffic element recognition device determines the traffic element templates corresponding to all of the preset categories, a plurality of traffic element templates are obtained. That is, in the embodiments of the present application, the plurality of traffic element templates correspond to the plurality of preset categories one by one.

[0212] It can be understood that the traffic element recognition device can directly weight the traffic element image corresponding to each preset category pixel by pixel to realize the fusion of the traffic element image corresponding to each preset category, or can input the traffic element template corresponding to each preset category into the image fusion model to realize the fusion of the traffic element image corresponding to each preset category through the image fusion model.

[0213] In the embodiment of the application, the traffic element recognition device fuses the traffic element image corresponding to each preset category to obtain the traffic element template corresponding to each preset category, so that the quality of the obtained multiple traffic element templates is higher.

[0214] In the following, an exemplary application of the embodiment of the application in an actual application scenario will be described.

[0215] The embodiment of the application is realized in the scenario of identifying traffic signs (traffic elements) in a road. Figure 11 is a schematic diagram of the process of identifying traffic signs in a road provided by the embodiment of the application. Referring to Figure 11 , the process includes:

[0216] S401, image acquisition.

[0217] The server (traffic element recognition device) photographs the road in front of the vehicle through the vehicle-mounted photographing device to form an image sequence (road image).

[0218] S402, sign detection.

[0219] The server uses the convolutional neural network in deep learning to extract features from the image sequence and detect the signs (to-be-classified traffic elements) appearing in the image.

[0220] Specifically, the server can obtain the features (element image features) of the images in the image sequence through the convolution layer, the normalization layer and the activation layer in the feature extraction layer (feature extraction model) in the convolutional neural network, and then generate Figure 8 9 detection candidate boxes (candidate regions) based on the features of the images, and select one detection box (candidate region corresponding to the highest confidence) from the 9 detection boxes according to the confidence, and identify the to-be-identified image (to-be-classified traffic element) in the detection box in the subsequent process.

[0221] S403, accurate identification of spatial potential field.

[0222] This step can be further divided into:

[0223] S4031, sign template (multiple traffic element templates) preparation.

[0224] For each known category of signboard, a fixed length of template feature is extracted by a convolutional neural network, thereby obtaining (feature of multiple traffic element templates), wherein z is the dimension of the feature, and n is the category of the signboard.

[0225] For each category of signboard, S4032 and S4033 are iterated.

[0226] S4032, the convolutional neural network extracts features and spatial potential field directions.

[0227] The server passes the pre-selected signboard template and the image to be recognized through the same convolutional neural network layer for feature extraction, then splices the features of the signboard template (template image feature vector) and the features of the image to be recognized (element image feature vector), and then performs convolution to fuse the features of the signboard template and the image to be recognized, to enrich the semantic information of the image features. Next, the server predicts the fused image features (fused image features), outputs two-dimensional information, which are the feature vector (to be classified feature) of the image to be recognized and the direction vector (spatial potential field feature) of the spatial potential field. Wherein z represents the dimension of the feature vector (feature dimension).

[0228] The advantage of introducing the spatial potential field direction vector is to increase the artificial prior knowledge in the convolutional neural network, to increase the model recognition ability of the convolutional neural network, and to increase the regulation of the model, to constrain the distance between the feature vector of the image to be recognized and the feature vector of the template image, that is, when the image to be recognized and the signboard template are of the same category, the spatial potential field method vector plays an attractive role, and narrows the distance between the two feature vectors, and vice versa. It plays a repulsive role, and enlarges the distance between the two feature vectors.

[0229] S4033, feature fusion and matching.

[0230] After the server obtains the feature vector of the image to be recognized and the direction vector of the spatial potential field, the spatial potential field direction vector is used to correct the feature vector of the image to be recognized in space, so as to more robustly recognize the category.

[0231] The direction vector of the spatial potential field is a z-dimensional vector, and each dimension (matching feature dimension) corresponds to each dimension (each feature dimension of the to-be-classified feature) of the feature vector of the image to be recognized. Assuming that the direction vector of the spatial potential field the value of the i-th dimension is a positive number (direction), indicating that the information of the i-th dimension and the feature of the label template have an attractive force, and the magnitude of the attractive force is the value of the i-th dimension (feature value); on the contrary, the direction vector of the spatial potential field the value of the i-th dimension is a negative number, indicating that the information of the i-th dimension and the feature of the template image have a repulsive force, and the magnitude of the repulsive force is the value of the i-th dimension (feature value). In this way, the feature vector of the identified image and the direction vector of the assumed spatial potential field can be fused by formula (1) to obtain the fused feature vector (correction feature).

[0232] The fused feature vector is matched with the feature of the prepared label template for similarity.

[0233] S404, determine the category.

[0234] The server finds the category of the most similar template (the matching element template corresponding to the largest feature similarity) as the final category (target category) of the to-be-identified image.

[0235] As can be seen from the above, by correcting the feature of the to-be-identified image through the direction vector of the spatial potential field, the problem of uneven sample distribution can be solved to enable the identification of label categories that are not covered by the identification model, and the problem of identification error can be solved, thereby improving the identification accuracy of the label.

[0236] The following continues to describe an exemplary structure of the traffic element identification apparatus 555 provided by the embodiments of the present application as a software module. In some embodiments, as shown in FIG. 5B, the software module stored in the traffic element identification apparatus 555 in the memory 550 can include: Figure 2

[0237] An image detection module 5551 is configured to detect traffic elements in the acquired road image to obtain to-be-classified traffic elements; wherein the traffic elements include traffic signs or monitoring facilities in the road.

[0238] A feature prediction module 5552 is configured to predict to-be-classified features from the to-be-classified traffic elements and a plurality of traffic element templates, and a plurality of spatial potential field features corresponding to the plurality of traffic element templates; wherein the spatial potential field features represent the attraction or repulsion degree between the to-be-classified traffic elements and the traffic element templates.

[0239] A template matching module 5553 is configured to find a matching element template corresponding to the to-be-classified traffic elements from the plurality of traffic element templates according to the plurality of spatial potential field features and the to-be-classified features.​

[0240] The category determining module 5554 is configured to determine the category corresponding to the matching element template as the target category of the traffic element to be classified.

[0241] In some embodiments of the present application, the template matching module 5553 is further configured to perform spatial correction on the feature to be classified based on the plurality of spatial potential field features, to obtain a plurality of corrected features corresponding to the plurality of spatial potential field features; and determine the matching element template corresponding to the traffic element to be classified from the plurality of traffic element templates according to the similarity between the plurality of corrected features and the features of the plurality of traffic element templates.

[0242] In some embodiments of the present application, the template matching module 5553 is further configured to perform spatial correction on the feature to be classified by using each spatial potential field feature in the plurality of spatial potential field features, to obtain a corrected feature corresponding to each spatial potential field feature; and obtain the plurality of corrected features corresponding to the plurality of spatial potential field features when the spatial correction on the feature to be classified is completed by using all the plurality of spatial potential field features.

[0243] In some embodiments of the present application, the template matching module 5553 is further configured to select, from a plurality of feature dimensions of the corrected feature corresponding to each spatial potential field feature, a matching feature dimension corresponding to each feature dimension of the feature to be classified; calculate a correction component for each feature dimension of the feature to be classified by using a feature value and a direction corresponding to the matching feature dimension; and fuse the feature component of each feature dimension of the feature to be classified and the correction component, to obtain the corrected feature corresponding to each spatial potential field feature.

[0244] In some embodiments of the present application, the template matching module 5553 is further configured to perform similarity calculation on each corrected feature in the plurality of corrected features and the features of the plurality of traffic element templates, to obtain a plurality of feature similarities; and select the matching traffic element template from the plurality of traffic element templates, wherein the matching traffic element template corresponds to the maximum feature similarity.

[0245] In some embodiments of the present application, the template matching module 5553 is further configured to perform similarity calculation on the feature components of a plurality of feature dimensions in each corrected feature and the feature components of a plurality of feature dimensions in the features of each traffic element template, to obtain a feature similarity corresponding to each traffic element template; and obtain the plurality of feature similarities when the corresponding feature similarity is calculated for the features of the plurality of traffic element templates.

[0246] In some embodiments of the present application, the template matching module 5553 is further configured to determine, for each of the plurality of feature dimensions of the rectified feature, a target feature dimension from the plurality of feature dimensions of the feature of the each traffic element template; calculate distances between feature components of the plurality of feature dimensions of the each rectified feature and feature components corresponding to the target feature dimension to obtain feature distances corresponding to the plurality of feature dimensions of the each rectified feature; and accumulate the feature distances corresponding to the plurality of feature dimensions of the each rectified feature to obtain the feature similarity corresponding to the each traffic element template.

[0247] In some embodiments of the present application, the feature prediction module 5552 is further configured to perform image feature extraction on the traffic element to be classified to obtain an element image feature; perform image feature extraction on the plurality of traffic element templates respectively to obtain a plurality of template image features corresponding to the plurality of traffic element templates; fuse the element image feature with the plurality of template image features respectively to obtain a plurality of fused image features corresponding to the plurality of traffic element templates; and predict the plurality of spatial potential field features corresponding to the traffic element to be classified and the plurality of traffic element templates from the plurality of fused image features.

[0248] In some embodiments of the present application, the element image feature includes an element image feature vector, and the template image feature includes a template image feature vector.

[0249] The feature prediction module 5552 is further configured to splice the element image feature vector with a plurality of template image feature vectors respectively to obtain the plurality of fused image features corresponding to the plurality of traffic element templates, or sum the element image feature vector with the plurality of template image feature vectors respectively to obtain the plurality of fused image features corresponding to the plurality of traffic element templates.

[0250] In some embodiments of the present application, the image detection module 5551 is further configured to perform feature extraction on the acquired road image to obtain a feature map to be detected; generate a plurality of candidate regions for each feature point of the feature map to be detected; perform detection of traffic elements on image features in the plurality of candidate regions respectively to obtain detection results and confidences corresponding to the plurality of candidate regions respectively; and determine image content in a candidate region corresponding to the highest confidence as the traffic element to be classified.

[0251] In some embodiments of the present application, the traffic element identification apparatus 555 further includes a template generation module 5555.

[0252] The template generation module 5555 is configured to acquire a plurality of traffic element images; perform category division on the plurality of traffic element images according to a plurality of preset categories of traffic elements, to obtain traffic element images corresponding to each of the plurality of preset categories; and generate a plurality of traffic element templates by using the traffic element images corresponding to each of the plurality of preset categories.

[0253] In some embodiments of the present application, the template generation module 5555 is further configured to fuse the traffic element images corresponding to each of the plurality of preset categories to obtain a traffic element template corresponding to each of the plurality of preset categories; and obtain the plurality of traffic element templates when a traffic element template is generated for each of the plurality of preset categories.

[0254] The embodiments of the present application provide a computer program product or a computer program, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the traffic element identification method provided in the embodiments of the present application.

[0255] The embodiments of the present application provide a computer readable storage medium storing executable instructions, wherein the executable traffic element identification instructions are stored in the computer readable storage medium. When the executable traffic element identification instructions are executed by a processor, the processor will execute the traffic element identification method provided in the embodiments of the present application, for example, the method shown in the above. Figure 3

[0256] In some embodiments, the computer readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disc, or CD-ROM; or various devices including one or any combination of the above memories.

[0257] In some embodiments, the executable traffic element identification instructions can be in the form of a program, software, software module, script or code, written in any form of programming language (including a compiled or interpreted language, or a declarative or procedural language), and can be deployed in any form, including being deployed as a standalone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0258] ​By way of example, executable traffic element recognition instructions can, but need not necessarily, correspond to a file in a file system, can be stored in a part of a file that is dedicated to the program in question, or, can be stored in multiple cooperating files (e.g., files that store one or more modules, sub programs, or code portions).

[0259] By way of example, executable traffic element recognition instructions can be deployed to execute on one computer device, or on multiple computer devices located at one site, or on multiple computer devices distributed among multiple sites and interconnected via a communication network.

[0260] The above description is only for the embodiments of the present application, and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement and improvement made within the spirit and scope of the present application shall fall within the protection scope of the present application.

Claims

1. A traffic element recognition method characterized by, The method comprises the following steps: detecting traffic elements in the acquired road image to obtain a to-be-classified traffic element; wherein the traffic elements include traffic signs or monitoring facilities in the road; extracting image features of the to-be-classified traffic element to obtain element image features, and extracting image features of a plurality of traffic element templates to obtain a plurality of template image features corresponding to the plurality of traffic element templates; fusing the element image features with the plurality of template image features to obtain a plurality of fused image features corresponding to the plurality of traffic element templates, and predicting a to-be-classified feature and a plurality of spatial potential field features corresponding to the plurality of traffic element templates from the plurality of fused image features; wherein the spatial potential field features represent the attraction or repulsion degree between the to-be-classified traffic element and the traffic element templates; finding a matching element template corresponding to the to-be-classified traffic element from the plurality of traffic element templates according to the plurality of spatial potential field features and the to-be-classified feature; determining the category of the matching element template as the target category of the to-be-classified traffic element.

2. The method of claim 1, wherein, The method of finding a matching element template corresponding to the to-be-classified traffic element from the plurality of traffic element templates according to the plurality of spatial potential field features and the to-be-classified feature comprises: performing spatial correction on the to-be-classified feature based on the plurality of spatial potential field features to obtain a plurality of corrected features corresponding to the plurality of spatial potential field features; determining the matching element template corresponding to the to-be-classified traffic element from the plurality of traffic element templates according to the similarity between the plurality of corrected features and the features of the plurality of traffic element templates.

3. The method of claim 2, wherein, The method of performing spatial correction on the to-be-classified feature based on the plurality of spatial potential field features to obtain a plurality of corrected features corresponding to the plurality of spatial potential field features comprises: performing spatial correction on the to-be-classified feature using each spatial potential field feature in the plurality of spatial potential field features to obtain a corrected feature corresponding to each spatial potential field feature; obtaining the plurality of corrected features corresponding to the plurality of spatial potential field features when spatial correction on the to-be-classified feature is completed using all the plurality of spatial potential field features.

4. The method of claim 3, wherein, The method of performing spatial correction on the to-be-classified feature using each spatial potential field feature in the plurality of spatial potential field features to obtain a corrected feature corresponding to each spatial potential field feature comprises: selecting a matching feature dimension corresponding to each feature dimension of the to-be-classified feature from a plurality of feature dimensions of the corrected feature corresponding to each spatial potential field feature; calculating a correction component for each feature dimension of the to-be-classified feature using a feature value and a direction corresponding to the matching feature dimension; fusing the feature component of each feature dimension of the to-be-classified feature with the correction component to obtain the corrected feature corresponding to each spatial potential field feature.

5. The method according to any one of claims 2 to 4, characterized in that, The method of determining the matching element template corresponding to the to-be-classified traffic element from the plurality of traffic element templates according to the similarity between the plurality of corrected features and the features of the plurality of traffic element templates comprises: The similarity of each of the plurality of correction features and the features of the plurality of traffic element templates is calculated to obtain a plurality of feature similarities; The matching element template is selected from the plurality of traffic element templates, wherein the matching element template corresponds to the maximum feature similarity.

6. The method of claim 5, wherein, The similarity of each of the plurality of correction features and the features of the plurality of traffic element templates is calculated to obtain a plurality of feature similarities, including: The feature similarity corresponding to each traffic element template is obtained by calculating the similarity of the feature components of the plurality of feature dimensions in each correction feature and the feature components of the plurality of feature dimensions in each traffic element template; When the corresponding feature similarity of the features of the plurality of traffic element templates is calculated, the plurality of feature similarities is obtained.

7. The method of claim 6, wherein, The feature similarity corresponding to each traffic element template is obtained by calculating the similarity of the feature components of the plurality of feature dimensions in each correction feature and the feature components of the plurality of feature dimensions in each traffic element template, including: For the plurality of feature dimensions in each correction feature, the target feature dimension is determined from the plurality of feature dimensions of the features of each traffic element template; The feature distance corresponding to the plurality of feature dimensions of each correction feature is obtained by calculating the distance between the feature components of the plurality of feature dimensions in each correction feature and the feature components corresponding to the target feature dimension; The feature distance corresponding to the plurality of feature dimensions of each traffic element template is obtained by accumulating the feature distance corresponding to the plurality of feature dimensions of each correction feature.

8. The method of claim 1, wherein, The element image feature includes an element image feature vector, and the template image feature includes a template image feature vector; The element image feature is fused with the plurality of template image features respectively to obtain a plurality of fusion image features corresponding to the plurality of traffic element templates, including: The element image feature vector is spliced with a plurality of template image feature vectors respectively to obtain the plurality of fusion image features corresponding to the plurality of traffic element templates; Or, The element image feature vector is summed with the plurality of template image feature vectors respectively to obtain the plurality of fusion image features corresponding to the plurality of traffic element templates.

9. The method according to any one of claims 1 to 4, 6 to 8, characterized in that, The detection of the traffic element of the obtained road image to obtain the traffic element to be classified includes: The feature extraction of the obtained road image is performed to obtain a feature map to be detected; For each feature point of the feature map to be detected, a plurality of candidate regions are generated; The image features in the plurality of candidate regions are detected respectively to obtain the detection result and the confidence corresponding to each of the plurality of candidate regions; The image content in the candidate region corresponding to the highest confidence is determined as the traffic element to be classified.

10. The method according to any one of claims 1 to 4, 6 to 8, characterized in that, Before the detection of the traffic element of the obtained road image to obtain the traffic element to be classified, the method further includes: Obtain a plurality of traffic element images; The traffic element images are classified according to a plurality of preset categories of traffic elements, to obtain traffic element images corresponding to each of the plurality of preset categories; The plurality of traffic element templates are generated by using the traffic element images corresponding to each of the plurality of preset categories.

11. The method of claim 10, wherein, The plurality of traffic element templates are generated by using the traffic element images corresponding to each of the plurality of preset categories. The traffic element images corresponding to each of the plurality of preset categories are fused to obtain a traffic element template corresponding to each of the plurality of preset categories; When the traffic element templates corresponding to the plurality of preset categories are generated, the plurality of traffic element templates are obtained.

12. A traffic element recognition apparatus characterized by comprising: It comprises: An image detection module is configured to detect traffic elements in a road image to obtain traffic elements to be classified, wherein the traffic elements include traffic signs or monitoring facilities in the road; A feature prediction module is configured to extract image features of the traffic elements to be classified, to obtain element image features, and to extract image features of a plurality of traffic element templates to obtain a plurality of template image features corresponding to the plurality of traffic element templates; the element image features are fused with the plurality of template image features to obtain a plurality of fused image features corresponding to the plurality of traffic element templates, and a classified feature and a plurality of spatial potential field features corresponding to the plurality of traffic element templates are predicted from the plurality of fused image features; wherein the spatial potential field features represent the attraction or repulsion degree between the traffic elements to be classified and the traffic element templates; A template matching module is configured to find a matching element template corresponding to the traffic elements to be classified from the plurality of traffic element templates according to the plurality of spatial potential field features and the classified feature; A category determination module is configured to determine a category corresponding to the matching element template as a target category of the traffic elements to be classified.

13. The apparatus of claim 12, wherein The template matching module is further configured to correct the classified feature based on the plurality of spatial potential field features to obtain a plurality of corrected features corresponding to the plurality of spatial potential field features; and determine the matching element template corresponding to the traffic elements to be classified from the plurality of traffic element templates according to similarities between the plurality of corrected features and features of the plurality of traffic element templates.

14. The apparatus of claim 13, wherein The template matching module is further configured to correct the classified feature based on each of the plurality of spatial potential field features to obtain a corrected feature corresponding to each of the plurality of spatial potential field features; and obtain the plurality of corrected features corresponding to the plurality of spatial potential field features when the classified feature is corrected based on each of the plurality of spatial potential field features.

15. A traffic element recognition apparatus characterized by comprising: It comprises: A memory is configured to store executable traffic element identification instructions; A processor is configured to execute the executable traffic element identification instructions stored in the memory to implement the method of any one of claims 1 to 11.

16. A computer readable storage medium characterized by: A computer program product comprising computer instructions stored on a computer readable medium, the computer instructions, when executed by a processor, implement the method of any one of claims 1 to 11.

17. A computer program product comprising computer instructions, characterized in that, The computer instructions, when executed by a processor, implement the method of any one of claims 1 to 11.

Images