Traffic road sign recognition method, device, equipment and storage medium

Abstract

The present invention provides a method, device, equipment and storage medium for identifying traffic road signs, the method comprising: in response to the existence of road signs in the acquisition field of view of the acquisition module, sequentially calculating the road sign type confidence of the image acquired by the acquisition module from the current moment; selecting a final frame image according to the calculated road sign type confidence; and identifying road sign information on the final frame image. The present invention can reduce the demand for on-site analysis computing power and optimize the recognition of road signs.

Description

Background Art

[0002] With the continuous development of transportation in my country, the working status and standardization of traffic road signs, as an important part of traffic management, directly affect people's travel, logistics and transportation experience, and the incidence of traffic accidents. Road management departments need to conduct regular inspections of road signs, but traditional inspections are mainly manual, and there is room for improvement in efficiency, cost, and accuracy.

[0003] The current traffic road sign recognition mainly has the following problems:

[0004] Conventional traffic road sign recognition methods require continuous collection of road surface information and a large amount of recognition calculations, so they require high on-site intelligent analysis computing power. Sometimes it is necessary to record the collected video and then perform recognition analysis after returning to the station, which is time-consuming and laborious;

[0005] Since traffic road signs often have multiple road signs at the same time, it is difficult to identify and locate the data, resulting in data confusion;

[0006] Although road signs can be recognized when the vehicle is driving, the recognition distance is far and the image pixels are low, resulting in blur. When the recognition distance is close, the road signs move quickly, which can also cause blurry images and may be missed.

[0007] The problem of repeated identification and registration of the same road sign;

[0008] During the inspection, there is also the need to continuously collect road information and compare road sign data, which requires high computing power and requires analysis after recording and returning to the base, resulting in untimely results of inspection problems.

[0009] Therefore, how to reduce the demand for on-site analysis computing power and optimize the recognition of road signs is a technical problem that needs to be urgently solved by technical personnel in this field.

[0010] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0011] In view of the problems in the prior art, the purpose of the present invention is to provide a method, device, equipment and storage medium for traffic road sign recognition, which overcomes the difficulties of the prior art, reduces the demand for on-site analysis computing power, and optimizes the recognition of road signs.

[0012] An embodiment of the present invention provides a method for identifying a traffic road sign, comprising:

[0013] In response to the presence of a road sign in the acquisition field of view of the acquisition module, sequentially calculating the road sign type confidence of the image acquired by the acquisition module since the current moment;

[0014] Selecting a final frame image according to the calculated road sign type confidence;

[0015] Road sign information is recognized on the final frame image.

[0016] In some embodiments of the present invention, after the road sign information recognition is performed on the final frame image, the method further includes:

[0017] The road sign information and the positioning information of the road sign are stored in a road sign database.

[0018] In some embodiments of the present invention, the positioning information of the road sign is obtained according to the following steps:

[0019] Get the time series of positioning information;

[0020] Obtaining the acquisition time of the final frame image;

[0021] The positioning information of the road sign is determined according to the matching of the acquisition time of the final frame image and the time series of the positioning information.

[0022] In some embodiments of the present invention, storing the road sign information and the positioning information of the road sign in a road sign database further includes:

[0023] The position information of the road sign in the final frame image is stored in a road sign database.

[0024] In some embodiments of the present invention, after storing the road sign information and the location information of the road sign in the road sign database, the method further includes:

[0025] During the inspection phase, collecting images at the location information in the road sign database;

[0026] Identify road sign information of road signs in the captured image;

[0027] Comparing the road sign information identified in the inspection phase with the road sign information in the road sign database;

[0028] Output the comparison results.

[0029] In some embodiments of the present invention, selecting the final frame image according to the calculated road sign type confidence comprises:

[0030] When the calculated road sign type confidence is greater than a set threshold, the current image is used as the final frame image.

[0031] In some embodiments of the present invention, after the road sign information recognition is performed on the final frame image, the method further includes:

[0032] The road signs with the same road sign information recognized in consecutive final frames are labeled.

[0033] According to another aspect of the present invention, there is also provided a traffic road sign recognition device, comprising:

[0034] a confidence calculation module configured to sequentially calculate the confidence of the road sign type of the image collected by the collection module from the current moment in response to the presence of a road sign in the collection field of view of the collection module;

[0035] A final frame selection module is configured to select a final frame image according to the calculated road sign type confidence;

[0036] The recognition module is configured to recognize road sign information on the final frame image.

[0037] According to another aspect of the present invention, there is also provided a traffic road sign recognition and processing device, comprising:

[0038] processor;

[0039] a memory storing executable instructions of the processor;

[0040] The processor is configured to execute the steps of the traffic road sign recognition method as described above by executing the executable instructions.

[0041] An embodiment of the present invention further provides a computer-readable storage medium for storing a program, wherein the program implements the steps of the above-mentioned traffic road sign recognition method when executed.

[0042] Compared with the prior art, the purpose of the present invention is:

[0043] The present invention responds to the existence of road signs in the acquisition field of view of the acquisition module, and sequentially calculates the road sign type confidence of the image acquired by the acquisition module from the current moment, thereby selecting a final frame image according to the calculated road sign type confidence, and performing road sign information recognition on the final frame image to achieve three-stage recognition and analysis. On the one hand, there is no need to recognize the road sign type and road sign information for all images, and on the other hand, only the road sign information can be recognized for the final frame image with a higher confidence, thereby reducing the high computing power requirements of the real-time road sign image recognition system and optimizing the recognition of road signs on the premise that the road sign information can be accurately recognized. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Other features, objectives and advantages of the present invention will become more apparent from a reading of the detailed description of non-limiting embodiments made with reference to the following accompanying drawings.

[0045] Figure 1 It is a flow chart of an embodiment of the method for recognizing traffic road signs of the present invention.

[0046] Figure 2 It is a flow chart of another embodiment of the traffic road sign recognition method of the present invention.

[0047] Figure 3 It is a flow chart of another embodiment of the traffic road sign recognition method of the present invention.

[0048] Figure 4 The present invention is a flow chart of a method for identifying a traffic road sign.

[0049] Figure 5 The present invention is a flow chart of the road sign inspection of the traffic road sign recognition method.

[0050] Figure 6 It is a module schematic diagram of an embodiment of the traffic road sign recognition device of the present invention.

[0051] Figure 7 It is a module schematic diagram of another embodiment of the traffic road sign recognition device of the present invention.

[0052] Figure 8 It is a module schematic diagram of an embodiment of the traffic road sign recognition system of the present invention.

[0053] Fig. 9 It is a structural schematic diagram of the traffic road sign recognition device of the present invention.

[0054] Fig.10 It is a schematic diagram of the structure of a computer-readable storage medium according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present invention will be comprehensive and complete and fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar structures, and thus their repeated description will be omitted.

[0056] See also Figure 1 , Figure 1The flowchart of an embodiment of the method for identifying traffic road signs of the present invention is shown in FIG. The embodiment of the present invention provides a method for identifying traffic road signs, comprising the following steps:

[0057] Step S110: In response to the existence of road signs in the acquisition field of view of the acquisition module, the confidence levels of road sign types of images acquired by the acquisition module from the current moment are calculated in sequence.

[0058] Specifically, the acquisition module may be disposed on the vehicle to acquire images / videos in the forward direction (the direction in which the vehicle is traveling). In some embodiments, step S110 may be implemented by identifying road signs in the images (also referred to as image frames) acquired by the acquisition module.

[0059] Specifically, the recognition of road signs can be the recognition of road sign outlines. The recognition of road sign outlines requires relatively low computing power. Therefore, the calculation of the road sign type confidence is performed only after the road sign outline is recognized. In other words, under normal circumstances, the traffic road sign recognition method performs the recognition of road sign outlines, thereby greatly reducing the computing power requirements for road sign recognition.

[0060] Specifically, when calculating the confidence of the road sign type of the image collected by the acquisition module at the current moment, the road sign type of the road sign is also identified. The road sign type of the road sign may include but is not limited to: warning signs, prohibition signs, instruction signs, guide signs, tourist signs, etc. The road sign type of the road sign can be identified by, for example, the characteristics of the road sign (such as the shape and color of the road sign) using a convolutional neural network algorithm. During the recognition process, the convolutional neural network algorithm can also output the confidence that the road sign in the image belongs to the identified road sign type. In some variations, the road sign type confidence can also be calculated using other algorithms.

[0061] Step S120: selecting a final frame image according to the calculated road sign type confidence.

[0062] Specifically, as the vehicle moves forward, the clarity of the sign increases, so the confidence of its prediction gradually increases. When the prediction confidence reaches the set threshold (which can be adjusted according to actual conditions), the frame can be used as the final frame of the sign. Therefore, selecting the final frame can avoid the trouble of recognition analysis after video recording and realize real-time analysis and recording.

[0063] Step S130: performing road sign information recognition on the final frame image.

[0064] Specifically, the road sign information identified in step S130 may include specific road sign information such as the meaning of the road sign and the text of the road sign.

[0065] The present invention responds to the existence of road signs in the acquisition field of view of the acquisition module, and sequentially calculates the road sign type confidence of the image acquired by the acquisition module from the current moment, thereby selecting a final frame image according to the calculated road sign type confidence, and performing road sign information recognition on the final frame image to achieve three-stage recognition and analysis. On the one hand, there is no need to recognize the road sign type and road sign information for all images, and on the other hand, only the road sign information can be recognized for the final frame image with a higher confidence, thereby reducing the high computing power requirements of the real-time road sign image recognition system and optimizing the recognition of road signs on the premise that the road sign information can be accurately recognized.

[0066] See below Figure 2 , Figure 2 FIG. 1 is a flow chart of another embodiment of the method for identifying traffic road signs of the present invention. Figure 2 As shown, the traffic road sign recognition method includes:

[0067] Step S201: In response to the existence of road signs in the acquisition field of view of the acquisition module, the confidence levels of road sign types of images acquired by the acquisition module from the current moment are calculated in sequence.

[0068] Step S202: when the calculated road sign type confidence is greater than a set threshold, the current image is used as the final frame image.

[0069] Step S203: performing road sign information recognition on the final frame image.

[0070] Step S204: Acquire the time series of positioning information.

[0071] Step S205: Obtaining the acquisition time of the final frame image.

[0072] Step S206: determining the positioning information of the road sign according to the matching between the acquisition time of the final frame image and the time series of the positioning information.

[0073] Step S207: marking the road signs with the same road sign information identified in the continuous final frames.

[0074] Specifically, when road signs with the same road sign information are identified in consecutive final frames, the final frame with the highest confidence in the road sign type can be selected, and based on the selected final frame, the road sign information identified from the final frame (as well as positioning information, the position information of the road sign in the final frame image) can be saved.

[0075] Step S208: storing the road sign information and positioning information of the road sign, and the position information of the road sign in the final frame image into a road sign database.

[0076] Step S209: During the inspection phase, an image is collected at the location information in the road sign database.

[0077] Step S210: Identify road sign information of road signs in the captured image.

[0078] Step S211: comparing the road sign information identified in the inspection phase with the road sign information in the road sign database.

[0079] Step S212: Output the comparison result.

[0080] The following are combined Figures 3 to 5 The specific implementation method of this application is described. Figure 3 It is a flow chart of another embodiment of the traffic road sign recognition method of the present invention. Figure 3 The following steps are shown:

[0081] Step S310: acquiring image frames at a preset sampling period and performing preprocessing.

[0082] Specifically, step S310 to step S360 are executed while the vehicle is moving.

[0083] The image acquisition module can be placed on the driving instrument panel of a moving vehicle or on the roof of the vehicle to facilitate the acquisition of forward images. The acquisition module can pre-process the acquired images, including extracting frames and saving them according to a preset time period (for example, an interval of 2-5 seconds can be selected, but this application is not limited to this), recording the acquisition time point, etc. The output of the acquisition module can be in the form of video or sequence images.

[0084] Step S320: Record the road sign video frames in time series, and correspondingly record the satellite positioning information when each frame is shot.

[0085] Specifically, since the steps in the present application may not be performed in sequence, step S320 may be performed after the final frame of the road sign is determined.

[0086] Due to the mobile environment, traffic road signs are viewed from far to near, and the same road sign information will appear in continuous images. However, the road sign resolution of the farther image is lower, while the closer image is faster, which can easily cause blur. Therefore, this application adopts a final frame acquisition algorithm to select the final image frame with better quality from the continuous image frames for analysis and storage. On the one hand, it reduces the computing power requirements for repeated recognition of each frame of the image and reduces the computing power waste of repeated predictions. Secondly, it also improves the prediction accuracy, and the selected positioning also lays the foundation for the accuracy of subsequent inspection data.

[0087] Step S330: Identify whether there is a road sign in the shooting field of view. If there is a road sign, start the final frame recognition method.

[0088] Step S340: In the road sign sequence, select a final frame based on the road sign type confidence.

[0089] Step S350: Perform secondary recognition on the final frame.

[0090] Specifically, the above steps S330 to S350 will be combined with Figure 4 Provide a detailed description.

[0091] Step S360: Write the identified road sign type, road sign meaning, road sign text and positioning information into the road sign database.

[0092] Specifically, after step S360, the road sign image structured data storage process can be performed: based on the final frame selection method, due to the road sign detection method based on target detection, the detection of all road sign targets in a single frame will be realized, and the corresponding road sign in the detection will return the road sign classification, the position information of the road sign in the image, the road sign positioning information, etc. Therefore, for the same scene, multiple road signs will be located and identified respectively, and recorded independently.

[0093] Specifically, in a multiple road sign scenario, if the same road sign appears in multiple consecutive final frames, the road signs with the same information in the consecutive final frames can be marked to facilitate direct deletion of duplicate items or manual screening to prevent duplicate registration.

[0094] Specifically, the vehicle's satellite positioning system synchronously outputs positioning data in time sequence, which is used to form structured data of the mapping relationship between road signs and geographical locations. The design of the relevant road sign database is shown in the following table:

[0095]

[0096] The above is only a schematic illustration of the data structure of the road sign of the present application.

[0097] See below Figure 4 , Figure 4 The present invention is a flow chart of a method for identifying a traffic road sign. Figure 4 The following steps are shown:

[0098] Step S401: performing road sign recognition on the captured image frame (only identifying whether there is a road sign in the image frame).

[0099] Step S402: After the road signs are identified in step S401, the road signs are numbered based on their features.

[0100] Step S403: In each image frame, for each numbered road sign, identify its road sign type and road sign pattern, and calculate its confidence.

[0101] Step S404: when the confidence level of a certain image frame reaches a set threshold, the image frame is used as the final frame of the road sign.

[0102] Therefore, under the normal condition of license plate recognition, the recognition module recognizes the outline of road signs through the image analysis and processing system to reduce the consumption of computing power. After the road sign is recognized, each sign is numbered according to the road sign type based on the road sign characteristics. If there are multiple signboards in the same frame, each signboard is numbered separately, and the type of road sign and road sign pattern are identified. And the confidence is calculated. Generally speaking, as the vehicle moves forward, the clarity of the signboard increases, so the confidence of its prediction gradually increases. After this prediction confidence reaches a certain value (which can be adjusted according to actual conditions), the frame can be used as the final frame of the signboard. This method of selecting the final frame avoids the trouble of recognition and analysis after video recording, and realizes real-time analysis and recording.

[0103] Furthermore, in the road sign recognition scheme of the present application, the recognition is carried out in three times: the first is to recognize the traffic road signs on the road under normal conditions; the second stage of recognition mainly recognizes the types of traffic road signs including: warning signs, prohibition signs, instruction signs, guide signs, and tourist signs. The type of traffic road sign can be recognized by: the shape, color and other features of the road sign, using a deep learning neural network algorithm for recognition, with a high recognition accuracy; through the second recognition, the confidence is obtained to obtain the final frame; the third stage of recognition, the road sign image, for road signs such as scenic spots, further text recognition (such as OCR (Optical Character Recognition)) can be performed to obtain data such as scenic spot information for entry into the road sign database.

[0104] See below Figure 5 , Figure 5 The present invention is a flow chart of the road sign inspection of the traffic road sign recognition method. Figure 5 The following steps are shown:

[0105] Step S501: When the vehicle is traveling at a constant speed, the positioning system determines whether it has reached the sign positioning position registered in the database.

[0106] Step S502: After reaching the recorded positioning position, photograph the sign.

[0107] Step S503: Identify the sign image frame to identify the sign, the number, and the position of the sign in the image.

[0108] Step S504: Perform secondary recognition on each recognized sign.

[0109] Step S505: Generate inspection sign data, compare it with all sign data at the current location in the sign database, and generate recognition confidence and accuracy.

[0110] Step S506: Conduct on-site inspection of low-confidence and misidentified signs.

[0111] Therefore, through the above steps, when the road administration unit conducts inspections, the image acquisition module is used to only collect images at the satellite positioning locations indicated (marked) by the road sign database, identify the road sign information at the location during the inspection, and compare it with the data information in the original road sign database. If the road sign is not identified or the data confidence level is lower than a certain threshold, it is determined that the road sign is damaged or broken. At this time, an alarm can be issued to remind road administration personnel to conduct manual verification and repair.

[0112] The existing technology uses a conventional single-shot acquisition and recognition method for traffic sign recognition, which has high computing power requirements and generally requires analysis and recognition after acquisition. This application reduces the computing power requirements of the equipment through three-stage analysis, which is conducive to the real-time analysis of traffic sign inspections.

[0113] The existing patents conduct road inspection, comparison, analysis and identification processes, which require high computing power of the equipment. This method is based on the existing positioning information, conducts targeted collection and comparison, and reduces the computing power requirements of the equipment.

[0114] Traffic signs are often installed in a centralized manner. Existing patented technologies may have problems such as incorrect identification and registration, and duplicate registration when identifying multiple road signs in the same scene. This patent proposes a method for processing and identifying multiple road signs in the same scene, which can solve the above problems.

[0115] Compared with the existing technology, the main advantages are: it can reduce the computing power required by the road sign recognition system under the condition of real-time collection and analysis, without the need to return to the station for retrospective analysis; it can reduce the misrecognition and erroneous entry in multi-sign scenarios, and provide a solution for duplicate registration of road signs; during the inspection process, it can reduce the computing power required by the inspection identification and comparison system, without the need to return to the station for retrospective analysis.

[0116] Compared with the existing technology, the main creativity lies in: reducing the high computing power requirements of the terminal for real-time road sign image recognition, which is conducive to the realization of real-time recognition and registration. Specifically, it is a recognition and analysis method based on three stages, and a method that uses only the road signs that have been recorded for comparison. Both methods can filter out unnecessary processing work in the early stage; improve the recognition quality of road traffic signs through the final frame selection method; design a method for recording and entering road sign data for multiple sign recognition in a single scene, and a method for marking duplicate registered road sign information to facilitate automatic or manual processing; perform road sign data analysis and comparison to identify whether the signs are missing or damaged.

[0117] The above is only a schematic description of the specific implementation of the present invention, and the present invention is not limited thereto. The splitting, merging, changing of the execution order of steps, splitting, merging of modules, and changing of information transmission are all within the protection scope of the present invention.

[0118] Figure 6 Schematic diagram of a module of an embodiment of the traffic road sign recognition device of the present invention. The traffic road sign recognition device 600 of the present invention, such as Figure 6 As shown, it includes but is not limited to: a confidence calculation module 610 , a final frame selection module 620 and a recognition module 630 .

[0119] The confidence calculation module 610 is configured to sequentially calculate the confidence of the road sign type of the image collected by the collection module from the current moment in response to the presence of a road sign in the collection field of view of the collection module;

[0120] The final frame selection module 620 is configured to select a final frame image according to the calculated road sign type confidence;

[0121] The recognition module 630 is configured to recognize road sign information on the final frame image.

[0122] The implementation principle of the above modules can be found in the relevant introduction of the traffic road sign recognition method, which will not be repeated here.

[0123] The traffic road sign recognition device of the present invention responds to the presence of road signs in the collection field of view of the collection module, and sequentially calculates the road sign type confidence of the image collected by the collection module from the current moment, thereby selecting a final frame image according to the calculated road sign type confidence, and performing road sign information recognition on the final frame image to achieve three-stage recognition and analysis. On the one hand, there is no need to identify the road sign type and road sign information for all images. On the other hand, road sign information can be recognized only for the final frame image with a higher confidence level. Therefore, on the premise of being able to accurately identify the road sign information, the high computing power requirement of the real-time road sign image recognition system is reduced, thereby optimizing the recognition of road signs.

[0124] Figure 7 700 is a module diagram of another embodiment of the traffic road sign recognition device of the present invention. The traffic road sign recognition device 700 of the present invention includes but is not limited to: a confidence calculation module 701, a final frame selection module 702, a recognition module 703, a time series acquisition module 704, a collection time acquisition module 705, a positioning information determination module 706, a labeling module 707, a storage module 708, a patrol collection module 709, a patrol recognition module 710, a comparison module 711 and an output module 712.

[0125] The confidence calculation module 701 is configured to sequentially calculate the confidence of the road sign type of the image collected by the collection module from the current moment in response to the existence of a road sign in the collection field of view of the collection module.

[0126] The final frame selection module 702 is configured to use the current image as the final frame image when the calculated road sign type confidence is greater than a set threshold.

[0127] The recognition module 703 is configured to recognize road sign information on the final frame image.

[0128] The time series acquisition module 704 is configured to acquire the time series of positioning information.

[0129] The acquisition time acquisition module 705 is configured to acquire the acquisition time of the final frame image.

[0130] The positioning information determination module 706 is configured to determine the positioning information of the road sign according to the matching between the acquisition time of the final frame image and the time series of the positioning information.

[0131] The marking module 707 is configured to mark the road signs with the same road sign information identified in the consecutive final frames.

[0132] The storage module 708 is configured to store the road sign information and positioning information of the road sign and the position information of the road sign in the final frame image into a road sign database.

[0133] The inspection collection module 709 is configured to collect images at the location information in the road sign database during the inspection phase.

[0134] The inspection and recognition module 710 is configured to recognize road sign information of road signs in the collected image.

[0135] The comparison module 711 is configured to compare the road sign information identified in the inspection phase with the road sign information in the road sign database.

[0136] The output module 712 is configured to output the comparison result.

[0137] The implementation principle of the above modules can be found in the relevant introduction of the traffic road sign recognition method, which will not be repeated here.

[0138] Figure 6 and Figure 7 The traffic road sign recognition devices 600 and 700 provided by the present invention are only schematically shown. Without violating the concept of the present invention, the splitting, merging and adding of modules are all within the protection scope of the present invention. The traffic road sign recognition devices 600 and 700 provided by the present invention can be implemented by software, hardware, firmware, plug-ins and any combination thereof, and the present invention is not limited thereto.

[0139] Figure 8 It is a module schematic diagram of an embodiment of the traffic road sign recognition system of the present invention.

[0140] like Figure 8 As shown, the traffic road sign recognition system includes an image acquisition module 801, a satellite positioning acquisition module 802, an identification module 804, a processing module 805, a road sign database 807, a comparison module 806, and an inspection result output module 808. Among them, the central processing unit 803 can include the identification module 804, the processing module 805 and the comparison module 806.

[0141] The image acquisition module 801 uses the vehicle-mounted image acquisition device to acquire images in front of the vehicle. The acquired images can be pre-processed, including extracting frames and saving them according to a preset time period (2-5 seconds interval can be selected), recording acquisition time point information, etc. The acquisition module output can be in the form of video or sequence images.

[0142] The recognition process method of the recognition module 804 is divided into three stages. The first stage recognizes traffic road signs on the road under normal conditions; the second stage mainly recognizes the type of traffic road signs and image features; the third stage recognizes that for road signs of scenic spots and other types, further OCR recognition can be performed to obtain data such as scenic spot information for entry into the road sign database.

[0143] The satellite positioning acquisition module 802 acquires the position and moving direction during the acquisition process.

[0144] The processing module 805 enters the prediction results and related data into the road sign database in a set format or outputs them to the comparison module.

[0145] During the inspection process, the comparison module 806 compares the field data collected by the fixed-point inspection with the data in the road sign database based on the location data collected by the existing road sign.

[0146] Inspection result output 808 outputs relevant inspection results to reporting systems.

[0147] The road sign database 807 stores information such as road sign text, positioning latitude and longitude, direction, shooting time, and images.

[0148] Figure 8 This is merely a schematic illustration of one implementation of the traffic road sign recognition system, and the present application is not limited thereto.

[0149] The embodiment of the present invention further provides a traffic road sign recognition and processing device, comprising a processor and a memory, wherein executable instructions of the processor are stored, wherein the processor is configured to execute the steps of the traffic road sign recognition method by executing the executable instructions.

[0150] As shown above, the traffic road sign recognition and processing device of this embodiment of the present invention responds to the presence of road signs in the collection field of view of the collection module, and sequentially calculates the road sign type confidence of the image collected by the collection module from the current moment, thereby selecting the final frame image according to the calculated road sign type confidence, and performing road sign information recognition on the final frame image to achieve three-stage recognition and analysis. On the one hand, there is no need to identify the road sign type and road sign information for all images. On the other hand, only the road sign information can be recognized for the final frame image with a higher confidence level. Therefore, on the premise of being able to accurately identify the road sign information, the real-time road sign image recognition system can reduce the requirement for high computing power and optimize the recognition of road signs.

[0151] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as systems, methods or program products. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits", "modules" or "platforms".

[0152] Fig. 9 This is a schematic diagram of the structure of the traffic road sign recognition and processing device of the present invention. Fig. 9 An electronic device 900 according to this embodiment of the present invention will be described. Fig. 9 The electronic device 900 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0153] like Fig. 9 As shown, the electronic device 900 is in the form of a general computing device. The components of the electronic device 900 may include, but are not limited to: at least one processing unit 910, at least one storage unit 920, a bus 930 connecting different platform components (including the storage unit 920 and the processing unit 910), a display unit 940, etc.

[0154] The storage unit stores program codes, which can be executed by the processing unit 910, so that the processing unit 910 executes the steps of various exemplary embodiments of the present invention described in the above-mentioned traffic road sign recognition method section of this specification. For example, the processing unit 910 can execute the following steps: Figure 1 Follow the steps shown in .

[0155] The storage unit 920 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 9201 and / or a cache storage unit 9202 , and may further include a read-only storage unit (ROM) 9203 .

[0156] The storage unit 920 may also include a program / utility 9204 having a set (at least one) of program modules 9205, such program modules 9205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0157] Bus 930 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0158] The electronic device 900 may also communicate with one or more external devices 9001 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 900, and / or communicate with any device that enables the electronic device 900 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 950. In addition, the electronic device 900 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 960. The network adapter 960 may communicate with other modules of the electronic device 900 via a bus 930. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, etc.

[0159] The embodiment of the present invention also provides a computer-readable storage medium for storing a program, and the steps of the traffic road sign recognition method are implemented when the program is executed. In some possible implementations, various aspects of the present invention can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of various exemplary embodiments of the present invention described in the above traffic road sign recognition method section of this specification.

[0160] As shown above, the computer-readable storage medium of this embodiment for performing traffic road sign recognition can, in response to the presence of road signs in the acquisition field of view of the acquisition module, sequentially calculate the road sign type confidence of the image acquired by the acquisition module from the current moment, thereby selecting a final frame image based on the calculated road sign type confidence, and performing road sign information recognition on the final frame image to achieve three-stage recognition and analysis. On the one hand, there is no need to identify the road sign type and road sign information for all images. On the other hand, only the road sign information can be recognized for the final frame image with a higher confidence level. This reduces the high computing power requirements of the real-time road sign image recognition system and optimizes the recognition of road signs while being able to accurately identify the road sign information.

[0161] Fig.10 Schematic diagram of the structure of the computer-readable storage medium of the present invention. Fig.10As shown, a program product 1000 for implementing the above method according to an embodiment of the present invention is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0162] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0163] Computer readable storage media may include data signals propagated in baseband or as part of a carrier wave, wherein readable program codes are carried. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program codes contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.

[0164] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0165] In summary, the present application responds to the presence of road signs in the acquisition field of view of the acquisition module, and sequentially calculates the road sign type confidence of the images acquired by the acquisition module from the current moment, thereby selecting a final frame image based on the calculated road sign type confidence, and performing road sign information recognition on the final frame image to achieve three-stage recognition and analysis. On the one hand, there is no need to identify the road sign type and road sign information for all images. On the other hand, road sign information can be recognized only for the final frame image with a higher confidence level. This reduces the requirements for high computing power of the real-time road sign image recognition system and optimizes the recognition of road signs while being able to accurately identify road sign information.

[0166] Furthermore, the present application can reduce the computing power required by the road sign recognition system under the condition of real-time collection and analysis, without the need to return to the station for retrospective analysis; reduce the misidentification and misentry in multi-sign scenarios, and provide a solution for duplicate registration of road signs; during the inspection process, reduce the computing power required by the inspection identification and comparison system, without the need to return to the station for retrospective analysis. At the same time, a method for recording and entering road sign data for multiple sign recognition in a single scenario, as well as a method for marking duplicate registration road sign information, are designed to facilitate automatic or manual processing.

[0167] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A method for identifying traffic road signs, characterized in that: include: In response to the presence of a road sign in the acquisition field of view of the acquisition module, sequentially calculating the road sign type confidence of the image acquired by the acquisition module since the current moment; Selecting a final frame image according to the calculated road sign type confidence; Performing road sign information recognition on the final frame image; storing the road sign information and the positioning information of the road sign in a road sign database; During the inspection phase, collecting images at the location information in the road sign database; Identify road sign information of road signs in the captured image; Comparing the road sign information identified in the inspection phase with the road sign information in the road sign database; Output the comparison results.

2. The method for identifying traffic road signs according to claim 1, characterized in that: The positioning information of the road sign is obtained according to the following steps: Get the time series of positioning information; Obtaining the acquisition time of the final frame image; The positioning information of the road sign is determined according to the matching of the acquisition time of the final frame image and the time series of the positioning information.

3. The method for identifying traffic road signs according to claim 1, characterized in that: The storing the road sign information and the positioning information of the road sign into the road sign database further comprises: The position information of the road sign in the final frame image is stored in a road sign database.

4. The method for identifying traffic road signs according to any one of claims 1 to 3, characterized in that: The selecting the final frame image according to the calculated road sign type confidence comprises: When the calculated road sign type confidence is greater than a set threshold, the current image is used as the final frame image.

5. The method for identifying traffic road signs according to any one of claims 1 to 3, characterized in that: After the road sign information is recognized on the final frame image, the following step is further included: The road signs with the same road sign information recognized in consecutive final frames are labeled.

6. A traffic road sign recognition device, characterized in that: include: A confidence calculation module is configured to sequentially calculate the confidence of the road sign type of the image collected by the collection module from the current moment in response to the presence of a road sign in the collection field of view of the collection module; A final frame selection module is configured to select a final frame image according to the calculated road sign type confidence; A recognition module, configured to recognize road sign information on the final frame image; A storage module, configured to store the road sign information and the positioning information of the road sign into a road sign database; An inspection collection module, configured to collect images at the location information in the road sign database during the inspection phase; An inspection and recognition module configured to recognize road sign information of road signs in the collected image; a comparison module configured to compare the road sign information identified in the inspection phase with the road sign information in the road sign database; An output module is configured to output comparison results.

7. A traffic road sign recognition and processing device, characterized in that: include: processor; a memory storing executable instructions of the processor; Wherein, the processor is configured to execute the steps of the traffic road sign recognition method according to any one of claims 1 to 5 by executing the executable instructions.

8. A computer-readable storage medium for storing a program, characterized in that: When the program is executed, the steps of the traffic road sign recognition method according to any one of claims 1 to 5 are implemented.

Images