Method, device and equipment for traffic signal lamp information identification and storage medium

Abstract

The invention relates to the technical field of traffic, and discloses a traffic signal lamp information identification method, device and equipment and a storage medium, and the method comprises the steps: building an incidence relation among a traffic signal lamp identifier, a traffic signal lamp type and a traffic control phase, and the traffic signal lamp type comprises a red lamp, a yellow lamp and a green lamp representing the color of the traffic signal lamp; acquiring a traffic signal lamp image, and based on the traffic signal lamp image, obtaining traffic signal lamp identification data, the traffic signal lamp identification data including a target color; and determining a target traffic signal lamp identifier and a target traffic control phase based on the target color and the association relationship. The traffic control phase, the traffic signal lamp identifier and the traffic signal lamp type are associated, and the target traffic signal lamp identifier and the target traffic control phase are determined based on the identified traffic signal lamp color, so that the traffic signal lamp identification response speed of the evaluation vehicle-road cooperation system can be improved; and the vehicle-road collaborative application function service quality evaluation requirement is met.

Description

Technical Field

[0001] The present invention relates to the field of transportation technologies, and in particular to a method, apparatus, device, and storage medium for traffic signal information recognition. Background Art

[0002] In the traffic light image recognition method in the related art, the left-eye image and the right-eye image of the target indicator light at the intersection are obtained to determine the first distance between the vehicle and the target indicator light; if the first distance is greater than the preset distance, the steps of obtaining the left-eye image and the right-eye image of the target indicator light at the intersection and obtaining the first distance are repeatedly executed; if the first distance is less than or equal to the preset distance, the target indicator light is recognized according to the left-eye image or the right-eye image to obtain the traffic signal of the target indicator light.

[0003] The service quality evaluation requirements for the vehicle-road collaborative application function include: evaluating the recognition accuracy and response speed of the vehicle-road collaborative system for traffic lights. Summary of the Invention

[0004] In view of this, the present invention provides a method, apparatus, device, and storage medium for traffic signal information recognition to solve the problem that the traffic light image recognition method in the related art cannot meet the service quality evaluation requirements for the vehicle-road collaborative application function.

[0005] In a first aspect, the present invention provides a method for traffic signal information recognition, the method comprising: establishing an association relationship between a traffic signal identifier, a traffic signal type, and a traffic control phase, the traffic signal type including: a red light, a yellow light, and a green light representing the color of the traffic signal; obtaining a traffic signal image, and based on the traffic signal image, obtaining traffic signal recognition data, the traffic signal recognition data including a target color; based on the target color and the association relationship, determining a target traffic signal identifier and a target traffic control phase.

[0006] In an optional implementation manner, the traffic signal type further includes a number representing the remaining traffic control time, the traffic signal recognition data further includes a target number and position data corresponding to the target number; the method further comprises: based on the target number and the position data, determining a target traffic signal identifier and the remaining traffic control time.

[0007] In an optional implementation manner, the method further comprises: obtaining high-precision time synchronization protocol timestamp information, and marking the timestamp of the traffic signal image based on the high-precision time synchronization protocol timestamp information.

[0008] In an alternative embodiment, the method further includes: labeling a number representing the remaining traffic control time with a label, where the label includes one of a red digital character, a yellow digital character, and a green digital character, and one of Arabic numeral characters; training a digital recognition model based on the number with the label to obtain a trained digital recognition model; obtaining the target number and the position data corresponding to the target number based on the trained digital recognition model and the traffic signal image; and if the number of the target numbers is multiple, determining the remaining traffic control time based on the multiple position data.

[0009] In an alternative embodiment, the method further includes: determining a coordinate region corresponding to the traffic signal based on the traffic signal image; extracting features from the image within the coordinate region, and classifying based on the extracted features to obtain the traffic signal recognition data.

[0010] In a second aspect, the present invention provides a device for traffic signal information recognition, where the device includes: a portable rack assembly, and a rear-end assembly disposed on the portable rack assembly; the rear-end assembly is configured to establish an association relationship between a traffic signal identifier, a traffic signal type, and a traffic control phase, where the traffic signal type includes: a red light, a yellow light, and a green light representing the colors of the traffic signal; obtaining a traffic signal image, and obtaining traffic signal recognition data based on the traffic signal image, where the traffic signal recognition data includes a target color; and determining a target traffic signal identifier and a target traffic control phase based on the target color and the association relationship.

[0011] In an alternative embodiment, the device further includes: an image acquisition component disposed on the portable rack assembly; the image acquisition component is communicatively connected to the rear-end assembly, and the image acquisition component is configured to acquire the traffic signal image, mark a timestamp of the traffic signal image based on the high-precision time synchronization protocol timestamp information, and send the timestamp mark and the traffic signal image to the rear-end assembly; an image time synchronization component, communicatively connected to the image acquisition component, and the image time synchronization component is configured to obtain the high-precision time synchronization protocol timestamp information and send the high-precision time synchronization protocol timestamp information to the image acquisition component.

[0012] In an alternative embodiment, the device further includes: a front-end component, communicatively connected to the rear-end component, and configured to provide a display interface for the rear-end component; the front-end component is also communicatively connected to the image acquisition component, and the front-end component is further configured to set the acquisition parameters of the image acquisition component.

[0013] In a third aspect, the present invention provides a computer device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to execute the method for traffic signal information recognition according to the first aspect or any corresponding embodiment thereof.

[0014] In a fourth aspect, the present invention provides a computer-readable storage medium, on which computer instructions are stored. The computer instructions are used to cause a computer to execute the method for traffic signal information recognition according to the first aspect or any corresponding embodiment thereof.

[0015] In a fifth aspect, the present invention provides a computer program product, including computer instructions, which are used to cause a computer to execute the method for traffic signal information recognition according to the first aspect or any corresponding embodiment thereof.

[0016] Associating the traffic control phase, traffic signal identifier, and traffic signal type, and determining the target traffic signal identifier and target traffic control phase based on the recognized traffic signal color can improve the recognition and response speed of the vehicle-road cooperation system to traffic signals and meet the evaluation requirements for the quality of service of vehicle-road cooperation application functions. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the related art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1 It shows a schematic flowchart of the method for traffic signal information recognition provided by an embodiment of the present invention;

[0019] Figure 2 It shows another schematic flowchart of the method for traffic signal information recognition provided by an embodiment of the present invention;

[0020] Figure 3 It shows a schematic structural diagram of the device for traffic signal information recognition provided by an embodiment of the present invention;

[0021] Figure 4 It shows another schematic structural diagram of the device for traffic signal information recognition provided by an embodiment of the present invention;

[0022] Figure 5It shows a schematic flowchart of information interaction among components in the device for traffic signal light information recognition provided by an embodiment of the present invention;

[0023] Figure 6 It shows another schematic structural diagram of the device for traffic signal light information recognition provided by an embodiment of the present invention;

[0024] Figure 7 It is a schematic hardware structure diagram of a computer device according to an embodiment of the present invention. Specific embodiments

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts fall within the scope of protection of the present invention.

[0026] The traffic light recognition system based on the target detection network in the related art: Obtain the left-eye image and right-eye image of the target indicator light at the intersection to determine the first distance between the vehicle and the target indicator light; if the first distance is greater than the preset distance, repeat the steps of obtaining the left-eye image and right-eye image of the target indicator light at the intersection and obtaining the first distance; if the first distance is less than or equal to the preset distance, identify the target indicator light according to the left-eye image or right-eye image to obtain the traffic signal of the target indicator light. Using the above method can not only improve the accuracy of traffic signal recognition of traffic lights at intersections, but also reduce the computing resources required.

[0027] The device for traffic light perception for autonomous vehicles in the related art: Utilize the characteristics of the downsampling ratio of the backbone in the network structure and the actual size of traffic lights in the image to improve the recognition accuracy of traffic lights in the distance in the image.

[0028] The system for annotating the position and category of traffic lights in the related art: Increase the number of small targets by copying the traffic light position area, so that the number of matching traffic lights also increases, thereby improving the detection rate of traffic lights.

[0029] The traffic light recognition method in the related art includes: Obtain an image containing traffic light information and select the area where the traffic light is located in the image, and then perform color correction on the traffic light; Obtain the position and category of the traffic light according to the corrected traffic light information. Thereby improving the detection efficiency and recognition accuracy.

[0030] In the related art of testing the perception ability of traffic lights, the process of collecting traffic light data is implemented by a detection system built with high-performance fixed or mobile devices, and there are the following deficiencies:

[0031] First, the traffic light recognition tools in the related art are mainly used in the field of autonomous driving of bicycles based on image processing technology, and are not applied to the traffic light recognition work related to the evaluation of the quality of service of road collaborative application functions.

[0032] Second, there is a possibility that the technical solutions in the related art are not related to traffic signal phases. This possibility will lead to chaos in the automatic output process of data information such as phases, light colors, and countdown numbers obtained by traffic light recognition, and difficulties in subsequent traffic light recognition data management.

[0033] According to an embodiment of the present invention, there is provided a method embodiment for traffic signal information recognition. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0034] In this embodiment, there is provided a method for traffic signal information recognition, which can be used in a mobile terminal, such as a mobile phone, a tablet computer, or a computing board, etc. Figure 1 The flowchart of the method for traffic signal information recognition provided by the embodiment of the present invention is shown, as Figure 1 shown, the process includes the following steps:

[0035] Step S101, establish the association relationship between the traffic signal identifier, the traffic signal type, and the traffic control phase. The traffic signal type includes: red light, yellow light, and green light representing the color of the traffic signal.

[0036] In this step, the association relationship between the traffic signal identifier, the traffic signal type, and the traffic control phase is shown in Table 1. The identifiers of multiple lights in the traffic signals in the same direction can be set.

[0037] Table 1 Association relationship between traffic signal identifier, traffic signal type, and traffic control phase

[0038] Traffic signal identifier Traffic signal type Traffic control phase 1 Red light 1 2 Yellow light 2 3 Green light 3 4 Digit 5 Digit

[0039] The traffic control phase refers to the time period in the traffic signal control system used to indicate the different driving directions and sequences of traffic flows at intersections. Each control phase represents a specific traffic flow, such as going straight, turning left, or turning right, and each control phase has its specific signal light color and duration.

[0040] Step S102: Obtain a traffic signal light image. Based on the traffic signal light image, obtain traffic signal light recognition data, where the traffic signal light recognition data includes the target color.

[0041] In this step, obtaining the traffic signal light image includes: in response to a test instruction issued by a client set in the front-end component, collecting the phase change optical signal of the traffic signal light through an optical device, and the optical device can be a camera. The collected traffic signal light image includes traffic signal light information and surrounding scene information, and the time stamp obtained from the camera can be associated with the traffic signal light image. The color of the traffic signal light can be recognized through a deep learning algorithm, and the traffic signal light recognition data can be saved.

[0042] During the process of recognizing the traffic signal light image, the color state information of the traffic signal light can be analyzed. The color state information includes: traffic signal light color data, traffic signal light pixel position, and traffic signal light identifier. Among them, the traffic signal light color data includes information such as red, yellow, green, and black.

[0043] The traffic signal light recognition process based on the traffic signal light image includes: performing image training on the traffic signal light cylinder shape label, which can prepare for the recognition algorithm area obtained later, adding labels of red, yellow, green, and black to the traffic signal light for training the color recognition model, and combining the traffic signal light image to achieve the recognition of the traffic signal light color.

[0044] Step S103: Based on the target color and the association relationship, determine the target traffic signal light identifier and the target traffic control phase.

[0045] In this step, after obtaining the target color by recognizing the traffic signal light image, based on the target color and the association relationship shown in Table 1, determine the traffic signal light type, determine the traffic signal light identifier based on the traffic signal light type, and then determine the corresponding traffic control phase.

[0046] After obtaining the traffic signal light recognition data, the structured data can be exported in the form of a table. Through the association of the traffic signal light identifier, traffic signal light color, and traffic control phase, it is possible to prevent the traffic signal light identifier from being updated due to the temporary occlusion of the traffic signal light image.

[0047] The method for traffic signal light information recognition provided in this embodiment associates the traffic control phase with the traffic signal light identifier and traffic signal light type, and determines the target traffic signal light identifier and target traffic control phase based on the recognized traffic signal light color, which can improve the recognition response speed of the evaluation vehicle-road collaborative system and meet the evaluation requirements for the service quality of vehicle-road collaborative application functions.

[0048] In some alternative embodiments, the traffic signal type further includes a number representing the remaining traffic control time, and the traffic signal recognition data further includes a target number and position data corresponding to the target number; the aforementioned method for traffic signal information recognition further includes: determining a target traffic signal identifier and the remaining traffic control time based on the target number and the position data.

[0049] In this embodiment, in response to a test instruction issued by the client, a traffic signal image is collected by an optical device, the target number of the traffic signal can be recognized through a deep learning algorithm, and the traffic signal recognition data is saved. In this way, an accurate comparison of the time stamps of the traffic signal recognition data and the Road Side Unit (RSU) broadcast message can be achieved with a relatively low time delay in the millisecond level, which can lay a foundation for evaluating the accuracy of the RSU broadcast traffic signal message.

[0050] In this way, by associating the traffic control phase, the traffic signal identifier, and the traffic signal type, and determining the target traffic signal identifier and the remaining traffic control duration based on the recognized traffic signal number, the recognition response speed of the vehicle-road coordination system to traffic signals can be improved, and the requirements for evaluating the service quality of vehicle-road coordination application functions can be met.

[0051] The working mode of the traffic light recognition device in some related technologies lacks the recognition function for real-time video, which has an adverse impact on the later use and maintenance of the tool, the verification of algorithm performance, and limits the ability of the system to expand and upgrade functions.

[0052] In some alternative embodiments, the recognition process of the traffic signal is guided by a test instruction, and the test instruction is issued by the client to the test device terminal. The recognition process includes an online recognition working mode and an offline recognition working mode. Among them, in the online recognition working mode, an image processing algorithm runs, structured data is output, and at the same time, videos and pictures are stored; in the offline recognition working mode, videos are recorded and stored through an image acquisition component for possible later system debugging, image recognition algorithm iteration, and later structured data sorting and other requirements, without the need for a machine recognition algorithm to run and without structured data output.

[0053] By designing two working modes of video recording and image recognition, the real-time road conditions during the test can be recorded, which is convenient for the later improvement and testing of traffic signal recognition tools, can improve the efficiency of project management, and is conducive to the rapid iterative upgrade of the vehicle-road coordination function service system.

[0054] The time synchronization accuracy of traffic light recognition devices and systems in some related technologies is low, and it is impossible to achieve a millisecond-level image acquisition time delay. There is a large error between the time stamps recorded by the devices and the actual event occurrence time, which leads to poor accuracy of data acquisition and makes it impossible to achieve millisecond-level time synchronization with signal receiving devices that send traffic light messages from RSU.

[0055] In some alternative embodiments, the aforementioned method for traffic signal information recognition further includes: obtaining high-precision time synchronization protocol (Precision Time Protocol, abbreviated as PTP) time stamp information, and marking the time stamp of the traffic light image based on the high-precision time synchronization protocol time stamp information.

[0056] In this embodiment, Figure 2 Another flowchart of the method for traffic signal information recognition is shown, as Figure 2 shown, step S201, marking the time stamp of the traffic light image based on the high-precision time synchronization protocol time stamp information; step S202, collecting the traffic light image; step S203, performing image recognition based on the collected traffic light image to obtain traffic light recognition data, and proceeding to step S205; step S204, establishing the association relationship between the traffic light identifier, traffic light type and traffic control phase, and proceeding to step S205; step S205, determining the target traffic light identifier, remaining traffic control time and target traffic control phase based on the traffic light recognition data and the association relationship, and storing the target traffic light identifier, remaining traffic control time and target traffic control phase in a structured manner.

[0057] In step S201, the time stamp of the acquisition picture moment can be marked based on the image time service component, and the Global Positioning System (GPS) time can be used. It is an atomic time reference composed of a GPS spaceborne atomic clock and a ground monitoring station atomic clock, which ensures the data quality of the time stamp obtained by the method of the present invention.

[0058] The traffic light recognition data can be stored in the format of time stamp data, traffic light color data, remaining traffic control time data and traffic control phase data. It can be sorted in ascending order with the GPS time stamp as the index, which can support the evaluation of the fusion perception ability of the edge cloud system of the cloud control basic platform from the dimensions of basic performance, traffic participant perception, traffic event perception and traffic flow perception.

[0059] In this way, based on the image timing component and the PTP time synchronization protocol, sub-microsecond time synchronization of the traffic light recognition data and the RSU broadcast traffic light message can be achieved, which can ensure that the time error between the traffic light recognition data and the roadside broadcast message is within an acceptable range; at the same time, the timestamp format can be standardized to prevent the timestamp error between the RSU broadcast message data and the traffic light recognition data from being unable to remain within the test requirements at a higher image output frequency of more than 50 frame rates, thereby improving the consistency and accuracy of the data; in addition, it can also make the decisions based on data analysis more scientific and reliable, which is of great significance for verifying the accuracy of the measured data (i.e., the RSU broadcast message) through the traffic light recognition data.

[0060] In some optional embodiments, the aforementioned method for identifying traffic light information further includes: marking a digital label to represent the remaining traffic control time, the label including: one of red digital characters, yellow digital characters, green digital characters, and one of Arabic numerals; training a digital recognition model based on the numbers in the marked labels to obtain a trained digital recognition model; obtaining a target number and position data corresponding to the target number based on the trained digital recognition model and the traffic light image; if there are multiple target numbers, determining the remaining traffic control time based on multiple position data.

[0061] In this embodiment, the digital label representing the remaining traffic control time specifically includes: red countdown digits, yellow countdown digits, green countdown digits, and Arabic numerals 0 to 9. By setting labels for the above information, marking and training, and then judging and combining the relative positions of the Arabic numerals, the countdown digits of the traffic light, which are actually two-digit information, are obtained, and finally the image recognition of the countdown information is realized.

[0062] In this way, compared with the method of setting labels and training for the shape of the light pole of the digital information representing the remaining traffic control time, the problem that the shape of the light pole of the digital information representing the remaining traffic control time has various shapes such as cylindrical and square can be solved. Traffic lights representing the digital information of the remaining traffic control time with large differences in shape can be identified at the same time, thereby improving the success rate of image recognition.

[0063] In some optional embodiments, the aforementioned method for traffic light information recognition also includes: determining the coordinate area corresponding to the traffic light based on the traffic light image; performing feature extraction based on the image within the coordinate area, and performing classification based on the extracted features to obtain traffic light recognition data.

[0064] In this embodiment, first, based on the image timing component, the timestamp for collecting the traffic signal light image is determined, and image collection is performed. Secondly, during the recognition process of the traffic signal light image, based on the collected traffic signal light image, the pixel coordinate area corresponding to the traffic signal light is calculated, and a Region of Interest (ROI) image is obtained, which can be achieved by methods such as a sliding window, image segmentation, or a selection tool. Within the ROI, features such as the color, brightness, or shape of the traffic signal light are extracted, and a classification algorithm is used to obtain the color state of the traffic signal light. The classification algorithm can specifically select a support vector machine or a convolutional neural network, etc., which can lay a foundation for subsequent state feedback. Finally, during the structured data storage process, the shooting timestamp of the current image, the color of the traffic signal light in the image, the countdown, etc. are fed back to the terminal receiving the data through state feedback, realizing the image recognition of the traffic signal light color and the countdown number of the traffic signal light, and the recognition data is saved through the structured data storage module.

[0065] The traffic light recognition device in the related art does not consider the portability design, which limits the on-site operation ability of the recognition device. The staff cannot quickly deploy the device, resulting in low work efficiency, and thus cannot quickly obtain the signal data of the current traffic signal light; it limits the operation position of the device. The fixed device may require a large space and specific environmental conditions, which is not conducive to flexible operation and movement.

[0066] In this embodiment, a device for traffic signal light information recognition is also provided. This device is used to implement the above-mentioned embodiments and preferred implementation manners, and those that have been described will not be repeated. As used hereinafter, the term "module" can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0067] This embodiment provides a device for traffic signal light information recognition. Figure 3 The structural schematic diagram of the device for traffic signal light information recognition provided by the embodiment of the present invention is shown. As Figure 3 shown, it includes:

[0068] A portable rack assembly 301 and a rear-end assembly 302 disposed on the portable rack assembly 301. The portable rack assembly 301 includes a space for accommodating an optical lens and an industrial area array camera, and a space for accommodating a computer algorithm processor. The rack interface supports Power over Ethernet (PoE) for the camera, direct current (DC) power supply for the computer algorithm processor, and an Ethernet data transmission interface.

[0069] In this way, based on the portable rack assembly, it is convenient for testers to directly carry this device to the work site for traffic signal recognition, expanding the scope of test services. For small-scale projects or short-term tasks, it can more effectively save costs and resources.

[0070] The backend component 302 is used to establish the association relationship between the traffic signal identifier, the traffic signal type, and the traffic control phase. The traffic signal types include: red light, yellow light, and green light representing the colors of traffic signals; obtain the traffic signal image, and based on the traffic signal image, obtain the traffic signal recognition data, where the traffic signal recognition data includes the target color; based on the target color and the association relationship, determine the target traffic signal identifier and the target traffic control phase.

[0071] The backend component 302 can be an image recognition component or a computer vision algorithm box, including: a processor, a cache, an operating system power interface, and a data interface, which can implement the output task of the image recognition result. After receiving the test instruction issued by the client, it establishes a physical data channel with the camera and the front-end component, saves the traffic signal recognition data, sends the traffic signal recognition data to the front-end component, outputs the traffic signal recognition result, and finally guides the completion of the test task by collecting traffic signal test data with timestamp information and RSU broadcast messages.

[0072] The traffic light recognition tools in the related technologies have relatively single functions and cannot realize on-site collection and output of high-quality image recognition results, which hinders the data post-management work of engineering and technical personnel.

[0073] Figure 4 Another structural schematic diagram of the device for traffic signal information recognition is shown, as Figure 4 shown. In some alternative embodiments, the aforementioned device for traffic signal information recognition further includes:

[0074] An image acquisition component 303 disposed on the portable rack assembly 301;

[0075] The image acquisition component 303 is communicatively connected to the backend component 302. The image acquisition component 303 is used to acquire the traffic signal image, mark the timestamp of the traffic signal image based on the timestamp information of the high-precision time synchronization protocol, and send the timestamp mark and the traffic signal image to the backend component 302.

[0076] The image time synchronization component 304 is communicatively connected to the image acquisition component 303. The image time synchronization component 304 is used to obtain the timestamp information of the high-precision time synchronization protocol and send the timestamp information of the high-precision time synchronization protocol to the image acquisition component 303.

[0077] The camera SDK can be triggered by hardware, and the GPS timestamp can be obtained from the image time synchronization component 304 in combination with the PTP protocol. The GPS timestamp has a time error at the millisecond level. Then, the traffic signal light image is saved to complete the image acquisition work.

[0078] The image acquisition component 303 includes: an optical lens and an industrial area array camera. Among them, the optical lens supports the adjustment of the light input amount and the focal length; the camera pixel is 2 million, supports the exposure time adjustment function, and can use a gigabit Ethernet interface.

[0079] Based on the image acquisition component 303, the original image information collected is marked with a timestamp by the image time synchronization component 304, and the obtained traffic signal light image is uploaded to a personal computer (PC for short) to support the real-time display of content such as image recognition status and picture stream push, and to perform further processes such as perception data conversion, timestamp matching, result visualization, and structured data storage.

[0080] The focal length of the lens of the industrial camera in the image acquisition component can be manually adjusted through the portable rack component to ensure the picture clarity. In this way, the quality of the collected traffic signal light images is relatively high and occupies less memory, which can further support computer vision algorithms to achieve accurate recognition of traffic signal light information. At the same time, the traffic signal light recognition device built by combining the image acquisition component and the backend component can accurately perform work such as image acquisition, image processing, and structured data output at the test site.

[0081] Most of the traffic signal light recognition devices in the related technologies do not have customized function designs that meet the business requirements of the client, resulting in problems such as restricting the positive interaction or monitoring between users and products, being difficult to perform remote configuration and update, and increasing the maintenance and management costs of the hardware.

[0082] In some optional embodiments, the aforementioned device for traffic signal light information recognition further includes:

[0083] A front-end component 305, which is communicatively connected to the image acquisition component 303 and is also communicatively connected to the backend component 302, is used to provide a display interface for the backend component 302 and is also used to set the acquisition parameters of the image acquisition component 303.

[0084] A communication component 306, which is used to provide the aforementioned communication connection.

[0085] The front - end component 305 can be a client in a personal computer, providing a service display and setting interface for the back - end component 302 for users, including functions such as real - time video display; setting of camera exposure time, frame rate, and resolution; data export of image recognition algorithms; association of traffic signal identifiers, traffic signal countdown numbers, and traffic control phases, as well as input of the association relationship between traffic signal identifiers and physical numbers.

[0086] Based on the front - end component 305 to set up the client, customized test services can be realized, which can meet specific test requirements, provide more accurate and effective solutions for customers, and provide real - time monitoring and feedback functions for testers to help them understand the current working status and performance of test equipment, etc. At the same time, by designing the function of customizing image resolution in the client software, the traffic signal pictures collected are ensured to have clear images and do not lose the key traffic signal details required for image recognition, so that the algorithm will not have incorrect recognition of traffic signal colors and other information and incorrect result export.

[0087] The communication component 306 supports the Ethernet communication mode and can realize communication in the client - server mode based on the front - end component and the back - end component. Furthermore, it enables the camera, the front - end component, and the back - end component to establish connections and provides sufficient bandwidth for test data upload and test instruction issuance on the premise of low latency.

[0088] The further function descriptions of the above - mentioned various modules and units are the same as those in the corresponding above - mentioned embodiments and will not be elaborated here.

[0089] The device for traffic signal information recognition in this embodiment is presented in the form of functional units. Here, the unit refers to an Application Specific Integrated Circuit (ASIC) circuit, a processor and a memory that execute one or more software or fixed programs, and / or other devices that can provide the above - mentioned functions.

[0090] The device for traffic signal information recognition provided by the embodiment of the present invention is portable, easy to operate, has strong image recognition accuracy, high real - time performance and can meet the requirements of traffic signal recognition services. It can provide structured data related to traffic signal states in real time for the test of traffic signal messages broadcast by RSU, supporting the high - quality output of test evaluation indicators.

[0091] Figure 5 It shows a schematic diagram of the information interaction process among the components in the device for traffic signal information recognition provided by the embodiment of the present invention. As Figure 5 shown, the information interaction process among the components in the device for traffic signal information recognition includes:

[0092] Step S501: The image timing component 304 sends timestamp information to the image acquisition component 303 based on the PTP protocol;

[0093] Step S502: Based on the front-end component 305, set the acquisition parameters of the image acquisition component 303 to ensure that the image frame rate during the traffic signal image acquisition meets the test requirements;

[0094] Step S503: Based on the front-end component 305, determine the association relationship between the traffic signal identifier and the traffic control phase, which can ensure the accurate correspondence between the subsequent traffic control phase and the traffic signal status information, and can export the traffic signal association relationship in the form of structured data;

[0095] Step S504: Based on the front-end component 305, enable the traffic signal image acquisition function and prepare for image acquisition;

[0096] Step S505: The image acquisition component 303 can acquire images of the traffic signal 51 at a frame rate higher than 50.

[0097] Step S506: Based on the front-end component 305, receive the GPS timestamp and accurately correspond it to the traffic signal image at the corresponding moment;

[0098] Step S507: Based on the front-end component 305, receive the traffic signal image acquired by the image acquisition component 303 and mark the corresponding timestamp;

[0099] Step 508: Based on the front-end component 305, enable the traffic signal recognition function and prepare for traffic signal image recognition.

[0100] Step S509: Based on the front-end component 305, push the picture stream to the back-end component 302 and send an HTTP request; based on the front-end component 305, transmit the traffic signal image to the back-end component 302 in the way of automatically pushing the data stream;

[0101] Step S510: The back-end component 302 returns the image recognition result including the light color and countdown information to the front-end component 305 based on the HTTP protocol. Among them, the back-end component 302 uses deep learning algorithms to implement traffic signal position recognition, color recognition, countdown digit recognition, and traffic signal identifier allocation, etc. Traffic signal position recognition is achieved through image recognition technologies such as color segmentation, region of interest extraction, signal light region determination and recognition. After obtaining the traffic signal recognition data, based on the HTTP protocol, push the recognition result to the front-end component 305 at a certain time interval.

[0102] In addition, the user can set the association relationship of traffic control phase numbers in the front-end component 305 client interface, visualize the recognition results of traffic signal colors, the recognition results of countdowns, and the tracking of traffic signal identifiers, and save the output structured data.

[0103] Figure 6 Another structural schematic diagram of the device for traffic signal information recognition provided by an embodiment of the present invention is shown. As Figure 6 shown, the device for traffic signal information recognition includes: an image timing module 601, an image acquisition module 602, an image recognition module 603, and a structured data storage module 604. Among them, through the image timing module 601, the GPS time is unified, and millisecond-level time synchronization is achieved based on the PTP protocol; through the image acquisition module 602, high-frame-rate traffic signal images are acquired, and the custom image acquisition resolution is set; through the image recognition module 603, online traffic signal image recognition or offline traffic signal image recognition is performed; based on the structured data storage module 604, the image acquisition timestamp information, the traffic signal color information obtained by recognition, the countdown information obtained by traffic signal recognition, and the target traffic control phase information obtained by recognition are stored.

[0104] The image timing module 601, implemented based on the PTP protocol, can provide low-error timing services for traffic signal recognition and the measured system (roadside facilities), and mark timestamps for structured data, ensuring the accuracy of subsequent verification with the measured data timestamps.

[0105] The image acquisition module 602 obtains high-quality pictures for available image recognition based on an optical lens and an industrial camera.

[0106] The image recognition module 603, designed based on deep learning algorithms combined with image recognition logic, includes region of interest extraction, label design, etc., and can achieve efficient and accurate image recognition.

[0107] The structured data storage module 604 can arrange the data obtained through image recognition according to the GPS timestamps of the measured data and store them classified. At the same time, timestamps can also be marked for the video during the acquisition process and compared with the RSU broadcast message. Among them, the RSU broadcast message needs to ensure that the time zone is correct, the data format meets the communication protocol requirements, and the time accuracy meets the requirements of the test system for the timestamps obtained based on the PTP protocol, so as to analyze and verify the timeliness of the message subsequently, and verify the accuracy of the broadcast message sent by the roadside device in combination with this device.

[0108] An embodiment of the present invention also provides a computer device having the above Figure 3The device for traffic signal information recognition shown in the figure.

[0109] Please refer to Figure 7 , Figure 7 which is a schematic structural diagram of a computer device provided by an optional embodiment of the present invention. As Figure 7 shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including a high-speed interface and a low-speed interface. Each component communicates with each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed within the computer device, including instructions stored in the memory or on the memory to display graphic information of a graphical user interface on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a set of blade servers, or a multi-processor system). Figure 7 In

[0110]

[0111]

[0112]

[0113] The memory 20 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some optional embodiments, the memory 20 may optionally include a memory remotely set relative to the processor 10, and these remote memories can be connected to the computer device through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and their combinations.

[0113] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid state drive; the memory 20 may further include a combination of the above types of memories.

[0114] The computer device further includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected through a bus or other means. Figure 7 Taking connection through a bus as an example.

[0115] The input device 30 can receive input digital or character information, and generate key signal inputs related to the user settings and function controls of the computer device, such as touch screen, keypad, mouse, trackpad, touchpad, pointing stick, one or more mouse buttons, trackball, joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (such as a light emitting diode), and a tactile feedback device (such as a vibration motor), etc. The above display device includes but is not limited to liquid crystal display, light emitting diode, display and plasma display. In some alternative embodiments, the display device may be a touch screen.

[0116] The embodiments of the present invention also provide a computer-readable storage medium. The methods according to the embodiments of the present invention can be implemented in hardware, firmware, or be implemented as computer code that can be recorded on a storage medium, or be implemented by downloading through a network the original computer code stored in a remote storage medium or a non-transitory machine-readable storage medium and to be stored in a local storage medium, so that the methods described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium may be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk or a solid state drive, etc.; further, the storage medium may further include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code, and when the software or computer code is accessed and executed by the computer, the processor or the hardware, the methods shown in the above embodiments are implemented.

[0117] A part of the present invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can call or provide the methods and / or technical solutions according to the present invention through the operations of the computer. Those skilled in the art should understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executes the instructions, or the computer compiles the instructions and then executes the corresponding compiled program, or the computer reads and executes the instructions, or the computer reads and installs the instructions and then executes the corresponding installed program. Herein, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to the computer.

[0118] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for identifying traffic light information, characterized in that: The method comprises: Establishing an association relationship between a traffic light identifier, a traffic light type and a traffic control phase, wherein the traffic light type includes: a red light, a yellow light and a green light representing the color of the traffic light; Acquire a traffic light image, and obtain traffic light recognition data based on the traffic light image, wherein the traffic light recognition data includes a target color; Based on the target color and the association relationship, a target traffic light identifier and a target traffic control phase are determined.

2. The method according to claim 1, characterized in that The traffic light type further includes a number representing the remaining traffic control time, and the traffic light identification data further includes a target number and position data corresponding to the target number; The method further comprises: Based on the target number and the location data, a target traffic light identifier and the remaining traffic control time are determined.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: High-precision time synchronization protocol timestamp information is acquired, and a timestamp of the traffic light image is marked based on the high-precision time synchronization protocol timestamp information.

4. The method according to claim 2, characterized in that: The method further comprises: The digital label representing the remaining traffic control time includes: one of red digital characters, yellow digital characters, green digital characters, and one of Arabic digital characters; Training a digital recognition model based on the numbers annotated with the labels to obtain a trained digital recognition model; Based on the trained digit recognition model and the traffic light image, obtaining the target digit and position data corresponding to the target digit; If there are multiple target numbers, the remaining traffic control time is determined based on multiple location data.

5. The method according to claim 1, characterized in that The method further comprises: Based on the traffic light image, determining a coordinate area corresponding to the traffic light; Feature extraction is performed based on the image in the coordinate area, and classification is performed based on the extracted features to obtain the traffic light recognition data.

6. A device for identifying traffic light information, characterized in that: The device comprises: A portable rack assembly, and a rear end assembly disposed on the portable rack assembly; The back-end component is used to establish an association relationship between a traffic light identifier, a traffic light type and a traffic control phase, wherein the traffic light type includes: red light, yellow light and green light representing the color of the traffic light; obtain a traffic light image, and based on the traffic light image, obtain traffic light recognition data, wherein the traffic light recognition data includes a target color; based on the target color and the association relationship, determine a target traffic light identifier and a target traffic control phase.

7. The device according to claim 6, characterized in that The device also includes: An image acquisition component disposed on the portable frame component; The image acquisition component is communicatively connected with the back-end component, and the image acquisition component is used to acquire the traffic light image; An image timing component, which is in communication connection with the image acquisition component, and is used to obtain high-precision time synchronization protocol timestamp information and send the high-precision time synchronization protocol timestamp information to the image acquisition component; The image acquisition component is further used to mark the timestamp of the traffic light image based on the timestamp information of the high-precision time synchronization protocol, and send the timestamp mark and the traffic light image to the back-end component.

8. The device according to claim 7, characterized in that The device also includes: A front-end component, which is in communication with the back-end component and is used to provide a display interface for the back-end component; The front-end component is also in communication connection with the image acquisition component, and the front-end component is also used to set acquisition parameters of the image acquisition component.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for traffic light information recognition according to any one of claims 1 to 5 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method for traffic light information recognition according to any one of claims 1 to 5.

Images