Traffic condition identification method and device, electronic equipment, vehicle and storage medium

Abstract

The invention discloses a traffic condition recognition method and device, electronic equipment, a vehicle and a storage medium, and relates to the technical field of image recognition, and the method comprises the steps: obtaining a road image; identifying and marking the position and the shape of a zebra crossing in the road image by using a pre-trained CNN model, and displaying the road image marked with the position and the shape of the zebra crossing on a display terminal of the vehicle; identifying and marking the position and the state of the traffic light in the road image by using a pre-trained YOLO model, and displaying the road image marked with the position and the state of the traffic light on a display terminal of the vehicle; and identifying and marking the position of the pedestrian in the road image by using a pre-trained SSD model, and displaying the road image marked with the position of the pedestrian on a display terminal of the vehicle. Road images are specifically recognized through the trained models, different traffic conditions are obtained, the traffic conditions can be displayed on the display terminal on the vehicle, and a driver is reminded to pay attention to driving safety.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a traffic condition recognition method, device, electronic equipment, vehicle and storage medium. Background Art

[0002] As the number of vehicles on the road increases, pedestrians and vehicles will face more uncertainty and risks, affecting traffic efficiency and safety. It is difficult for relevant technologies to efficiently and accurately identify traffic conditions, making it difficult to achieve efficient and safe road traffic, especially in the complex traffic environment of modern cities, where the above problems are more prominent. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to propose a traffic condition recognition method, device, electronic device, vehicle and storage medium to accurately identify traffic conditions and thereby improve road safety.

[0004] To achieve the above objective, an embodiment of the present application provides a method for identifying traffic conditions, the method comprising the following steps:

[0005] Acquire road images;

[0006] Using a pre-trained CNN model to identify and annotate the position and shape of the zebra crossing in the road image, and displaying the road image annotated with the position and shape of the zebra crossing on a display terminal of the vehicle;

[0007] Using a pre-trained YOLO model to identify and annotate the position and status of traffic lights in the road image, and displaying the road image annotated with the position and status of traffic lights on the display terminal of the vehicle;

[0008] The positions of pedestrians in the road image are identified and annotated using a pre-trained SSD model, and the road image annotated with the positions of pedestrians is displayed on the display terminal of the vehicle.

[0009] In some embodiments, the acquiring of the road image comprises the following steps:

[0010] Acquiring the road image by a camera disposed on the vehicle;

[0011] Alternatively, the road image is acquired by a camera arranged on the road.

[0012] In some embodiments, before using the pre-trained CNN model to identify and mark the position and shape of the zebra crossing in the road image, the method further includes the step of training the CNN model, and the step of training the CNN model includes:

[0013] Acquire a first road image;

[0014] Performing grayscale processing and normalization processing on the first road image according to the input requirement of the CNN model to obtain the preprocessed first road image;

[0015] Annotating the position and shape of the zebra crossing on the preprocessed first road image as first training data;

[0016] The CNN model is trained using the first training data to obtain the pre-trained CNN model.

[0017] In some embodiments, before using the pre-trained YOLO model to identify and annotate the position and state of the traffic light in the road image, the method further includes the step of training the YOLO model, and the step of training the YOLO model includes:

[0018] acquiring a second road image;

[0019] Adjusting the size of the second road image and performing color space conversion according to the input requirement of the YOLO model to obtain a preprocessed second road image;

[0020] annotating the position and state of the traffic light on the preprocessed second road image as second training data;

[0021] The YOLO model is trained using the second training data to obtain the pre-trained YOLO model.

[0022] In some embodiments, before using the pre-trained SSD model to identify and mark the position of the pedestrian in the road image, the method further includes the step of training the SSD model, and the step of training the SSD model includes:

[0023] acquiring a third road image;

[0024] According to the input requirement of the SSD model, the size of the third road image is adjusted and the color space conversion is performed to obtain the preprocessed third road image;

[0025] annotating the positions of pedestrians on the preprocessed third road image as third training data;

[0026] The SSD model is trained using the third training data to obtain the pre-trained SSD model.

[0027] To achieve the above-mentioned purpose, another aspect of the embodiment of the present application provides a traffic condition recognition device, the device comprising:

[0028] An image acquisition unit, used for acquiring a road image;

[0029] a zebra crossing recognition and display unit, used to recognize and mark the position and shape of the zebra crossing in the road image using a pre-trained CNN model, and display the road image with the position and shape of the zebra crossing marked on a display terminal of the vehicle;

[0030] a traffic light recognition and display unit, configured to recognize and annotate the position and status of traffic lights in the road image using a pre-trained YOLO model, and display the road image annotated with the position and status of traffic lights on the display terminal of the vehicle;

[0031] A pedestrian recognition and display unit is used to recognize and mark the positions of pedestrians in the road image using a pre-trained SSD model, and display the road image with the positions of pedestrians marked on the display terminal of the vehicle.

[0032] In some embodiments, the image acquisition unit includes:

[0033] A vehicle-mounted camera image acquisition unit, used for acquiring the road image through a camera arranged on the vehicle;

[0034] The road camera image acquisition unit is used to acquire the road image through a camera arranged on the road.

[0035] In some embodiments, the apparatus further comprises a unit for training the CNN model before identifying and annotating the position and shape of the zebra crossing in the road image using the pre-trained CNN model, and the unit for training the CNN model comprises:

[0036] A first image acquisition subunit, used to acquire a first road image;

[0037] A first preprocessing unit, configured to perform grayscale processing and normalization processing on the first road image according to an input requirement of the CNN model to obtain the preprocessed first road image;

[0038] a zebra crossing marking unit, configured to mark the position and shape of the zebra crossing on the preprocessed first road image as first training data;

[0039] The first model training unit is used to train the CNN model using the first training data to obtain the pre-trained CNN model.

[0040] In some embodiments, the apparatus further comprises a unit for training the YOLO model before using the pre-trained YOLO model to identify and annotate the position and state of the traffic light in the road image, the unit for training the YOLO model comprising:

[0041] A second image acquisition subunit, used to acquire a second road image;

[0042] A second preprocessing unit, configured to adjust the size of the second road image and perform color space conversion according to an input requirement of the YOLO model to obtain a preprocessed second road image;

[0043] a traffic light labeling unit, configured to label the position and state of the traffic light on the preprocessed second road image as second training data;

[0044] The second model training unit is used to train the YOLO model by using the second training data to obtain the pre-trained YOLO model.

[0045] In some embodiments, the apparatus further comprises a unit for training the SSD model before identifying and marking the position of the pedestrian in the road image using the pre-trained SSD model, and the unit for training the SSD model comprises:

[0046] A third image acquisition subunit, used to acquire a third road image;

[0047] A third preprocessing unit, configured to adjust the size of the third road image and perform color space conversion according to an input requirement of the SSD model to obtain the preprocessed third road image;

[0048] a pedestrian marking unit, configured to mark the positions of pedestrians in the preprocessed third road image as third training data;

[0049] The third model training unit is used to train the SSD model using the third training data to obtain the pre-trained SSD model.

[0050] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned traffic condition identification method when executing the computer program.

[0051] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application provides a vehicle, which includes a traffic condition recognition device as described above, or an electronic device as described above.

[0052] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned traffic condition identification method is implemented.

[0053] The embodiments of the present application include at least the following beneficial effects:

[0054] The present application can obtain road images; use a pre-trained CNN model to identify and annotate the position and shape of zebra crossings in road images, and display road images annotated with the position and shape of zebra crossings on the vehicle's display terminal; use a pre-trained YOLO model to identify and annotate the position and status of traffic lights in road images, and display road images annotated with the position and status of traffic lights on the vehicle's display terminal; use a pre-trained SSD model to identify and annotate the position of pedestrians in road images, and display road images annotated with the position of pedestrians on the vehicle's display terminal. Through each trained model, road images are specifically identified to obtain different traffic conditions, and the identified traffic conditions can be displayed on the display terminal on the vehicle to remind the driver to pay attention to driving safety, thereby achieving efficient and safe road traffic. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0056] Figure 1 A flow chart of a traffic condition identification method provided in an embodiment of the present application;

[0057] Figure 2 An example step diagram for obtaining a road image provided in an embodiment of the present application;

[0058] Figure 3 An example step diagram of training a CNN model provided in an embodiment of the present application;

[0059] Figure 4 An example step diagram of training a YOLO model provided in an embodiment of the present application;

[0060] Figure 5 An example step diagram of training an SSD model provided in an embodiment of the present application;

[0061] Figure 6 A schematic diagram of the structure of a traffic condition recognition device provided in an embodiment of the present application;

[0062] Figure 7 A schematic diagram of the structure of an image acquisition unit provided in an embodiment of the present application;

[0063] Figure 8 A schematic diagram of the structure of a unit for training a CNN model provided in an embodiment of the present application;

[0064] Fig. 9 A schematic diagram of the structure of a unit for training a YOLO model provided in an embodiment of the present application;

[0065] Fig.10 A schematic diagram of the structure of a unit for training an SSD model provided in an embodiment of the present application;

[0066] Fig.11 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.

[0068] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0069] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0070] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0071] Before describing the embodiments of the present application in detail, the terms and related technologies involved in the embodiments of the present application are first described as follows:

[0072] CNN: Convolutional Neural Networks, convolutional neural network.

[0073] YOLO: You Only Look Once, an object detection algorithm.

[0074] SSD: Single Shot Multibox Detector, a single-stage object detection algorithm.

[0075] As the level of vehicle intelligence continues to improve, there is an increasing need for full-scene, immersive smart cross-border designs to enhance user experience. For road safety, cars usually have to stop at signs or signals for pedestrians crossing the road, such as zebra crossings, and wait for pedestrians to pass completely before continuing to drive. Even vehicles that are moving will slow down when approaching zebra crossings, and if pedestrians pass, they will stop and wait for pedestrians to pass. Relevant technologies are difficult to identify traffic conditions efficiently and accurately, and both pedestrians and drivers will face more uncertainties and risks, affecting traffic efficiency and safety. Especially in the complex traffic environment of modern cities, the above problems are more prominent.

[0076] Therefore, the embodiments of the present application provide a method, device, electronic device, vehicle and storage medium for identifying traffic conditions. The technical solution of the present application includes: acquiring a road image; using a pre-trained CNN model to identify and mark the position and shape of the zebra crossing in the road image, and displaying the road image marked with the position and shape of the zebra crossing on the display terminal of the vehicle; using a pre-trained YOLO model to identify and mark the position and status of the traffic light in the road image, and displaying the road image marked with the position and status of the traffic light on the display terminal of the vehicle; using a pre-trained SSD model to identify and mark the position of pedestrians in the road image, and displaying the road image marked with the position of pedestrians on the display terminal of the vehicle. Through each trained model, the road image is specifically identified to obtain different traffic conditions, and the identified traffic conditions can be displayed on the display terminal on the vehicle to remind the driver to pay attention to driving safety, thereby achieving efficient and safe road traffic.

[0077] The embodiment of the present application provides a method for identifying traffic conditions, which relates to the field of image recognition technology. The method for identifying traffic conditions provided in the embodiment of the present application can be applied to a terminal, can be applied to a server, or can be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a vehicle-mounted terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or can be configured as a server cluster or a distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application that implements the method for identifying traffic conditions, etc., but is not limited to the above forms.

[0078] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0079] Reference Figure 1 The embodiment of the present application provides a method for identifying traffic conditions, which may include but is not limited to S100 to S130, as follows:

[0080] S100: Acquire a road image.

[0081] It is understandable that, in this embodiment, an image of any road section can be acquired as a road image, and then at least one of S110, S120, and S130 can be executed.

[0082] Reference Figure 2 , S100 may further include S101 or S102:

[0083] S101: Acquire the road image through a camera arranged on the vehicle.

[0084] Exemplarily, in this embodiment, the road image can be acquired through a driving recorder on the vehicle, or through a mobile smart terminal on the vehicle.

[0085] S102: Acquire the road image through a camera arranged on the road.

[0086] It is understandable that the present embodiment can acquire road images through any type of camera set on the road, such as a surveillance camera, a speed measurement camera, or a violation shooting camera.

[0087] S110: using a pre-trained CNN model to identify and annotate the position and shape of the zebra crossing in the road image, and displaying the road image annotated with the position and shape of the zebra crossing on a display terminal of the vehicle.

[0088] Exemplarily, this embodiment can use a marking box to mark the position and shape of the zebra crossing in the road image, and then display the road image marked with the position and shape of the zebra crossing on a display terminal such as the vehicle's central control terminal, mobile terminal, etc. to remind the driver to pay attention to driving safety.

[0089] Reference Figure 3 Before S110, the embodiment of the present application may further include a step of training a CNN model, and the step of training a CNN model includes S111 to S114:

[0090] S111: Acquire a first road image;

[0091] S112: performing grayscale processing and normalization processing on the first road image according to the input requirement of the CNN model to obtain the preprocessed first road image;

[0092] S113: marking the position and shape of the zebra crossing on the preprocessed first road image as first training data;

[0093] S114: Use the first training data to train the CNN model to obtain the pre-trained CNN model.

[0094] S120: Using a pre-trained YOLO model to identify and annotate the position and status of traffic lights in the road image, and displaying the road image annotated with the position and status of traffic lights on the display terminal of the vehicle.

[0095] Exemplarily, this embodiment can use a marking box to mark the position of traffic lights in a road image and mark the shape of the traffic lights, such as a red light, a green light, or a speed measurement interval, etc., and then display the road image marked with the position and shape of the traffic lights on a display terminal such as a vehicle's central control terminal, a mobile terminal, etc., to remind the driver to pay attention to driving safety.

[0096] Reference Figure 4 Before S120, the embodiment of the present application may further include a step of training a YOLO model, and the step of training the YOLO model includes S121 to S124:

[0097] S121: Acquire a second road image;

[0098] S122: adjusting the size of the second road image and performing color space conversion according to the input requirement of the YOLO model to obtain a preprocessed second road image;

[0099] S123: annotating the position and state of the traffic light on the preprocessed second road image as second training data;

[0100] S124: Train the YOLO model using the second training data to obtain the pre-trained YOLO model.

[0101] S130: using a pre-trained SSD model to identify and mark the positions of pedestrians in the road image, and displaying the road image with the positions of pedestrians marked on the display terminal of the vehicle.

[0102] Exemplarily, this embodiment can use a marking frame to mark the position of pedestrians in the road image, and then display the road image marked with the position of pedestrians on a display terminal such as a vehicle's central control terminal, a mobile terminal, etc., to remind the driver to pay attention to driving safety.

[0103] Reference Figure 5 Before S130, the embodiment of the present application may further include a step of training an SSD model, and the step of training the SSD model includes S131 to S134:

[0104] S131: Acquire a third road image;

[0105] S132: adjusting the size of the third road image and performing color space conversion according to the input requirement of the SSD model to obtain the preprocessed third road image;

[0106] S133: marking the positions of pedestrians in the preprocessed third road image as third training data;

[0107] S134: Use the third training data to train the SSD model to obtain the pre-trained SSD model.

[0108] Next, the solution of the embodiment of the present application will be introduced and explained in conjunction with specific application examples:

[0109] Specifically, this embodiment may include S1 to S4:

[0110] S1: Obtain road images through the camera mounted on the vehicle.

[0111] S2: Zebra crossing recognition.

[0112] Image preprocessing: Preprocess road images, including grayscale conversion, normalization and other operations, to meet the input requirements of the CNN model.

[0113] Model loading: Load a pre-trained CNN model.

[0114] Zebra crossing recognition: Use a pre-trained CNN model to recognize road images and obtain the location and shape of zebra crossings.

[0115] Result processing: Based on the recognition result, the position and shape of the zebra crossing are marked in the road image, and the road image with the position and shape of the zebra crossing marked is displayed on the display terminal of the vehicle.

[0116] S3: Traffic light recognition.

[0117] Image preprocessing: Preprocess road images, including resizing and color space conversion.

[0118] To illustrate the color space conversion, the following is explained:

[0119] Pixel is the most basic unit of an image. A pixel is a small, discrete point on an image. An image is composed of a series of pixels, each of which has a corresponding position and attributes. In a computer, images are usually stored in the form of a pixel matrix. A pixel matrix is ​​a two-dimensional array, and each element represents a pixel in the image. Each pixel can contain different information, such as color value, grayscale value, etc., depending on the type and representation of the image. Three-color three-channel (RGB) is a common image representation. In RGB representation, each pixel is represented by the values ​​of three color channels, namely red (R), green (G), and blue (B). The value of each color channel is usually between 0 and 255, representing the color intensity of the channel. When the values ​​of the three color channels are combined, the color of each pixel in the image is formed. For example, if the RGB value of a pixel is (255,0,0), it means that the pixel is red. If the RGB value of a pixel is (0,255,0), it means that the pixel is green. If the RGB value of a pixel is (0,0,255), it means that the pixel is blue. When the values ​​of the three color channels are equal, the pixel is grayscale. Various colors can be obtained through different combinations in the RGB color space. Various colors can be produced by adjusting the color value of each channel. For example, (255,255,255) represents white, and (0,0,0) represents black. In summary, the pixel is the basic unit of the image. Each pixel in the RGB representation is composed of the values ​​of the three color channels, representing the color of the pixel. Various image effects can be formed through reasonable combinations of color values.

[0120] Model loading: Load a pre-trained YOLO model.

[0121] Traffic light detection: Use the pre-trained YOLO model to recognize road images and identify the location and status of traffic lights.

[0122] Result processing: Based on the recognition results, the traffic lights are marked, and a road image with the locations of the traffic lights marked is displayed on the vehicle's display terminal, and corresponding sound prompts are issued according to the status of the traffic lights.

[0123] S4: Pedestrian recognition.

[0124] Image preprocessing: Preprocess road images, such as resizing and color space conversion.

[0125] Model loading: Load a pre-trained SSD model.

[0126] Pedestrian Detection: Use the pre-trained SSD model to recognize road images and identify the location of pedestrians.

[0127] Result processing: Based on the recognition results, a rectangular frame is marked on the road image to mark the position of the pedestrian, the road image with the pedestrian's position marked is displayed on the vehicle's display terminal, and a voice prompt is used to remind the pedestrian to pay attention to the vehicle.

[0128] In modern traffic environments, there is often uncertainty in the interaction between vehicles and pedestrians, especially at zebra crossings and traffic light intersections. The beneficial effects of this embodiment include:

[0129] Improve pedestrian safety: By accurately identifying zebra crossings, traffic lights and pedestrians, it can intelligently judge the intentions of pedestrians and use clear voice prompts to let pedestrians understand the intentions of vehicles, thereby avoiding traffic accidents.

[0130] Reduce driver confusion: Automatically detect and identify pedestrians, zebra crossings and traffic signals, and provide real-time prompts and suggestions to help drivers better understand pedestrian behavior and improve the accuracy of driving decisions.

[0131] Diversified interaction methods: Compared with the traditional single prompt method, this embodiment uses multiple prompt methods of vision and sound to communicate with pedestrians more clearly and effectively.

[0132] Improve traffic efficiency: By reducing the uncertainty of interactions between pedestrians and vehicles, it is possible to optimize traffic flow, improve overall traffic efficiency, and reduce traffic congestion and delays.

[0133] In summary, this embodiment can efficiently and accurately solve the safety and efficiency issues in the interaction between vehicles and pedestrians.

[0134] Reference Figure 6 The embodiment of the present application further provides a traffic condition recognition device, which can implement the above-mentioned traffic condition recognition method, and the device includes:

[0135] An image acquisition unit, used for acquiring a road image;

[0136] a zebra crossing recognition and display unit, used to recognize and mark the position and shape of the zebra crossing in the road image using a pre-trained CNN model, and display the road image with the position and shape of the zebra crossing marked on a display terminal of the vehicle;

[0137] a traffic light recognition and display unit, configured to recognize and annotate the position and status of traffic lights in the road image using a pre-trained YOLO model, and display the road image annotated with the position and status of traffic lights on the display terminal of the vehicle;

[0138] A pedestrian recognition and display unit is used to recognize and mark the positions of pedestrians in the road image using a pre-trained SSD model, and display the road image with the positions of pedestrians marked on the display terminal of the vehicle.

[0139] Reference Figure 7 In some embodiments, the image acquisition unit includes:

[0140] A vehicle-mounted camera image acquisition unit, used for acquiring the road image through a camera arranged on the vehicle;

[0141] The road camera image acquisition unit is used to acquire the road image through a camera arranged on the road.

[0142] Reference Figure 8 In some embodiments, the apparatus further comprises a unit for training the CNN model before using the pre-trained CNN model to identify and annotate the position and shape of the zebra crossing in the road image, and the unit for training the CNN model comprises:

[0143] A first image acquisition subunit, used to acquire a first road image;

[0144] A first preprocessing unit, configured to perform grayscale processing and normalization processing on the first road image according to an input requirement of the CNN model to obtain the preprocessed first road image;

[0145] a zebra crossing marking unit, configured to mark the position and shape of the zebra crossing on the preprocessed first road image as first training data;

[0146] The first model training unit is used to train the CNN model using the first training data to obtain the pre-trained CNN model.

[0147] Reference Fig. 9 In some embodiments, the apparatus further comprises a unit for training the YOLO model before using the pre-trained YOLO model to identify and annotate the position and state of the traffic light in the road image, and the unit for training the YOLO model comprises:

[0148] A second image acquisition subunit, used to acquire a second road image;

[0149] A second preprocessing unit, configured to adjust the size of the second road image and perform color space conversion according to an input requirement of the YOLO model to obtain a preprocessed second road image;

[0150] a traffic light labeling unit, configured to label the position and state of the traffic light on the preprocessed second road image as second training data;

[0151] The second model training unit is used to train the YOLO model by using the second training data to obtain the pre-trained YOLO model.

[0152] Reference Fig.10 In some embodiments, the apparatus further comprises a unit for training the SSD model before identifying and labeling the positions of pedestrians in the road image using the pre-trained SSD model, and the unit for training the SSD model comprises:

[0153] A third image acquisition subunit, used to acquire a third road image;

[0154] A third preprocessing unit, configured to adjust the size of the third road image and perform color space conversion according to an input requirement of the SSD model to obtain the preprocessed third road image;

[0155] a pedestrian marking unit, configured to mark the positions of pedestrians in the preprocessed third road image as third training data;

[0156] The third model training unit is used to train the SSD model using the third training data to obtain the pre-trained SSD model.

[0157] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0158] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned traffic condition recognition method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0159] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0160] See also Fig.11 , Fig.11 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0161] The processor 1101 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0162] The memory 1102 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1102 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1102, and the processor 1101 calls and executes the traffic condition recognition method of the embodiment of this application;

[0163] Input / output interface 1103, used to implement information input and output;

[0164] The communication interface 1104 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0165] A bus 1105 that transmits information between various components of the device (e.g., the processor 1101, the memory 1102, the input / output interface 1103, and the communication interface 1104);

[0166] The processor 1101 , the memory 1102 , the input / output interface 1103 and the communication interface 1104 are connected to each other in communication within the device via the bus 1105 .

[0167] The embodiment of the present application also provides a vehicle, which includes a traffic condition identification device as described above, or an electronic device as described above. Specifically, the vehicle can be a private car, such as a sedan, SUV, MPV or pickup truck. The vehicle can also be an operating vehicle, such as a van, bus, small truck or large trailer. The vehicle can be a gasoline vehicle or a new energy vehicle. When the vehicle is a new energy vehicle, it can be a hybrid vehicle or a pure electric vehicle.

[0168] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned traffic condition identification method is implemented.

[0169] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0170] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0171] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0172] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0173] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0174] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0175] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0176] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0177] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0178] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0179] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0180] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0181] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A traffic condition recognition method, It is characterized in that The method comprises the following steps: Acquire road images; Using a pre-trained CNN model to identify and annotate the position and shape of the zebra crossing in the road image, and displaying the road image annotated with the position and shape of the zebra crossing on a display terminal of the vehicle; Using a pre-trained YOLO model to identify and annotate the position and status of traffic lights in the road image, and displaying the road image annotated with the position and status of traffic lights on the display terminal of the vehicle; The positions of pedestrians in the road image are identified and annotated using a pre-trained SSD model, and the road image annotated with the positions of pedestrians is displayed on the display terminal of the vehicle.

2. A traffic condition recognition method according to claim 1, It is characterized in that The method of acquiring the road image comprises the following steps: Acquiring the road image by a camera disposed on the vehicle; Alternatively, the road image is acquired by a camera arranged on the road.

3. A traffic condition recognition method according to claim 1, It is characterized in that Before using the pre-trained CNN model to identify and mark the position and shape of the zebra crossing in the road image, the method further includes a step of training the CNN model, and the step of training the CNN model includes: Acquire a first road image; Performing grayscale processing and normalization processing on the first road image according to the input requirement of the CNN model to obtain the preprocessed first road image; Annotating the position and shape of the zebra crossing on the preprocessed first road image as first training data; The CNN model is trained using the first training data to obtain the pre-trained CNN model.

4. A traffic condition recognition method according to claim 1, It is characterized in that Before using the pre-trained YOLO model to identify and annotate the position and state of the traffic light in the road image, the method further includes the step of training the YOLO model, and the step of training the YOLO model includes: acquiring a second road image; Adjusting the size of the second road image and performing color space conversion according to the input requirement of the YOLO model to obtain a preprocessed second road image; annotating the position and state of the traffic light on the preprocessed second road image as second training data; The YOLO model is trained using the second training data to obtain the pre-trained YOLO model.

5. A traffic condition recognition method according to claim 1, It is characterized in that Before using the pre-trained SSD model to identify and mark the positions of pedestrians in the road image, the method further includes a step of training the SSD model, and the step of training the SSD model includes: acquiring a third road image; According to the input requirement of the SSD model, the size of the third road image is adjusted and the color space conversion is performed to obtain the preprocessed third road image; annotating the positions of pedestrians on the preprocessed third road image as third training data; The SSD model is trained using the third training data to obtain the pre-trained SSD model.

6. A traffic condition recognition device, It is characterized in that The device comprises: An image acquisition unit, used for acquiring a road image; a zebra crossing recognition and display unit, used to recognize and mark the position and shape of the zebra crossing in the road image using a pre-trained CNN model, and display the road image with the position and shape of the zebra crossing marked on a display terminal of the vehicle; a traffic light recognition and display unit, configured to recognize and annotate the position and status of traffic lights in the road image using a pre-trained YOLO model, and display the road image annotated with the position and status of traffic lights on the display terminal of the vehicle; A pedestrian recognition and display unit is used to recognize and mark the positions of pedestrians in the road image using a pre-trained SSD model, and display the road image with the positions of pedestrians marked on the display terminal of the vehicle.

7. A traffic condition recognition device according to claim 6, It is characterized in that The image acquisition unit comprises: A vehicle-mounted camera image acquisition unit, used for acquiring the road image through a camera arranged on the vehicle; The road camera image acquisition unit is used to acquire the road image through a camera arranged on the road.

8. A traffic condition recognition device according to claim 6, It is characterized in that The device further includes a unit for training the CNN model before using the pre-trained CNN model to identify and annotate the position and shape of the zebra crossing in the road image, and the unit for training the CNN model includes: A first image acquisition subunit, used to acquire a first road image; A first preprocessing unit, configured to perform grayscale processing and normalization processing on the first road image according to an input requirement of the CNN model to obtain the preprocessed first road image; a zebra crossing marking unit, configured to mark the position and shape of the zebra crossing on the preprocessed first road image as first training data; The first model training unit is used to train the CNN model using the first training data to obtain the pre-trained CNN model.

9. A traffic condition recognition device according to claim 6, It is characterized in that The apparatus further includes a unit for training the YOLO model before using the pre-trained YOLO model to identify and annotate the position and state of the traffic light in the road image, wherein the unit for training the YOLO model includes: A second image acquisition subunit, used to acquire a second road image; A second preprocessing unit, configured to adjust the size of the second road image and perform color space conversion according to an input requirement of the YOLO model to obtain a preprocessed second road image; a traffic light labeling unit, configured to label the position and state of the traffic light on the preprocessed second road image as second training data; The second model training unit is used to train the YOLO model by using the second training data to obtain the pre-trained YOLO model.

10. A traffic condition recognition device according to claim 6, It is characterized in that The apparatus further includes a unit for training the SSD model before using the pre-trained SSD model to identify and annotate the positions of pedestrians in the road image, and the unit for training the SSD model includes: A third image acquisition subunit, used to acquire a third road image; A third preprocessing unit, configured to adjust the size of the third road image and perform color space conversion according to an input requirement of the SSD model to obtain the preprocessed third road image; a pedestrian marking unit, configured to mark the positions of pedestrians in the preprocessed third road image as third training data; The third model training unit is used to train the SSD model using the third training data to obtain the pre-trained SSD model.

11. An electronic device, It is characterized in that The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements a traffic condition identification method as described in any one of claims 1 to 5 when executing the computer program.

12. A vehicle, It is characterized in that The vehicle includes a traffic condition recognition device as claimed in claim 6, or an electronic device as claimed in claim 11.

13. A computer-readable storage medium storing a computer program. It is characterized in that When the computer program is executed by a processor, a traffic condition recognition method according to any one of claims 1 to 5 is implemented.

Images