Vehicle identification early warning method, device and system

Abstract

The invention discloses a vehicle identification early warning method, device and system, and the method comprises the steps: obtaining a road condition image which is determined according to an original image collected by an electronic rearview mirror camera; identifying vehicles in the road condition image and vehicle types of the vehicles, and obtaining an area-of-interest image of each vehicle; performing image segmentation processing on each region-of-interest image to obtain a vehicle contour; according to the size relation between the pixel value of the vehicle contour and a pixel threshold value, whether a vehicle corresponding to the vehicle contour needs early warning or not is determined, and the pixel threshold value is related to the vehicle type; and when it is determined that early warning is needed, the outline of the vehicle is displayed on a display screen of the electronic rearview mirror. On the basis, the outline of the vehicle needing to be pre-warned is displayed on the display screen of the electronic rearview mirror, so that a driver is reminded to pay attention to the vehicle needing to be pre-warned, and the driving safety of the vehicle is improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle identification, and in particular to a vehicle identification warning method, device and system. Background Art

[0002] Conventional rearview mirrors of vehicles are installed on the outside of the vehicle. The clarity of the rearview mirrors outside the vehicle is easily affected by bad weather outside, which can easily lead to the driver's inaccurate observation of the driving environment outside the vehicle.

[0003] Therefore, on the basis of complying with GB 15084-2022 "Performance and Installation Requirements for Indirect Vision Devices for Motor Vehicles", the application value of CMS (Camera Monitor System, electronic rearview mirror) is reflected. CMS is used to process the original RAW format images collected by the sensor through ISP (Image Signal Processor), convert them into digital signals and transmit them to the system controller. The controller is deployed in the display screen and communicates with other control units through CAN (Controller Area Network). After processing the video signal, it is displayed in real time on the CMS display screen, so that the driver can observe the driving environment without being affected by bad weather on the field of vision and clarity. In addition, the driver can also manually adjust the screen ratio and field of view angle of the driving environment on the appropriate display screen.

[0004] However, during driving, the driver will be distracted if he manually adjusts the display screen according to the environment, which may cause driving hazards. If the screen size and viewing angle are not in the optimal position, it will greatly affect the driver's judgment of the road conditions and cause safety hazards. Summary of the invention

[0005] The embodiments of the present application provide a vehicle identification and warning method, device and system, which improve the safety of vehicle driving.

[0006] In a first aspect, an embodiment of the present application provides a vehicle identification and warning method, the method comprising:

[0007] Acquire a road condition image, wherein the road condition image is determined based on an original image captured by an electronic rearview mirror camera;

[0008] Identify vehicles and vehicle types in the road condition image, and obtain an image of a region of interest of each vehicle;

[0009] Performing image segmentation processing on each of the images of the region of interest to obtain a vehicle outline;

[0010] Determining whether a vehicle corresponding to the vehicle outline needs an early warning according to a magnitude relationship between a pixel value of the vehicle outline and a pixel threshold, wherein the pixel threshold is related to the vehicle type;

[0011] When it is determined that a warning is needed, the vehicle outline is displayed on the electronic rearview mirror display screen.

[0012] In a second aspect, an embodiment of the present application provides a vehicle identification and warning system, the system comprising: an electronic rearview mirror camera for collecting original images; wherein the electronic rearview mirror camera comprises a camera module and a support for supporting the camera module, and the camera module is connected to the electronic rearview mirror display screen through a wiring harness; a controller for executing the method described in the first aspect; and an electronic rearview mirror display screen for displaying the vehicle outline and outputting warning information.

[0013] In a third aspect, an embodiment of the present application provides a vehicle identification and warning device, the device comprising:

[0014] An acquisition module, used for acquiring a road condition image, wherein the road condition image is determined based on an original image collected by an electronic rearview mirror camera;

[0015] An identification module, used to identify vehicles and vehicle types in the road condition image, and obtain an image of a region of interest of each vehicle;

[0016] A processing module, used for performing image segmentation processing on each of the images of the region of interest to obtain a vehicle outline;

[0017] A determination module, used to determine whether the vehicle corresponding to the vehicle outline needs an early warning according to the size relationship between the pixel value of the vehicle outline and a pixel threshold, wherein the pixel threshold is related to the vehicle type;

[0018] The display module is used to display the vehicle outline on the electronic rearview mirror display screen when it is determined that an early warning is needed.

[0019] In a fourth aspect, an embodiment of the present application provides a terminal device, the device comprising: a processor and a memory storing computer program instructions;

[0020] When the processor executes the computer program instructions, the vehicle identification and warning method of the first aspect is implemented.

[0021] In a fifth aspect, an embodiment of the present application provides a computer storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the vehicle identification and warning method of the first aspect is implemented.

[0022] In a sixth aspect, an embodiment of the present application provides a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the vehicle identification and warning method as in the first aspect.

[0023] The vehicle identification and warning method and system of the embodiments of the present application obtain road condition images, identify vehicles and vehicle types in the road condition images, thereby obtaining image regions of interest for each vehicle, and then performing image segmentation processing on each image region of interest to obtain vehicle contours from the image region of interest. Therefore, it is possible to determine whether a vehicle requires a warning based on the relationship between the pixel value of the vehicle contour and the pixel threshold. For vehicles that require a warning, the vehicle contour of the vehicle is displayed on the electronic rearview mirror display screen, so that the driver can see the contour of the vehicle that requires a warning on the electronic rearview mirror display screen, thereby reminding the driver to pay attention to the vehicle that requires a warning, thereby improving the safety of vehicle driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0025] Figure 1 is a flow chart of a vehicle identification and warning method according to an exemplary embodiment;

[0026] Figure 2 is a structural block diagram of a vehicle identification and warning system according to an exemplary embodiment;

[0027] Figure 3 is a structural block diagram of a vehicle identification and warning device according to an exemplary embodiment;

[0028] Figure 4 The figure is a structural block diagram of a vehicle identification and warning device according to an exemplary embodiment. DETAILED DESCRIPTION

[0029] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0030] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-otherwise inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0031] As described in the background technology section, in the prior art, if the driver manually adjusts the display screen according to the environment, it will distract the driver's attention and pose a driving hazard. If the screen size and viewing angle are not in the optimal position, it will greatly affect the driver's judgment of the road conditions and cause safety hazards.

[0032] Based on this, in the vehicle identification and warning method provided in the embodiment of the present application, for vehicles that need warning, the vehicle outline of the vehicle is displayed on the electronic rearview mirror display screen, so that the driver can see the vehicle outline of the vehicle that needs warning from the electronic rearview mirror display screen, thereby reminding the driver to pay attention to the vehicle that needs warning, thereby improving the safety of vehicle driving.

[0033] The present application provides a vehicle identification and early warning method and system. The vehicle identification and early warning method provided in the embodiment of the present application is first introduced below.

[0034] Figure 1 A schematic diagram of a vehicle identification and warning method provided by an embodiment of the present application is shown. Figure 1 As shown, the method may include the following steps:

[0035] S101, obtaining a road condition image, where the road condition image is determined based on an original RAW image collected by an electronic rearview mirror camera, and is processed by an ISP module to output a color road condition image;

[0036] S102, identifying vehicles and vehicle types in the road condition image through a vehicle target detection network model, and obtaining an image of a region of interest for each vehicle;

[0037] S103, performing image segmentation processing on each region of interest image to obtain a vehicle outline;

[0038] S104, determining whether a vehicle corresponding to the vehicle outline needs an early warning based on a magnitude relationship between a pixel value of the vehicle outline and a pixel threshold, wherein the pixel threshold is related to the vehicle type;

[0039] S105, when it is determined that a warning is required, the vehicle outline is displayed on the electronic rearview mirror display screen.

[0040] Based on the above embodiment, the original road condition image is collected through the electronic rearview mirror, and the vehicles and vehicle types in the road condition image are identified to obtain the region of interest image of each vehicle. Then, each region of interest image is subjected to image segmentation processing to obtain the vehicle contour from the region of interest image. Therefore, it is possible to determine whether the vehicle needs to be warned based on the relationship between the pixel value of the vehicle contour and the pixel threshold. For the vehicle that needs to be warned, the vehicle contour of the vehicle is displayed on the electronic rearview mirror display screen, so that the driver can see the vehicle contour of the vehicle that needs to be warned from the electronic rearview mirror display screen, without the driver having to adjust the display screen on the electronic rearview mirror display screen, thereby improving the safety of vehicle driving.

[0041] In the above S101, the electronic rearview mirror camera captures the original image of the exterior of the vehicle while the vehicle is traveling, and after image processing is performed on the original image, a road condition image can be obtained.

[0042] As an example, the electronic rearview mirror may be a CMS electronic rearview mirror.

[0043] Specifically, the road condition image is a color image, and the color image may include: at least one of an RGB image or a YUV image.

[0044] The above S101 may include:

[0045] S1011, obtaining an original image captured by the electronic rearview mirror camera;

[0046] S1012, performing image processing on the original image to obtain a color image.

[0047] In the above S1011, the captured scene image is converted into an original image through the photosensitive sensor and the analog-to-digital converter in the rearview mirror camera.

[0048] As an example, the original image is a RAW image.

[0049] In the above S1012, the image processing module performs color interpolation, color correction, brightness adjustment and other processing on the original image to complete color reconstruction and image reconstruction to obtain a road condition image, and the road condition image will be displayed on the electronic rearview mirror display screen.

[0050] As an example, the image processing module is an ISP.

[0051] In the above S102, the road condition image is input into the vehicle recognition model, and the vehicle type recognition model will identify each vehicle in the road condition image and the vehicle type corresponding to each vehicle, so that the area of ​​interest image of each vehicle can be extracted from the road condition image according to the vehicle type.

[0052] As an example, the vehicle types may be classified according to parameters such as the vehicle's external dimensions, internal space, wheelbase, displacement, curb weight, etc. Specifically, the vehicle types include: Class A vehicles, Class B vehicles, Class C vehicles, and the like.

[0053] As an example, the vehicle recognition model is a target detection network model, which is divided into four parts: input, backbone, neck and output. The specific recognition process is as follows: after inputting the road condition image at the input, the vehicle recognition model is improved during the training process of the input, mainly including data enhancement, adaptive anchor frame calculation and adaptive image scaling. Data enhancement is to expand the data set by splicing in the form of random scaling, random cropping and random arrangement. The adaptive anchor frame calculates the best anchor frame value in different training sets. The adaptive image scaling completes image scaling and filling by calculating the scaling ratio, the scaled size and the black edge filling value. The image feature set extracted from the backbone network is the feature layer, and three feature layers are obtained for the next step of network model construction. The neck network is usually located in the middle of the backbone network and the head network. The feature maps of different levels are fused together through upsampling and downsampling operations to generate a multi-scale feature pyramid and obtain three enhanced effective feature layers. The output end (Yolo Head) serves as the classifier and regressor of the vehicle recognition model. It judges the feature points, determines whether there are objects corresponding to the feature points and classifies them. Finally, it outputs the results of recognition and classification and captures the image of the area of ​​interest.

[0054] Specifically, the above S102 may include:

[0055] S1021, classifying vehicle types according to a vehicle recognition model;

[0056] S1022, determining a region of interest of each of the vehicles in the road condition image according to the recognition result of the vehicle type;

[0057] S1023: Extracting a region of interest image of each region of interest from the road condition image.

[0058] In this embodiment, since different types of vehicles have different external dimensions, the dimensions of the regions of interest extracted from each type of vehicle are also different. Therefore, it is necessary to determine the region of interest of each vehicle in the road condition image according to the vehicle type.

[0059] As an example, a region of interest (ROI) refers to a specific area selected from a region of interest, which is the focus of analysis or processing. ROI can be an area of ​​any shape outlined in the form of a box, circle, ellipse, irregular polygon, etc. These areas can be specific targets in the image, such as vehicles, pedestrians, buildings, etc., or specific areas in the image, such as roads, traffic signs, etc. In this application, the specific target is a vehicle.

[0060] In one embodiment, the vehicle type classification according to the vehicle recognition model includes:

[0061] Inputting the road condition image to the input end, performing data enhancement, anchor frame calculation and image scaling on the road condition image through the input end to obtain a first image;

[0062] Inputting the first image into the backbone network, performing feature extraction on the first image through the backbone network to obtain a plurality of feature layers of different scales, each feature layer containing information of different levels;

[0063] Inputting the plurality of feature layers of different scales into the neck network, and performing feature enhancement on the feature layers through the neck network;

[0064] The enhanced feature layer is input into the output end, and the feature layer is classified and regressed through the output end to obtain the recognition result of the vehicle type.

[0065] In this embodiment, the vehicle recognition model includes an input end, a backbone network, a neck network and an output end. As an example, the vehicle recognition module can be a YOLOv5s target detection network.

[0066] The following is an introduction to the recognition process of the vehicle recognition model:

[0067] The YOLOv5s target detection network is divided into four parts: input, backbone, neck, and output. It is the network with the smallest depth and the smallest feature map width in the YOLOv5 network series. The specific target detection algorithm process is as follows: after the vehicle image collected by the CMS rearview mirror with a size of 608×608 is input at the input, the YOLOv5s network model is improved during the training process of the input, mainly including Mosaic data enhancement, adaptive anchor frame calculation, and adaptive image scaling. Mosaic data enhancement is to expand the data set by splicing in a random scaling, random cropping, and random arrangement. The adaptive anchor frame calculates the best anchor frame value in different training sets. The adaptive image scaling completes image scaling and filling by calculating the scaling ratio, the scaled size, and the black edge filling value. In the backbone network, the CSPDarknet structure and the Focus structure are used as the benchmark network. The extracted image feature set is the feature layer, and 3 feature layers are obtained for the next step of network construction. The neck network (Neck) is usually located in the middle of the backbone network and the head network. It uses the FPN+PAN module to fuse feature maps of different levels through upsampling and downsampling operations to generate a multi-scale feature pyramid and obtain three enhanced effective feature layers. The output end (Yolo Head) serves as the classifier and regressor of YOLOv5s to judge the feature points, determine whether there is an object corresponding to the feature points and classify them, and finally output the recognition and classification results.

[0068] In the above S1022 and S1023, after the region of interest of each vehicle in the road condition image is determined, the region of interest of each vehicle is extracted from the road condition image by using an image segmentation algorithm to obtain a region of interest image.

[0069] Based on this, by training the vehicle target detection network model, the region of interest image is extracted from the road condition image. Obtaining the region of interest image can narrow the processing scope, improve the processing speed and accuracy, and reduce unnecessary calculation and resource consumption.

[0070] In the above S103, the image of the region of interest includes not only the vehicle but also the environmental background outside the vehicle. Therefore, it is necessary to perform image segmentation processing on the image of the region of interest to extract the vehicle contour related to the vehicle from the image of the region of interest, so as to accurately obtain image information related to the vehicle from the image of the region of interest.

[0071] As an example, the above S103 may include:

[0072] S1031, performing image filtering processing on each region of interest image to obtain a filtered region of interest image;

[0073] S1032, performing enhancement processing on the filtered region of interest image to obtain an enhanced region of interest image;

[0074] S1033, segmenting the enhanced region of interest image into a background region and a vehicle region according to a preset threshold;

[0075] S1034: extracting a vehicle outline from the vehicle area.

[0076] In the above S1031, the region of interest image is filtered by using Wiener filtering and adaptive median filtering algorithms, so as to obtain a filtered region of interest image.

[0077] Since motion blur will be generated during driving, the use of Wiener filtering can effectively suppress motion blur and Gaussian white noise. Wiener filtering is usually used to extract useful signals contaminated by noise. It performs filtering based on the minimum mean square error criterion. The Wiener filtering method is a statistical method. It uses the correlation matrix of the image and the noise as the optimal criterion, and adjusts the output of the filter according to the local variance of the image. The larger the local variance, the stronger the smoothing effect of the filter. The present invention adopts an adaptive median filtering algorithm, which dynamically changes the window size of the median filter according to the set conditions, can smooth salt and pepper noise and other non-impulse noise, and reduce the impact of noise on image quality.

[0078] In the above S1032, the filtered region of interest image is subjected to image enhancement processing again to obtain an enhanced region of interest image, thereby making it easier to perform subsequent image segmentation in the enhanced region of interest image.

[0079] The Sobel image sharpening algorithm is used to enhance the image quality. The template is used as the kernel to perform convolution and operation with each pixel of the image, and then a suitable threshold is selected to extract the edge of the image. The obtained edge image is fused with the original image to obtain the enhanced image of the region of interest.

[0080] In the above S1033, the shooting environment is complex during driving, and the image of the region of interest may still contain other objects after being extracted, so it is necessary to perform image segmentation on the target vehicle.

[0081] Specifically, the enhanced region of interest image may be segmented into a background region and a vehicle region according to a preset threshold.

[0082] In S1034, extracting the vehicle contour from the vehicle region may include:

[0083] Performing morphological processing on the vehicle region to obtain a processed vehicle region;

[0084] The edge of the processed vehicle area is extracted through the edge detection algorithm to obtain the vehicle contour.

[0085] Based on the above embodiments, the present invention uses a morphologically based maximum inter-class variance image segmentation algorithm to perform image segmentation and further extract the accurate contour of the vehicle. The maximum inter-class variance image segmentation algorithm selects the best threshold image to be divided into two parts: the background area and the target area. If the inter-class variance of the two parts is larger, the difference is larger, the correlation is smaller, and the image segmentation effect is better. The threshold when the inter-class variance obtains the maximum value is the preset threshold. After the segmentation algorithm divides the enhanced processed region of interest image into two parts: the background area and the vehicle area, the contour of the vehicle is more obvious. Then, the image burrs are removed by morphology to fill small holes and breakpoints in the image, so that the vehicle contour is smoother. The Canny edge detection algorithm is used to extract the vehicle edge, so as to extract the accurate vehicle contour from the processed vehicle area.

[0086] In the above S104, in one embodiment, S104 includes:

[0087] When the pixel value of the vehicle outline is greater than or equal to the pixel threshold, determining that the vehicle corresponding to the vehicle outline needs to be warned;

[0088] When the pixel value of the vehicle outline is less than the pixel threshold, it is determined that the vehicle corresponding to the vehicle outline does not need warning.

[0089] In this embodiment, the pixel value of the vehicle contour is calculated using the extracted vehicle contour, and the pixel value is compared with the pixel threshold. If it exceeds the pixel threshold, it indicates danger and a warning is required. If it does not exceed the pixel threshold, no warning is required.

[0090] In the above S105, if a warning is needed, it means that the distance between the vehicle and the host vehicle is less than the warning distance, and a warning is needed, then the vehicle outline is displayed on the electronic rearview mirror display screen to remind the driver.

[0091] Specifically, S105 may include:

[0092] S1051, when it is determined that a warning is required, obtaining a size of a first display area on the electronic rearview mirror display screen;

[0093] S1052, adjusting the size of the vehicle outline according to the size of the first display area;

[0094] S1053: Display the adjusted vehicle outline in the first display area.

[0095] Based on the above embodiment, by obtaining the size of the first display area and adjusting the size of the vehicle outline according to the size, the adjusted vehicle outline can be displayed in the first display area, so that the electronic rearview mirror display screen not only displays the road condition image, but also the vehicle outline that needs to be warned, thereby improving the display effect of the electronic rearview mirror display screen.

[0096] In the above S1051, since the electronic rearview mirror display screen also needs to display the road condition image, the outline of the vehicle that needs to be warned can only be displayed in a part of the area (first display area) on the electronic rearview mirror display screen, so it is necessary to obtain the size of the first display area.

[0097] In the above S1052, according to the size of the first display area, the outline of the vehicle that needs a warning may be adaptively cropped, scaled, or adjusted to adjust the size of the outline of the vehicle that needs a warning.

[0098] In the above S1053, the adjusted vehicle outline is displayed in the first display area on the electronic rearview mirror display screen.

[0099] In order to accurately determine the pixel threshold, the present application also provides another implementation of the vehicle identification and warning method.

[0100] Before the above S104, the method may further include the following steps:

[0101] S201, obtaining a safety distance corresponding to a preset vehicle type;

[0102] S202, determining a pixel value corresponding to a safety distance of the preset vehicle type according to a correlation relationship between a safety distance of the vehicle and a pixel value of the contour area;

[0103] S203: Determine the pixel value of the contour area of ​​the preset vehicle type at the safety distance as the pixel threshold corresponding to the preset vehicle type.

[0104] The safety distance of vehicles of different levels is calibrated in advance, and the pixel value of the contour area of ​​vehicles of different levels is calculated. The proportion of this pixel value in the overall image pixel value is inversely proportional to the safety distance. According to the safety distance of vehicles of different levels, the threshold of vehicles of different levels when exceeding the safety distance is calculated. This threshold is the preset pixel threshold, which serves as the judgment basis of the main control module. According to the relationship between the preset vehicle type and the preset pixel threshold, the pixel threshold corresponding to the vehicle type is determined.

[0105] Based on the above embodiment, the vehicle type of each vehicle in the road condition image is determined by using a vehicle type recognition model, thereby improving the recognition accuracy of the vehicle type of each vehicle in the road condition image, and then using the relationship between the preset threshold and the preset vehicle type to determine the pixel threshold corresponding to the vehicle type, thereby improving the accuracy of the pixel value.

[0106] By preparing a large number of vehicle images, labeling each vehicle image with the vehicle type, and inputting both the vehicle images and labels into a deep learning model for training, a trained vehicle type recognition model can be obtained, and then the road condition images can be input into the vehicle type recognition model. According to the output results of the vehicle type recognition model, the vehicle type can be determined.

[0107] In order to improve the warning efficiency, the present application also provides another implementation of the vehicle identification warning method.

[0108] The method may further comprise the following steps:

[0109] S301, when it is determined that a warning is needed, controlling the electronic rearview mirror display screen to output a warning message.

[0110] Based on the above embodiment, the electronic rearview mirror display screen is controlled to output warning information, thereby reminding the driver to pay attention to the electronic rearview mirror display screen, thereby improving the warning efficiency.

[0111] In the above S301, the warning information may include: display style information, audio information, etc., wherein the display style information may include the electronic rearview mirror display screen flashing, and the audio information may be voice prompt information. When a warning is needed, the electronic rearview mirror display screen may be controlled to flash, and the voice prompt information may be played.

[0112] It should be noted that the application scenarios described in the above-mentioned embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field can know that with the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0113] Based on the same inventive concept, the present application also provides a vehicle identification and warning system. Figure 2 Provide detailed explanation.

[0114] Figure 2 A schematic diagram of the hardware structure of a vehicle identification and warning system 200 provided in an embodiment of the present invention is shown.

[0115] like Figure 2 As shown, the vehicle identification warning system 200 may include:

[0116] An electronic rearview mirror camera 210, used for collecting original images;

[0117] A controller 220, configured to execute the method in the above method embodiment;

[0118] The electronic rearview mirror display screen 230 is used to display the vehicle outline and output warning information.

[0119] The vehicle identification and warning system 200 provided in the embodiment of the present application can implement each process implemented in the above method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0120] like Figure 3 As shown, the embodiment of the present application further provides a vehicle identification and warning device 300, the device comprising:

[0121] An acquisition module 301 is used to acquire a road condition image, wherein the road condition image is determined based on an original image collected by an electronic rearview mirror camera;

[0122] The recognition module 302 is used to recognize vehicles and vehicle types in the road condition image, and obtain an image of a region of interest of each vehicle;

[0123] The processing module 303 is used to perform image segmentation processing on each of the images of the region of interest to obtain a vehicle outline;

[0124] A determination module 304, configured to determine whether a vehicle corresponding to the vehicle outline needs an early warning according to a magnitude relationship between a pixel value of the vehicle outline and a pixel threshold, wherein the pixel threshold is related to the vehicle type;

[0125] The display module 305 is used to display the vehicle outline on the electronic rearview mirror display screen when it is determined that a warning is needed.

[0126] It should be noted that the vehicle identification and warning device 300 is a device corresponding to the above-mentioned vehicle identification and warning method. All implementation methods in the above-mentioned method embodiment are applicable to the embodiment of the device and can achieve the same technical effect.

[0127] Figure 4 A schematic diagram of the hardware structure of a terminal device provided in an embodiment of the present application is shown.

[0128] The terminal device may include a processor 401 and a memory 402 storing computer program instructions.

[0129] Specifically, the processor 401 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0130] Memory 402 may include a large capacity memory for data or instructions. By way of example and not limitation, memory 402 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, memory 402 may include a removable or non-removable (or fixed) medium. In appropriate cases, memory 402 may be inside or outside of an integrated gateway disaster recovery device. In a specific embodiment, memory 402 is a non-volatile solid-state memory.

[0131] In certain embodiments, the memory 402 may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage media device, an optical storage media device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0132] The processor 401 implements any one of the sequence detection methods in the above embodiments by reading and executing computer program instructions stored in the memory 402 .

[0133] In one example, the terminal device may further include a communication interface 404 and a bus 410. Figure 4 As shown, the processor 401, the memory 402, and the communication interface 404 are connected via a bus 410 and communicate with each other.

[0134] The communication interface 404 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0135] Bus 410 includes hardware, software or both, and the parts of online data flow billing equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 410 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.

[0136] In addition, in combination with the sequence detection method in the above embodiment, the embodiment of the present application can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the sequence detection methods in the above embodiment is implemented.

[0137] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0138] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0139] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0140] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable sequence detection device to produce a machine so that these instructions executed by the processor of the computer or other programmable sequence detection device enable the realization of the function / action specified in one or more boxes of the flowchart and / or block diagram. This processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of the boxes in the block diagram and / or flowchart can also be realized by the dedicated hardware that performs the specified function or action, or can be realized by the combination of dedicated hardware and computer instructions.

[0141] The above are only specific implementation methods of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A vehicle identification and early warning method, characterized in that: The method comprises: Acquire a road condition image, wherein the road condition image is determined based on an original image captured by an electronic rearview mirror camera; Identify vehicles and vehicle types in the road condition image, and obtain an image of a region of interest of each vehicle; Performing image segmentation processing on each of the images of the region of interest to obtain a vehicle outline; Determining whether a vehicle corresponding to the vehicle outline needs an early warning according to a magnitude relationship between a pixel value of the vehicle outline and a pixel threshold, wherein the pixel threshold is related to the vehicle type; When it is determined that a warning is needed, the vehicle outline is displayed on the electronic rearview mirror display screen.

2. The method according to claim 1, characterized in that The identifying the vehicles and the vehicle types in the road condition image to obtain an image of a region of interest of each of the vehicles includes: Classify vehicle types based on vehicle recognition models; Determining the region of interest of each of the vehicles in the road condition image according to the recognition result of the vehicle type; Extracting a region of interest image of each region of interest from the road condition image.

3. The method according to claim 2, characterized in that The vehicle identification model includes an input end, a backbone network, a neck network and an output end; The vehicle type classification according to the vehicle recognition model includes: Inputting the road condition image to the input end, performing data enhancement, anchor frame calculation and image scaling on the road condition image through the input end to obtain a first image; Inputting the first image into the backbone network, performing feature extraction on the first image through the backbone network to obtain a plurality of feature layers of different scales, each feature layer containing information of different levels; Inputting the plurality of feature layers of different scales into the neck network, and performing feature enhancement on the feature layers through the neck network; The enhanced feature layer is input into the output end, and the feature layer is classified and regressed through the output end to obtain the recognition result of the vehicle type.

4. The method according to claim 1, characterized in that The performing image segmentation processing on each of the region of interest images to obtain the vehicle contour includes: Performing image filtering processing on each of the region of interest images to obtain a filtered region of interest image; Performing enhancement processing on the filtered region of interest image to obtain an enhanced region of interest image; Segmenting the enhanced region of interest image into a background region and a vehicle region according to a preset threshold; A vehicle contour is extracted from the vehicle region.

5. The method according to claim 4, characterized in that The step of extracting a vehicle contour from the vehicle region comprises: Performing morphological processing on the vehicle region to obtain a processed vehicle region; The processed vehicle area is subjected to edge extraction by an edge detection algorithm to obtain the vehicle contour.

6. The method according to claim 1, characterized in that Before determining whether the vehicle corresponding to the vehicle contour needs an early warning based on the size relationship between the pixel value of the vehicle contour and the pixel threshold, the method further includes: Get the safety distance corresponding to the preset vehicle type; Determining a pixel value corresponding to the safety distance of the preset vehicle type according to a correlation relationship between the safety distance of the vehicle and the pixel value of the contour area; The pixel value of the contour area of ​​the preset vehicle type at the safety distance is determined as the pixel threshold corresponding to the preset vehicle type.

7. The method according to claim 6, characterized in that The determining whether the vehicle corresponding to the vehicle outline needs an early warning according to the size relationship between the pixel value of the vehicle outline and the pixel threshold comprises: When the pixel value of the vehicle outline is greater than or equal to the pixel threshold, determining that the vehicle corresponding to the vehicle outline needs to be warned; When the pixel value of the vehicle outline is less than the pixel threshold, it is determined that the vehicle corresponding to the vehicle outline does not need warning.

8. The method according to claim 1, characterized in that When it is determined that a warning is needed, displaying the vehicle outline on the electronic rearview mirror display screen includes: When it is determined that an early warning is required, obtaining a size of a first display area on the display screen of the electronic rearview mirror; adjusting the size of the vehicle outline according to the size of the first display area; The adjusted vehicle outline is displayed in the first display area.

9. A vehicle identification and warning system, characterized in that: The system comprises: An electronic rearview mirror camera, used to collect original images; wherein the electronic rearview mirror camera includes a camera module and a support member for supporting the camera module, and the camera module is connected to the electronic rearview mirror display screen through a wiring harness; A controller, configured to execute the method according to any one of claims 1 to 8; The electronic rearview mirror display screen is used to display the vehicle outline and output warning information.

10. A vehicle identification and warning device, characterized in that: The device comprises: An acquisition module, used for acquiring a road condition image, wherein the road condition image is determined based on an original image collected by an electronic rearview mirror camera; An identification module, used to identify vehicles and vehicle types in the road condition image, and obtain an image of a region of interest of each vehicle; A processing module, used for performing image segmentation processing on each of the images of the region of interest to obtain a vehicle outline; A determination module, used to determine whether the vehicle corresponding to the vehicle outline needs an early warning according to the size relationship between the pixel value of the vehicle outline and a pixel threshold, wherein the pixel threshold is related to the vehicle type; The display module is used to display the vehicle outline on the electronic rearview mirror display screen when it is determined that an early warning is needed.

Images