Method and system for detecting object attached to vehicle-mounted camera, and terminal

Abstract

A method and system for detecting an object attached to a vehicle-mounted camera, and a terminal. The method comprises: acquiring consecutive video frames collected by a vehicle-mounted camera, and storing same in a frame list; preprocessing each video frame in the frame list; performing difference detection on all preprocessed adjacent video frames, and identifying pixel-invariant regions between the adjacent video frames on the basis of difference detection results; and performing multi-frame similarity comparison on the pixel-invariant regions, so as to detect the presence of an attached object. The present application can effectively improve the speed and accuracy of detection of an object attached to a vehicle-mounted camera, and can also reduce the requirements for computing resources; and the present application has the advantages of high detection accuracy, high real-time performance, high robustness, low-cost implementation, high reliability, etc.

Description

Vehicle camera attachment detection method, system and terminal TECHNICAL FIELD

[0001] The application belongs to the technical field of automobile automatic driving, and particularly relates to a vehicle camera attachment detection method, system and terminal. BACKGROUND

[0002] With the rapid development of automatic driving technology and advanced driving assistance systems (ADAS), the perception ability of vehicles to the surrounding environment becomes crucial. As a key component of the perception system, the video data quality provided by the vehicle camera directly affects the decision accuracy and driving safety of the system. However, the lens surface of the vehicle camera may be affected by the attachment of rain, snow, ice, dust, insect residues or other contaminants, which may cause the field of view to be blurred or blocked, thereby affecting the clarity and accuracy of the data. Therefore, timely detection and processing of these attachments is an important prerequisite for ensuring the normal operation of the camera and thus ensuring the performance of the automatic driving and ADAS system.

[0003] Traditional vehicle camera attachment detection methods usually rely on image processing techniques such as edge detection, blur detection, etc. to identify abnormalities in the image. These methods may be feasible in static or controlled environments, but face multiple challenges in actual applications. First, these methods often require complex calculations and model analysis on each frame of image, which not only consumes a large amount of computing resources, but also is difficult to achieve real-time performance, making it unsuitable for embedded systems that require fast response. Second, traditional methods have difficulty in distinguishing between normal scene changes (such as weather changes, light changes) and abnormal changes caused by attachments, and are easily affected by environmental factors, leading to false positives or false negatives. In addition, for dynamic driving environments such as high-speed moving vehicles and constantly changing backgrounds, the stability and accuracy of traditional detection methods will be greatly compromised. SUMMARY

[0004] The purpose of the present application is to provide a vehicle camera attachment detection method, system and terminal for improving the speed and accuracy of vehicle camera attachment detection while reducing the demand for computing resources.

[0005] In a first aspect, the present application provides a vehicle camera attachment detection method, comprising: acquiring continuous video frames captured by a vehicle camera and saving them to a frame list; preprocessing each video frame in the frame list; performing difference detection on all adjacent video frames after preprocessing, and identifying pixel invariant regions between the adjacent video frames based on the difference detection result; and performing multi-frame similarity comparison on the pixel invariant regions to detect the presence of attachments.

[0006] In an implementation form of the first aspect, acquiring continuous video frames captured by a vehicle camera and saving them to a frame list comprises:

[0007] collecting the image by the vehicle-mounted camera according to a preset time interval to obtain the continuous video frames when the vehicle speed exceeds a first preset threshold value;

[0008] saving the continuous video frames into the frame list in sequence.

[0009] In an implementation form of the first aspect, the pre-processing each video frame in the frame list comprises:

[0010] removing a static and unchanged region in the video frame due to a shooting angle limitation of the vehicle-mounted camera;

[0011] performing a grayscale processing on a valid region retained in the video frame to convert an original color image into a grayscale image;

[0012] performing a binaryzation processing on the grayscale image to obtain a black-and-white image.

[0013] In an implementation form of the first aspect, performing a difference detection on all the pre-processed adjacent video frames and identifying a pixel unchanged region between the adjacent video frames based on a difference detection result comprises:

[0014] calculating an absolute difference value of corresponding position pixels between each group of the adjacent video frames;

[0015] judging whether the absolute difference value is greater than a second preset threshold value;

[0016] if yes, marking a position of the corresponding pixel as a pixel changed region;

[0017] otherwise, marking the position of the corresponding pixel as a pixel unchanged region.

[0018] In an implementation form of the first aspect, performing a multi-frame similarity comparison on the pixel unchanged region to detect the existence of the adherend comprises:

[0019] constructing a contour set of the pixel unchanged region between all the adjacent video frames, and adding the contour set into a pixel unchanged region area list in sequence;

[0020] calculating a similarity between two adjacent contour sets in the pixel unchanged region area list, and detecting the existence of the adherend based on a similarity calculation result.

[0021] In an implementation form of the first aspect, constructing the contour set of the pixel unchanged region between the adjacent video frames and adding the contour set into the pixel unchanged region area list in sequence comprises:

[0022] extracting a contour of the pixel unchanged region;

[0023] obtaining coordinate information and area of the contour;

[0024] determining whether there is a contour with an area greater than a third preset threshold value;

[0025] if yes, combining all contours with an area greater than the third preset threshold value to form a contour set L i [L i1 ,L i2 ,…,L in ] and adding the contour set L i [L i1 ,L i2 ,…,L in ] to a pixel-invariant region area list, wherein L in represents a contour of an nth pixel-invariant region with an area greater than the third preset threshold value between the i-th set of adjacent video frames, i∈[1,N-1], N is the length of the frame list, and N and n are natural numbers;

[0026] otherwise, constructing a contour set L i [L i1 ,L i2 ,…,L in ] of the pixel-invariant region based on the next set of adjacent video frames.

[0027] In an implementation form of the first aspect, further comprising: determining whether the length of the pixel-invariant region area list exceeds a fourth preset threshold value; if yes, removing the contour set that is added to the pixel-invariant region area list earliest; otherwise, constructing a contour set of the pixel-invariant region based on the next set of adjacent video frames until the length of the pixel-invariant region area list reaches the fourth preset threshold value.

[0028] In an implementation form of the first aspect, calculating the similarity between two adjacent contour sets in the pixel-invariant region area list and detecting the presence of the adhering object based on the similarity calculation result comprises:

[0029] performing consistency matching on contours in the two adjacent contour sets based on the coordinate information of the contours to obtain a plurality of matched contour pairs;

[0030] calculating the similarity between the contours in each contour pair;

[0031] determining whether the similarity is greater than a fifth preset threshold value for Q consecutive times, wherein Q is a natural number;

[0032] if yes, determining that there is an adhering object on the vehicle-mounted camera;

[0033] Otherwise, it is determined that there is no attachment on the vehicle-mounted camera.

[0034] In a second aspect, the present application provides a vehicle-mounted camera attachment detection system, comprising: a video frame acquisition module, configured to acquire continuous video frames collected by a vehicle-mounted camera and save them to a frame list; a preprocessing module, configured to preprocess each video frame in the frame list; a difference detection module, configured to perform difference detection on all adjacent video frames after preprocessing and identify a pixel-invariant region between the adjacent video frames based on the difference detection result; and a similarity comparison module, configured to perform multi-frame similarity comparison on the pixel-invariant region to detect the presence of an attachment.

[0035] In a third aspect, the present application provides a terminal, comprising: a processor and a memory; the memory is configured to store a computer program; and the processor is configured to execute the computer program stored in the memory, so that the terminal executes the vehicle-mounted camera attachment detection method described in any of the above aspects.

[0036] As described above, the vehicle-mounted camera attachment detection method, system and terminal of the present application have the following beneficial effects:

[0037] (1) High-precision detection: the present application utilizes the dynamic difference between continuous video frames and combines multi-frame similarity comparison of the pixel-invariant region, which can effectively distinguish normal environmental changes from abnormal changes caused by attachments. Even in complex dynamic environments, it can maintain a high accuracy, thereby significantly improving the accuracy of detection.

[0038] (2) Strong real-time performance: the detection method of the present application is simple and efficient, supports real-time detection during vehicle driving, and can quickly respond to the presence of attachments.

[0039] (3) High robustness: the present application shows good adaptability to various environmental changes. Whether in light changes, different weather conditions (such as rainy or snowy days), or complex traffic scenes, it can maintain a high performance level.

[0040] (4) Low-cost implementation: the present application only relies on the video frame data of the vehicle-mounted camera, without additional sensor equipment, which not only significantly reduces the hardware cost of the system, but also simplifies the system design and maintenance work.

[0041] (5) High reliability: the present application can effectively ensure the clarity of the field of view of the vehicle-mounted camera, ensure the correct perception and decision-making of the automatic driving system and advanced driving assistance system (ADAS), reduce the risk of accidents, and improve the overall driving safety. BRIEF DESCRIPTION OF DRAWINGS

[0042] Fig. 1 shows a schematic diagram of a hardware application scenario of the vehicle camera attachment detection method, system and terminal according to an embodiment of the present application.

[0043] Fig. 2 shows a schematic diagram of a structure of a mobile terminal according to an embodiment of the present application.

[0044] Fig. 3 shows a flowchart of a vehicle camera attachment detection method according to an embodiment of the present application.

[0045] Fig. 4 shows a flowchart of a vehicle camera attachment detection method according to another embodiment of the present application.

[0046] Fig. 5 shows a pixel correspondence diagram of adjacent video frames according to an embodiment of the present application.

[0047] Fig. 6 shows a flowchart of a vehicle camera attachment detection method according to still another embodiment of the present application.

[0048] Fig. 7 shows a flowchart of a vehicle camera attachment detection method according to yet another embodiment of the present application.

[0049] Fig. 8 shows a contour matching diagram of a vehicle camera attachment detection method according to an embodiment of the present application.

[0050] Fig. 9 shows a fisheye diagram of a vehicle camera attachment detection method according to an embodiment of the present application.

[0051] Fig. 10 shows a schematic diagram of a structure of a vehicle camera attachment detection system according to an embodiment of the present application.

[0052] Fig. 11 shows a schematic diagram of a structure of a terminal according to an embodiment of the present application. DETAILED DESCRIPTION

[0053] The present application is described in detail below by specific embodiments. Those skilled in the art can easily understand other advantages and effects of the present application from the disclosure of the present specification. The present application can also be implemented or applied by different specific embodiments, and various modifications or changes can be made to the details in the present specification based on different views and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0054] It should be noted that the diagrams provided in the following embodiments only schematically illustrate the basic concepts of the present application, and the diagrams only show the components related to the present application rather than the number, shape and size of the components in actual implementation. The shapes, number and proportions of the components in actual implementation can be arbitrarily changed, and the layout pattern of the components can be more complex.

[0055] In addition, the descriptions such as "first", "second" and the like in the present application are only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the technical features indicated, or implicitly indicating the number of the technical features indicated. Therefore, the features defined as "first", "second" can explicitly or implicitly include at least one of the features. In addition, the technical solutions of various embodiments can be combined with each other, but it must be based on the realization of ordinary skilled in the art, when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, nor within the protection scope required by the present application.

[0056] The following embodiments of the present application provide a vehicle-mounted camera attachment detection method, system and terminal, which can effectively improve the speed and accuracy of vehicle-mounted camera attachment detection, while reducing the demand for computing resources. The technical solution of the present application is suitable for real-time detection of attachments on the vehicle-mounted camera during vehicle driving.

[0057] Please refer to FIG. 1, which shows a hardware application scene schematic diagram of the vehicle-mounted camera attachment detection method, system and terminal described in the present application in an embodiment. As shown in FIG. 1, the vehicle-mounted monitoring system of a vehicle is usually equipped with vehicle-mounted cameras facing four directions of front, rear, left and right. As the core component of the environmental perception system, the vehicle-mounted camera is widely used to capture key visual information of the surrounding environment of the vehicle, including road conditions, pedestrians and adjacent vehicles, etc., aiming to provide real-time driving environment data for the driver to improve driving safety.

[0058] The vehicle-mounted camera is usually connected with the vehicle-mounted computer system, which is responsible for converting the captured visual information into data that can be understood and applied by the vehicle. For example, these data can be applied in the automatic driving auxiliary system to realize functions such as driving record, blind spot monitoring, lane departure warning, etc.

[0059] During vehicle driving, the continuous dynamic change of the environment causes the attachments (such as raindrops, dust or dirt, etc.) on the vehicle-mounted camera to remain relatively stationary. By detecting the dynamic difference between adjacent video frames and combining the characteristics of the pixel invariance of the attachment area, the present application can effectively identify the foreign matter attached to the vehicle-mounted camera and improve the accuracy and stability of attachment detection, thereby ensuring that the automatic driving system and the advanced driving assistance system (ADAS) obtain high-quality environmental perception data.

[0060] The vehicle-mounted camera attachment detection method provided in the embodiments of the present application can run in similar devices such as mobile terminals and computer terminals. Taking the running on the mobile terminal as an example, Fig. 2 is a hardware structure block diagram of the mobile terminal of the vehicle-mounted camera attachment detection method. As shown in Fig. 2, the mobile terminal can include a processor and a memory. The processor can be a central processing unit, and the memory is used to store data. The mobile terminal in Fig. 2 is only used for example and does not limit the specific structure of the mobile terminal.

[0061] Optionally, the mobile terminal can further include a communication transmission device and an input and output device.

[0062] Optionally, the memory can be used to store computer programs, such as software programs of application software and modules. The memory can include a high-speed random access memory and can further include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory can further include a memory remotely arranged with respect to the processor, which can be connected to the mobile terminal through a network. Examples of the network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0063] Optionally, the communication transmission device can be used to receive or send data via a network, which can include a wireless network provided by a communication provider of the mobile terminal. The communication transmission device can include a NIC (Network Interface Controller), which can be connected to other network devices through a base station so as to communicate with the Internet.

[0064] The technical solutions in the embodiments of the present application will be described in detail below with reference to the drawings in the embodiments of the present application.

[0065] Please refer to Fig. 3, which shows a flowchart of the vehicle-mounted camera attachment detection method in an embodiment of the present application.

[0066] As shown in Fig. 3, the present embodiment provides a vehicle-mounted camera attachment detection method, which includes the following steps S100 to S400.

[0067] In step S100, continuous video frames captured by a vehicle-mounted camera are acquired and saved to a frame list.

[0068] In an embodiment of the present application, acquiring the continuous video frames captured by the vehicle-mounted camera and saving them to the frame list includes: when the vehicle speed exceeds a first preset threshold, controlling the vehicle-mounted camera to capture images according to a preset time interval to acquire the continuous video frames; and saving the continuous video frames to the frame list in sequence.

[0069] Referring to FIG. 4, a flowchart of another embodiment of the vehicle-mounted camera attachment detection method described in the present application is shown.

[0070] In the embodiment of the present application, the first preset threshold is defined as A, and the frame list is [1, 2, …, N]. Only when the speed of the vehicle exceeds the first preset threshold A, the image acquisition process is started, which can effectively avoid invalid image acquisition when the vehicle is stationary or driving at low speed, thereby improving the working efficiency and accuracy of the system.

[0071] It should be noted that the length N of the frame list is a variable parameter, which can be adaptively adjusted according to the change of the vehicle speed. For example, when the vehicle speed is 60 km / h, N can be set to 10 frames. When the vehicle speed is 80 km / h, considering that the road conditions change more rapidly at high speed, in order to capture key road conditions information in time, N can be set to 5 frames. This kind of dynamic adjustment strategy can effectively reduce the amount of calculation and improve the real-time performance.

[0072] The preset time interval can be dynamically calculated according to the vehicle speed and the length N of the frame list, aiming to balance the image quality and the use efficiency of storage space.

[0073] In step S200, each video frame in the frame list is preprocessed.

[0074] In an embodiment of the present application, preprocessing each video frame in the frame list includes the following steps S201 to S203.

[0075] In step S201, the static invariant region in the video frame due to the limitation of the shooting angle of the vehicle-mounted camera is removed.

[0076] Since the installation angle of the vehicle-mounted camera is fixed, its viewing angle is also fixed, which will result in a part of the region being static and invariant in the imaging process of the camera. Taking the front-facing vehicle-mounted camera in the vehicle monitoring system as an example, the camera is responsible for capturing the real-time image in front of the vehicle, but the license plate installed on the front of the vehicle may appear in a fixed position in the image, thereby forming a static invariant region. Such static invariant region is irrelevant to the analysis of the vehicle driving condition and the front road condition, and may even distract the attention of the driver or the system. Therefore, in order to improve the effectiveness of the image data, the static invariant region in the video frame needs to be removed.

[0077] In an embodiment of the present application, a mask technique in the field of computer vision is applied to accurately remove the static regions in the video frames. The mask is similar to a filter, which "removes" the static regions visually by setting the pixel values of the static regions as transparent or filling them with a background color, leaving only the dynamic elements such as road conditions, pedestrians, and other vehicles, thus "removing" the static regions visually.

[0078] It should be noted that the mask technique used in the present application can be replaced by other methods for removing static regions. Any technical means that can achieve similar effects can be applied within the scope of protection of the present application.

[0079] In step S202, the effective regions remaining in the video frames are subjected to a grayscale processing to convert the original color images into grayscale images.

[0080] Specifically, the original video frames are usually color images, and the grayscale processing can convert the values of the three color channels (red, green, and blue, RGB) of each pixel point in the color image into a single grayscale value. The grayscale value is usually obtained by weighted average of the three color channels, and the weight of each channel can be different to reflect the sensitivity of the human eye to different colors. Each pixel in the converted grayscale image has only one luminance value and no longer contains color information, thus greatly reducing the dimension and complexity of the data, reducing the demand for storage resources, improving the image processing speed, and providing convenience for subsequent image analysis and processing tasks.

[0081] In step S203, the grayscale image is subjected to a binarization processing to obtain a black-and-white image.

[0082] In an embodiment of the present application, the regions where the attachments are located usually exhibit lower grayscale values in the grayscale image, which can be due to the material, color, or other characteristics of the attachments that absorb more light. Therefore, after the binarization processing, these regions will appear black. On the contrary, the background of the dynamic regions, such as moving objects or changing lighting conditions, often has higher grayscale values, and thus appears white after binarization.

[0083] Specifically, the binarization processing includes selecting an appropriate threshold value, which can be fixed globally or dynamically adjusted according to different regions of the image. Once the threshold value is determined, all pixels with a grayscale value higher than the threshold value will be set to white, and pixels with a grayscale value lower than the threshold value will be set to black. This binarization representation method makes the boundary between the attachments and the background more clear.

[0084] At step S300, difference detection is performed on all the pre-processed adjacent video frames, and pixel-invariant regions between the adjacent video frames are identified based on the difference detection results.

[0085] Under normal circumstances, consecutive video frames captured by a camera should exhibit consistent dynamic changes, which reflect activities in the scene, such as pedestrians walking, vehicles driving, etc. However, when there is an attachment, such as dust, water marks or other contaminants, on the lens of a vehicle-mounted camera, these elements will interfere with the normal dynamic change pattern. In order to identify and handle this situation, the present application needs to perform difference detection on all the pre-processed adjacent video frames.

[0086] Referring to FIG. 5, a pixel correspondence diagram of adjacent video frames in an embodiment of the present application is shown.

[0087] In an embodiment of the present application, the difference detection on all the pre-processed adjacent video frames, and the identification of pixel-invariant regions between the adjacent video frames based on the difference detection results include: calculating the absolute difference value of the corresponding position pixels between each group of the adjacent video frames; determining whether the absolute difference value is greater than a second preset threshold value; if yes, marking the position of the corresponding pixel as a pixel change region; otherwise, marking the position of the corresponding pixel as a pixel-invariant region.

[0088] In an embodiment of the present application, the absolute difference value is used to quantify the degree of change of the corresponding position pixels between two adjacent video frames. Since the pixel values in the attachment area usually tend to zero, it indicates that the changes between consecutive frames in these regions will be very small or almost no change.

[0089] In order to clearly present the identification results, the pixel change region and the pixel-invariant region can be marked in the latter frame of the adjacent video frames. This marking method can help users quickly understand the dynamic changes in the scene, so as to better identify and analyze key information. In this way, not only the visualization effect of information can be improved, but also an intuitive tool is provided for users to support the instant understanding of dynamic scenes.

[0090] At step S400, multi-frame similarity comparison is performed on the pixel-invariant regions to detect the presence of the attachment.

[0091] In an embodiment of the present application, the multi-frame similarity comparison on the pixel-invariant regions to detect the presence of the attachment includes the following steps S410 and S420.

[0092] At step S410, a contour set of the pixel-invariant regions between all the adjacent video frames is constructed, and the contour set is sequentially added to a pixel-invariant region area list.

[0093] Referring to FIG. 6, a flowchart of another embodiment of the vehicle-mounted camera attachment detection method is shown. According to the aforementioned frame list of the application, N video frames are included in theory, and (N-1) pairs of adjacent video frames are formed in theory. For each pair of adjacent video frames, a contour set of the pixel invariant region between the adjacent video frames is constructed, and the contour set is sequentially added to the pixel invariant region area list, including the following steps S411 and S415.

[0094] In step S411, the contour of the pixel invariant region is extracted.

[0095] Specifically, the contour of the pixel invariant region refers to a closed curve formed by outlining the edges of all pixel invariant regions gathered together in the video frame. The closed curve connects the pixel points located on the boundary between the pixel invariant region and the pixel change region, thereby generating a contour line clearly defining the range of the region.

[0096] In step S412, the coordinate information and the area of the contour are obtained.

[0097] In step S413, it is determined whether there is a contour with an area greater than a third preset threshold.

[0098] In an embodiment of the application, the third preset threshold S is defined, and the value of S can be 500 pixels.

[0099] In step S414, if there is a contour with an area greater than the third preset threshold, all contours with an area greater than the third preset threshold are combined to form a contour set L i [L i1 ,L i2 ,…,L in ], and the contour set L i [L i1 ,L i2 ,…,L in ] is added to the pixel invariant region area list, where L in represents the contour of the pixel invariant region with an area greater than the third preset threshold between the nth pair of adjacent video frames in the ith group, i∈[1,N-1], N is the length of the frame list, and N and n are natural numbers.

[0100] In step S415, if there is no contour with an area greater than the third preset threshold, the contour set L i [L i1 ,L i2 ,…,L in ] is constructed based on the next group of adjacent video frames.

[0101] In the embodiments of the present application, the contours of a plurality of pixel invariant regions of different sizes can be included between a group of adjacent video frames. Considering that small foreign objects (for example, small black dots) do not affect the driving of the vehicle, the contours of pixel invariant regions with an area less than S are not processed in the present application to improve the accuracy and speed of detection.

[0102] In an embodiment of the present application, the pixel invariant region area list is defined as M, and M can be 10 or 20, which can be set according to the vehicle computing power.

[0103] Suppose that video frame N and video frame N-1 are a first group of adjacent video frames, and there are 3 contours of pixel invariant regions with an area greater than S in the first group of adjacent frames, then the contour set corresponding to the first group of adjacent video frames can be represented as L1[L 11 ,L 12 ,L 13 ], and L1 is added to M. Similarly, video frame N-1 and video frame N-2 are a second group of adjacent video frames, and there are 2 contours of pixel invariant regions with an area greater than S in the second group of adjacent frames, then the contour set corresponding to the second group of adjacent video frames can be represented as L2[L 21 ,L 23 ], and L2 is added to M. Video frame N-2 and video frame N-3 are a third group of adjacent video frames, and there are 2 groups of contours meeting the requirements in the third group of adjacent video frames, then the contour set corresponding to the third group of adjacent video frames can be represented as L3[L 32 ,L 33 ], and L3 is added to M. Video frame N-3 and video frame N-4 are a fourth group of adjacent video frames, and there are 2 groups of contours meeting the requirements in the fourth group of adjacent video frames, then the contour set corresponding to the fourth group of adjacent video frames can be represented as L4[L 42 ,L 43 ], and L4 is added to M.

[0104] In an embodiment of the present application, the vehicle-mounted camera attachment detection method provided by the present application further comprises: judging whether the length of the pixel invariant region area list exceeds a fourth preset threshold; if yes, removing the contour set that is added to the pixel invariant region area list earliest; otherwise, constructing the contour set of the pixel invariant region based on the next group of adjacent video frames until the length of the pixel invariant region area list reaches the fourth preset threshold.

[0105] In consideration of the limited computing power of the system, in order to ensure the efficiency and effectiveness of the calculation, the present application implements a strategy of real-time monitoring and management of the pixel-invariant region area list M, so as to maintain the length of the pixel-invariant region area list M within a preset range. When the pixel-invariant region area list M approaches the upper limit of its capacity, the system will automatically clear the contour set that has been in M for the longest time according to the principle of first-in first-out, so as to make room for the necessary storage space for new contour sets. This mechanism not only ensures that the latest contour sets can be processed in time, but also ensures that it will not affect other functions of the vehicle. The present implementation optimizes the use of resources without sacrificing performance, and ensures the efficient operation of the system.

[0106] In step S420, the similarity between two adjacent contour sets in the pixel-invariant region area list is calculated, and the presence of the adherend is detected based on the similarity calculation result.

[0107] Referring to FIG. 7, a flowchart of another embodiment of the vehicle-mounted camera adherend detection method according to the present application is shown.

[0108] As shown in FIG. 7, in an embodiment of the present application, calculating the similarity between two adjacent contour sets in the pixel-invariant region area list and detecting the presence of the adherend based on the similarity calculation result includes steps S421 to S424.

[0109] In step S421, the contours in the two adjacent contour sets are matched for consistency based on the coordinate information of the contours, so as to obtain a plurality of matched contour pairs.

[0110] Referring to FIG. 8, a contour matching schematic diagram of an embodiment of the vehicle-mounted camera adherend detection method according to the present application is shown.

[0111] As shown in FIG. 8, according to the coordinate information of L1[L 11 ,L 12 ,L 13 ] and L2[L 21 ,L 23 ], it can be determined that L 11 and L 21 are closest, and L 13 and L 23 are closest. Therefore, it is determined that L 11 and L 21 are matched with each other, and can form a contour pair (L 11 , L 21 ); L 13 and L 23 are matched with each other, and can form a contour pair (L 13 , L 23 ); since there is no contour in L2 that is matched with L12 The coordinates of L and L are closest, thus L and L cannot form a contour pair.

[0112] Similarly, according to the coordinate information of L2[L 21 ,L 23 ] and L3[L 32 ,L 33 ], it can be determined that L 23 and L 33 are closest, thus it is determined that L 23 and L 33 match each other and can form a contour pair (L 23 , L 33 ). According to the coordinate information of L3[L 32 ,L 33 ] and L4[L 42 ,L 43 ], it can be determined that L 32 and L 42 are closest, and L 33 and L 43 are closest. Thus it is determined that L 32 and L 42 match each other and form a contour pair (L 32 , L 42 ); L 33 and L 43 match each other and form a contour pair (L 33 , L 43 ).

[0113] In step S422, the similarity between the contours in each contour pair is calculated.

[0114] In an embodiment of the present application, the similarity between the contours in each contour pair can be calculated by any similarity measurement method. The similarity measurement method includes but is not limited to Structural Similarity Index (SSIM), cosine similarity algorithm, histogram-based algorithm and Mutual Information (MI).

[0115] In step S423, it is determined whether the similarity is greater than a fifth preset threshold for Q consecutive times, where Q is a natural number.

[0116] In an embodiment of the present application, the fifth preset threshold is defined as F, and the value of F can be 90%.

[0117] It should be noted that the fifth preset threshold F is a variable parameter, and can be dynamically adjusted according to environmental conditions. For example, F can be set to 80% in rainy days.

[0118] In this implementation, different environmental changes can be more flexibly adapted, the sensitivity of system detection is improved, the effectiveness and reliability of the detection process are guaranteed, and high recognition accuracy can be maintained under adverse weather conditions.

[0119] In step S424, if the similarity is greater than the fifth preset threshold value for Q consecutive times, it is determined that there is an adherend on the vehicle-mounted camera; otherwise, it is determined that there is no adherend on the vehicle-mounted camera.

[0120] In an embodiment of the present application, the similarity between L 11 and L 21 is 70%, the similarity between L 13 and L 23 is 90%, the similarity between L 23 and L 33 is 91%, the similarity between L 33 and L 43 is 92%, the value of Q is 3, and the value of the fifth preset threshold value F is 90%.

[0121] Since the similarity between L 13 and L 23 , L 33 and L 43 exceeds 90% for three consecutive frames, it can be determined that the pixel invariant region corresponding to L 13 , L 23 , L 33 and L 43 has an adherend.

[0122] In this implementation, by continuously observing and evaluating the change of the similarity, environmental interference can be effectively excluded, and the pixel loss caused by the adherend can be accurately detected, thereby improving the accuracy and reliability of the adherend detection in the video frame.

[0123] In an embodiment of the present application, the vehicle-mounted camera adherend detection method provided by the present application further includes: mapping the coordinates of the detected adherend to a fisheye image for visualization. Please refer to FIG. 9, which shows the fisheye image of the vehicle-mounted camera adherend detection method according to an embodiment of the present application.

[0124] As a special form of panoramic image, the fisheye image can show a wider viewing angle than the traditional camera capture range. Therefore, by marking the exact position of the adherend on the fisheye image, it can help maintenance personnel quickly locate and remove the adherend, ensuring that the camera can maintain the best operating state, providing convenience for vehicle maintenance and research, and providing strong data support for the decision-making process of the automatic driving system.

[0125] After identifying the camera attachment, the application further has the function of triggering an alarm or starting a cleaning measure. For example, once the presence of the attachment is detected, the system can automatically activate a cleaning program to ensure the cleanliness of the camera lens, thereby maintaining the clarity and accuracy of the monitoring video. This function not only improves the autonomous maintenance capability of the camera system, but also ensures the continuous high quality of the monitoring data, which is crucial for ensuring vehicle driving safety and improving the decision-making efficiency of the autonomous driving system.

[0126] It should be noted that the protection scope of the vehicle-mounted camera attachment detection method described in the embodiments of the application is not limited to the execution order of the steps listed in the embodiments. Any scheme achieved by adding, replacing or modifying the steps of the prior art according to the principles of the application is included in the protection scope of the application.

[0127] Please refer to FIG. 10, which shows a structural schematic diagram of a vehicle-mounted camera attachment detection system according to an embodiment of the application.

[0128] As shown in FIG. 10, the application provides a vehicle-mounted camera attachment detection system, which includes a video frame acquisition module, a preprocessing module, a difference detection module and a similarity comparison module.

[0129] The video frame acquisition module is configured to acquire continuous video frames captured by a vehicle-mounted camera and save them to a frame list.

[0130] The preprocessing module is configured to preprocess each video frame in the frame list.

[0131] The difference detection module is configured to perform difference detection on all adjacent video frames after preprocessing and identify the pixel-invariant region between the adjacent video frames based on the difference detection result.

[0132] The similarity comparison module is configured to perform multi-frame similarity comparison on the pixel-invariant region to detect the presence of the attachment.

[0133] It should be noted that the structure and principles of the video frame acquisition module, the preprocessing module, the difference detection module and the similarity comparison module described in the embodiments of the application correspond one-to-one to the steps in the vehicle-mounted camera attachment detection method described above, and therefore will not be described here.

[0134] The vehicle-mounted camera attachment detection system provided in the embodiments of the application can implement the vehicle-mounted camera attachment detection method described in the application, but the implementation device of the vehicle-mounted camera attachment detection method described in the application includes but is not limited to the structure of the vehicle-mounted camera attachment detection system listed in the embodiments. Any structural deformation and replacement of the prior art according to the principles of the application is included in the protection scope of the application.

[0135] Referring to FIG. 11, a structural schematic diagram of an embodiment of the terminal is shown.

[0136] As shown in FIG. 11, the application provides a terminal, comprising a processor and a memory.

[0137] The memory is configured to store a computer program.

[0138] The processor is configured to execute the computer program stored in the memory, so that the terminal executes the vehicle-mounted camera attachment detection method described in any one of the above.

[0139] Preferably, the processor can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; and can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The memory can be realized by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0140] The embodiment also includes one or more of a multimedia component, an input / output (I / O) interface, and a communication component.

[0141] The multimedia component can include a screen and audio components. The screen can be a touch screen, for example. The audio components can be for output and / or input of audio signals. For example, the audio components can include a microphone for receiving an external audio signal. The received audio signal can be further stored in the memory or transmitted via the communication component. The audio components can further include at least one speaker for outputting audio signals. The I / O interface provides an interface between the processor and other interface modules, which can be a keyboard, a mouse, a button, etc. The buttons can be virtual buttons or physical buttons. The communication component is for wired or wireless communication between the timer and other devices. The wireless communication, for example, Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more of them, so the corresponding communication component can include a Wi-Fi module, a Bluetooth module, an NFC module.

[0142] In an embodiment, the timer can be implemented by one or more Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor or other electronic elements for executing the above-mentioned methods of deploying and cooperating cache for UAV network.

[0143] In several embodiments provided in the present application, it should be understood that the disclosed system, device or method can be implemented by other ways. For example, the above-described device embodiments are only illustrative, for example, the division of modules / units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of modules or units can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed each other can be through some interfaces, indirect coupling or communication connection between devices or modules or units, which can be electrical, mechanical or other forms.

[0144] The modules / units described as separated components can or can not be physically separated, and the components shown as modules / units can or can not be physical modules, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules / units can be selected according to actual needs to achieve the purposes of the embodiments of the present application. For example, the functional modules / units in each embodiment of the present application can be integrated in one processing module, or each module / unit can be physically present separately, or two or more modules / units can be integrated in one module / unit.

[0145] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in general terms above. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0146] The embodiments of the present application also provide a computer readable storage medium. Those of ordinary skill in the art can understand that all or part of the steps of the methods described above can be completed by a processor instructed by a program, and the program can be stored in a computer readable storage medium. The storage medium is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid state disk, a magnetic tape, a floppy disk, an optical disc, and any combination thereof. The storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid state disk (SSD)), etc.

[0147] The embodiments of the present application can also provide a computer program product including one or more computer instructions. When the computer instructions are loaded and executed on a computing device, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, the computer instructions can be transmitted from one website site, computer or data center to another website site, computer or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.).

[0148] When the computer program product is executed by a computer, the computer executes the method described in the foregoing method embodiments. The computer program product can be a software installation package, and when the foregoing method needs to be used, the computer program product can be downloaded and executed on the computer.

[0149] The description of the corresponding processes or structures of each of the above figures has its own emphasis, and the parts not described in detail in a certain process or structure can be referred to the related description of other processes or structures.

[0150] In summary, the present application provides a vehicle-mounted camera attachment detection method, system and terminal, the method comprising: acquiring continuous video frames collected by a vehicle-mounted camera and saving them to a frame list; preprocessing each video frame in the frame list; performing difference detection on all adjacent video frames after preprocessing, and identifying a pixel-invariant region between the adjacent video frames based on the difference detection result; and performing multi-frame similarity comparison on the pixel-invariant region to detect the presence of an attachment. By utilizing the dynamic difference between continuous frames and combining multi-frame similarity comparison of the pixel-invariant region, the present application can effectively distinguish between normal environmental changes and abnormal changes caused by attachments. Even in a complex dynamic environment, it can maintain a high accuracy, thereby significantly improving the accuracy of detection. The detection method of the present application is simple and efficient, supports real-time detection during vehicle driving, and can quickly respond to the presence of attachments, thereby ensuring driving safety. The present application shows good adaptability to various environmental changes. Whether it is light changes, different weather conditions (such as rainy or snowy days), or complex traffic scenes, it can maintain a high performance level. The present application only relies on video frame data of the vehicle-mounted camera, without the need for additional sensor equipment. This not only significantly reduces the hardware cost of the system, but also simplifies system design and maintenance work. The present application can effectively ensure the clarity of the field of view of the vehicle-mounted camera, ensure the correct perception and decision-making of the autonomous driving system and advanced driver assistance system (ADAS), reduce the risk of accidents, and improve overall driving safety.

[0151] The above embodiments are only illustrative of the principles of the present application and its effects, and are not intended to limit the present application. Any modification or change made by any person skilled in the art without departing from the spirit and scope of the present application shall be covered by the claims of the present application.

Claims

1. A method of detecting an attachment of a car camera, characterized by, The method comprises the following steps: acquiring continuous video frames collected by a vehicle-mounted camera and saving the video frames to a frame list; preprocessing each video frame in the frame list; performing difference detection on all adjacent video frames after preprocessing, and identifying pixel-invariant regions between the adjacent video frames based on the difference detection result; performing multi-frame similarity comparison on the pixel-invariant regions to detect the presence of an adherend.

2. The method of claim 1, wherein, The step of acquiring continuous video frames collected by a vehicle-mounted camera and saving the video frames to a frame list comprises the following steps: when the vehicle speed exceeds a first preset threshold, controlling the vehicle-mounted camera to collect images at a preset time interval to acquire the continuous video frames; saving the continuous video frames to the frame list in sequence.

3. The method of claim 1, wherein, The step of preprocessing each video frame in the frame list comprises the following steps: removing static-invariant regions in the video frame which are collected due to the shooting angle limitation of the vehicle-mounted camera; performing grayscale processing on the effective region remaining in the video frame to convert the original color image into a grayscale image; performing binaryzation processing on the grayscale image to obtain a black-and-white image.

4. The method of claim 1, wherein, The step of performing difference detection on all adjacent video frames after preprocessing, and identifying pixel-invariant regions between the adjacent video frames based on the difference detection result comprises the following steps: calculating the absolute difference value of the pixels at the corresponding positions between each group of adjacent video frames; judging whether the absolute difference value is greater than a second preset threshold; if yes, marking the position of the corresponding pixel as a pixel change region; otherwise, marking the position of the corresponding pixel as a pixel-invariant region.

5. The method of claim 1, wherein, The step of performing multi-frame similarity comparison on the pixel-invariant regions to detect the presence of an adherend comprises the following steps: constructing a contour set of the pixel-invariant regions between all adjacent video frames, and sequentially adding the contour set to a pixel-invariant region area list; calculating the similarity between two adjacent contour sets in the pixel-invariant region area list, and detecting the presence of the adherend based on the similarity calculation result.

6. The method of claim 5, wherein, The step of constructing a contour set of the pixel-invariant regions between adjacent video frames, and sequentially adding the contour set to a pixel-invariant region area list comprises the following steps: extracting the contour of the pixel-invariant region; acquiring the coordinate information and area of the contour; judging whether there is a contour with an area greater than a third preset threshold; If yes, combine all the contours with area greater than the third preset threshold to form a contour set L i [L i1 ,L i2 ,…,L in ] and add the contour set L i [L i1 ,L i2 ,…,L in ] to a pixel-invariant region area list, wherein L in represents a contour of the pixel-invariant region with the n-th area greater than the third preset threshold between the i-th set of adjacent video frames, i∈[1,N-1], N is the length of the frame list, and N and n are both natural numbers. Otherwise, based on the next set of said adjacent video frames, a set of contours L of said pixel invariant region is constructed i [L i1 ,L i2 ,…,L in ] 7. The method of claim 6, wherein, The method further comprises the following steps: judging whether the length of the pixel-invariant region area list exceeds a fourth preset threshold; if yes, removing the contour set that is added to the pixel-invariant region area list earliest; otherwise, constructing a contour set of the pixel-invariant region based on the next group of adjacent video frames until the length of the pixel-invariant region area list reaches the fourth preset threshold.

8. The method of claim 5, wherein, The step of calculating the similarity between two adjacent contour sets in the pixel-invariant region area list, and detecting the presence of the adherend based on the similarity calculation result comprises the following steps: based on the coordinate information of the contour, performing consistency matching on the contours in the two adjacent contour sets to obtain a plurality of contour pairs that match with each other; calculating the similarity between the contours in each contour pair; judging whether the similarity is greater than a fifth preset threshold for Q consecutive times, wherein Q is a natural number; If yes, it is determined that there is an attachment on the vehicle-mounted camera; Otherwise, it is determined that there is no attachment on the vehicle-mounted camera.

9. A system for detecting an attachment of a vehicle camera, the system comprising: The method comprises the steps of: acquiring a continuous video frame collected by a vehicle-mounted camera and saving the video frame to a frame list; preprocessing each video frame in the frame list; performing difference detection on all adjacent video frames after preprocessing, and identifying a pixel-invariant region between the adjacent video frames based on a difference detection result; performing multi-frame similarity comparison on the pixel-invariant region to detect the presence of an attachment.

10. A terminal, characterized by comprising: The method comprises the steps of: a processor and a memory; the memory is used to store a computer program; the processor is used to execute the computer program stored in the memory, so that the terminal executes the vehicle-mounted camera attachment detection method in any one of claims 1 to 8.

Images