Shielding detection method and device, vehicle, medium and program product

Abstract

The invention relates to a shielding detection method and device, a vehicle, a medium and a program product, which can be applied to the technical field of automatic driving, and the method comprises the following steps: continuously shooting a target object through a shooting device to obtain a plurality of target images; for each target image, first similar feature information of the target image is determined, second similar feature information of the target image and a target image adjacent to the target image is determined, the first similar feature information is used for representing the target image shot by the shooting device, and the second similar feature information is used for representing the target image shot by the shooting device; the second similar feature information is used for representing the similarity between the target image and the adjacent target image, and the second similar feature information is used for representing the similarity between the target image and the adjacent target image; according to the first similar feature information and the second similar feature information, a shielding detection result of the shooting device is determined, and the shielding detection result is used for representing whether the shooting device is completely shielded or not when shooting the target image.

Description

Technical Field

[0001] This disclosure relates to the field of autonomous driving technology, and in particular to an occlusion detection method, device, vehicle, medium, and program product. Background Technology

[0002] Intelligent driving systems rely on sensors to collect vehicle and road information, enabling the control system to make correct control decisions. Cameras are crucial sensors in autonomous driving, responsible for perceiving various scene elements such as vehicles, lane markings, and traffic lights. However, human error or accidental lens obstruction can severely interfere with the onboard cameras' acquisition of road information, affecting the stability and even safety of the intelligent driving system.

[0003] Existing occlusion detection methods typically determine the presence of occlusion in an image based on the pixel difference between the foreground and background. However, when the camera lens is completely obscured, the image captured by the camera will appear almost entirely black, which is very similar to a camera image taken in a low-visibility scene at night without streetlights. The aforementioned occlusion detection methods cannot effectively distinguish between a completely black occluded image and a nighttime image, thus failing to achieve accurate detection of image occlusion. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides an occlusion detection method, device, vehicle, medium, and program product.

[0005] According to a first aspect of the present disclosure, an occlusion detection method is provided, comprising:

[0006] Multiple target images are obtained by continuously capturing images of the target object using an imaging device;

[0007] For each target image, a first similarity feature information of the target image is determined, and a second similarity feature information of the target image and its neighboring target images is determined. The first similarity feature information is used to characterize the similarity between the target image captured by the shooting device and the image captured by the shooting device when it is completely occluded. The second similarity feature information is used to characterize the similarity between the target image and the neighboring target images.

[0008] Based on the first similarity feature information and the second similarity feature information, the occlusion detection result of the shooting device is determined, and the occlusion detection result is used to characterize whether the shooting device is completely occluded when shooting the target image.

[0009] Optionally, determining the first similarity feature information of the target image and the second similarity feature information between the target image and its neighboring target images for each target image includes:

[0010] Feature sampling is performed on the target image to obtain a first fused feature map corresponding to the target image;

[0011] For each target image, the first similarity feature information and the second similarity feature information are determined from the first fused feature map.

[0012] Optionally, the step of performing feature sampling on the target image to obtain the first fused feature map corresponding to the target image includes:

[0013] The target image is uniformly sampled to obtain a sampled feature map;

[0014] By recombining the sampled feature map, a first fused feature map corresponding to the target image is obtained.

[0015] Optionally, determining the first similarity feature information includes:

[0016] Occlusion detection is performed on the first fused feature map to obtain the first similarity feature information of the target image.

[0017] Optionally, the step of performing occlusion detection on the first fused feature map to obtain the first similarity feature information of the target image includes:

[0018] The first difference is determined based on the feature matrix of the first fused feature map and the preset black-grayscale feature matrix;

[0019] Determine the first modulus and the first average value of the first difference;

[0020] Based on the first modulus and the first average value, occlusion detection is performed on the first fused feature map to obtain the first similarity feature information of the target image.

[0021] Optionally, determining the second similarity feature information includes:

[0022] Determine the second fusion feature map of the adjacent target images;

[0023] Based on the first fusion feature map and the second fusion feature map, second similarity feature information between the target image and its neighboring target images is determined.

[0024] Optionally, determining the second similarity feature information between the target image and its neighboring target images based on the first fusion feature map and the second fusion feature map includes:

[0025] The second difference is determined based on the feature matrix of the first fused feature map and the feature matrix of the second fused feature map;

[0026] Determine the second modulus and the second average of the second difference;

[0027] Based on the second modulus and the second average value, the second similarity feature information between the target image and its neighboring target images is determined.

[0028] Optionally, determining the occlusion detection result of the shooting device based on the first similarity feature information and the second similarity feature information includes:

[0029] If the first similarity feature information is less than or equal to the first preset threshold, and the second similarity feature information is less than or equal to the second preset threshold, it is determined that the shooting device is completely occluded when shooting the target image.

[0030] According to a second aspect of the present disclosure, a camera detection apparatus is provided, comprising:

[0031] The acquisition module is configured to continuously capture images of the target object using a shooting device to obtain multiple target images;

[0032] The first determining module is configured to, for each target image, determine first similarity feature information of the target image and second similarity feature information of the target image and its neighboring target images. The first similarity feature information is used to characterize the similarity between the target image captured by the shooting device and an image captured by the shooting device when it is completely occluded. The second similarity feature information is used to characterize the similarity between the target image and its neighboring target images.

[0033] The second determining module is configured to determine the occlusion detection result of the shooting device based on the first similarity feature information and the second similarity feature information, wherein the occlusion detection result is used to characterize whether the shooting device is completely occluded when shooting the target image.

[0034] Optionally, the first determining module includes:

[0035] The sampling submodule is configured to perform feature sampling on the target image to obtain a first fused feature map corresponding to the target image;

[0036] The determination submodule is configured to determine the first similarity feature information and the second similarity feature information for each target image corresponding to the first fused feature map.

[0037] Optionally, the sampling submodule is configured to uniformly sample the target image to obtain a sampling feature map; and to reconstruct the data from the sampling feature map to obtain a first fused feature map corresponding to the target image.

[0038] Optionally, the determining submodule is configured to perform occlusion detection on the first fused feature map to obtain the first similarity feature information of the target image.

[0039] Optionally, the determining submodule is configured to determine a first difference based on the feature matrix of the first fused feature map and a preset black-grayscale feature matrix; determine a first modulus and a first average value of the first difference; and perform occlusion detection on the first fused feature map based on the first modulus and the first average value to obtain the first similarity feature information of the target image.

[0040] Optionally, the determining submodule is configured to determine a second fusion feature map of the adjacent target images; and to determine second similarity feature information between the target image and its adjacent target images based on the first fusion feature map and the second fusion feature map.

[0041] Optionally, the determining submodule is configured to determine a second difference based on the feature matrix of the first fused feature map and the feature matrix of the second fused feature map; determine a second modulus and a second average value of the second difference; and determine the second similarity feature information of the target image and its neighboring target images based on the second modulus and the second average value.

[0042] Optionally, the second determining module is configured to determine that the shooting device is completely occluded when shooting the target image if the first similar feature information is less than or equal to a first preset threshold and the second similar feature information is less than or equal to a second preset threshold.

[0043] According to a third aspect of the present disclosure, a vehicle is provided, comprising:

[0044] processor;

[0045] Memory used to store processor-executable instructions;

[0046] The processor is configured as follows:

[0047] Multiple target images are obtained by continuously capturing images of the target object using an imaging device;

[0048] For each target image, a first similarity feature information of the target image is determined, and a second similarity feature information of the target image and its neighboring target images is determined. The first similarity feature information is used to characterize the similarity between the target image captured by the shooting device and the image captured by the shooting device when it is completely occluded. The second similarity feature information is used to characterize the similarity between the target image and the neighboring target images.

[0049] Based on the first similarity feature information and the second similarity feature information, the occlusion detection result of the shooting device is determined, and the occlusion detection result is used to characterize whether the shooting device is completely occluded when shooting the target image.

[0050] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, having stored thereon computer program instructions that, when executed by a processor, implement the steps of the occlusion detection method provided in the first aspect of the present disclosure.

[0051] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the occlusion detection method provided in the first aspect of the present disclosure.

[0052] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0053] Multiple target images are obtained by continuously capturing images of a target object using a shooting device. For each target image, a first similarity feature information is determined, and a second similarity feature information is determined between the target image and its neighboring target images. The first similarity feature information characterizes the similarity between the target image captured by the shooting device and an image captured when the shooting device is completely occluded. The second similarity feature information characterizes the similarity between the target image and its neighboring target images. Based on the first and second similarity feature information, an occlusion detection result of the shooting device is determined, which characterizes whether the shooting device was completely occluded when capturing the target image. In this way, for any target image acquired, based on the first similarity feature determined with the image captured when the shooting device is completely occluded, the second similarity feature with the dynamic target image of the preceding and following frames can be further considered. Static detection and dynamic detection are fused during detection, taking into account both the static features of the current frame and the dynamic features of the adjacent periods. This avoids the problem that a single detection algorithm cannot effectively distinguish between completely black occluded images and nighttime images. It can detect the occlusion of the vehicle camera lens in real time and accurately, thus improving the accuracy of image occlusion detection.

[0054] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0055] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0056] Figure 1 This is a flowchart illustrating an occlusion detection method according to an exemplary embodiment.

[0057] Figure 2 It is based on Figure 1 The illustrated embodiment shows a flowchart of step S12.

[0058] Figure 3 This is a schematic diagram illustrating a sampled image according to an exemplary embodiment.

[0059] Figure 4 This is a block diagram illustrating a camera detection device according to an exemplary embodiment.

[0060] Figure 5 It is based on Figure 4 The illustrated embodiment shows a block diagram of a first determining module.

[0061] Figure 6 This is a block diagram illustrating a vehicle according to an exemplary embodiment. Detailed Implementation

[0062] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0063] The embodiments described in the following examples of this disclosure are not representative of all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0064] It should be noted that all actions involving the acquisition of signals, information, or data in this disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with authorization from the owner of the relevant device.

[0065] Before detailing the specific embodiments of this disclosure, its application scenarios will first be explained. This disclosure can be applied to scenarios involving occlusion detection of vehicle-mounted cameras. Currently, sensors can collect vehicle and road information to enable the control system to make correct control decisions and achieve intelligent driving. Cameras are crucial sensors in autonomous driving, responsible for perceiving various scene elements such as vehicles, lane lines, and traffic lights. However, human-caused or accidental lens occlusion can severely interfere with the acquisition of road information by vehicle-mounted cameras, affecting the stability and even safety of the intelligent driving system. Therefore, a discriminative feature detection algorithm is typically used to identify camera dirt / occlusion. This algorithm usually determines whether occlusion exists in the image based on the pixel difference between the foreground and background, and performs a function degradation operation when dirt / occlusion is detected to avoid driving risks.

[0066] However, existing occlusion detection methods produce images that are nearly completely black when the camera lens is completely blocked. This is very similar to images taken in low-visibility nighttime scenes without streetlights. Therefore, these occlusion detection methods cannot effectively distinguish between completely black occluded images and nighttime images, and thus cannot achieve accurate detection of image occlusion.

[0067] To overcome the technical problems existing in the above-mentioned related technologies, this disclosure provides an occlusion detection method, device, vehicle, medium, and program product. The method involves continuously capturing images of a target object using an imaging device to obtain multiple target images. For each target image, a first similarity feature information is determined, and a second similarity feature information is determined between the target image and its neighboring target images. The first similarity feature information characterizes the similarity between the target image captured by the imaging device and an image captured when the imaging device is completely occluded. The second similarity feature information characterizes the similarity between the target image and its neighboring target images. Based on the first and second similarity feature information, an occlusion detection result is determined, which characterizes whether the imaging device was completely occluded when capturing the target image. In this way, for any target image acquired, based on the first similarity feature determined with the image captured when the shooting device is completely occluded, the second similarity feature with the dynamic target image of the preceding and following frames can be further considered. Static detection and dynamic detection are fused during detection, taking into account both the static features of the current frame and the dynamic features of the adjacent periods. This avoids the problem that a single detection algorithm cannot effectively distinguish between completely black occluded images and nighttime images. It can detect the occlusion of the vehicle camera lens in real time and accurately, thus improving the accuracy of image occlusion detection.

[0068] The present disclosure will now be described in conjunction with specific embodiments.

[0069] Figure 1 This is a flowchart illustrating an occlusion detection method according to an exemplary embodiment, such as... Figure 1 As shown, the occlusion detection method is applied to electronic devices. These electronic devices can be ordinary computers, servers, or smart terminal devices, or in-vehicle terminals such as vehicle-mounted computers or industrial personal computers (IPCs). The method may include the following steps.

[0070] In step S11, the target object is continuously photographed by the imaging device to obtain multiple target images.

[0071] The shooting device may include a vehicle-mounted camera, such as a rangefinder RGB camera, fisheye camera, or binocular camera, etc., which are not limited in this disclosure. The target object may be any object in a preset scene at a preset distance from the shooting device.

[0072] In this step, during the process of photographing the target object using the imaging device, the target object can be photographed continuously according to a preset shooting cycle to obtain multiple consecutive target images. These target images can be captured in real-time during the vehicle's movement, or they can be images captured and stored by the imaging device.

[0073] In step S12, for each target image, the first similarity feature information of the target image is determined, and the second similarity feature information of the target image and its neighboring target images is determined.

[0074] The first similarity feature information is used to characterize the similarity between the target image captured by the shooting device and the image captured by the shooting device when it is completely obscured, and the second similarity feature information is used to characterize the similarity between the target image and the adjacent target image.

[0075] Optionally, any one of the multiple images captured by the shooting device can be used as the target image. However, considering the safety of intelligent driving, the image with the closest capture time to the current moment is usually used as the target image. This can ensure real-time occlusion detection of the shooting device and improve the safety of intelligent driving.

[0076] Furthermore, after obtaining the real-time target image, the feature vector of the target image can be obtained first. Then, based on the feature matrix of a typical black occlusion and the feature vector of the target image, the first similarity feature information between the target image and the image captured by the shooting device when it is completely occluded can be determined. Based on the first similarity feature information, it can be determined whether the shooting device is completely occluded when capturing the target image.

[0077] Furthermore, the previous frame of the acquired target image can be used as the neighboring target image. Then, based on the feature vector of the target image and the feature vector of the neighboring target image, the second similarity feature information between the target image and the neighboring target image can be determined, and the similarity between the target image and the neighboring target image can be determined based on the second similarity feature information.

[0078] In step S13, the occlusion detection result of the shooting device is determined based on the first similarity feature information and the second similarity feature information.

[0079] The occlusion detection result is used to characterize whether the shooting device is completely occluded when capturing the target image.

[0080] Using the above technical solution, multiple target images can be obtained by continuously capturing images of a target object using a shooting device. For each target image, a first similarity feature information of the target image is determined, and a second similarity feature information of the target image and its neighboring target images is determined. The first similarity feature information is used to characterize the similarity between the target image captured by the shooting device and an image captured by the shooting device when it is completely occluded. The second similarity feature information is used to characterize the similarity between the target image and its neighboring target images. Based on the first and second similarity feature information, an occlusion detection result of the shooting device is determined. The occlusion detection result is used to characterize whether the shooting device is completely occluded when capturing the target image. In this way, for any target image acquired, based on the first similarity feature determined with the image captured when the shooting device is completely occluded, the second similarity feature with the dynamic target image of the preceding and following frames can be further considered. Static detection and dynamic detection are fused during detection, taking into account both the static features of the current frame and the dynamic features of the adjacent periods. This avoids the problem that a single detection algorithm cannot effectively distinguish between completely black occluded images and nighttime images. It can detect the occlusion of the vehicle camera lens in real time and accurately, thus improving the accuracy of image occlusion detection.

[0081] In some embodiments, such as Figure 2 As shown, step S12 above may include the following steps.

[0082] In step S121, feature sampling is performed on the target image to obtain the first fused feature map corresponding to the target image.

[0083] Optionally, the target image can be uniformly sampled first to obtain a sampled feature map, and then the first fused feature map corresponding to the target image can be obtained by recombining the data of the sampled feature map.

[0084] For example, when using the camera image of the k-th frame as the target image, the camera image of the k-th frame can first be uniformly sampled to extract a sampled feature map. For instance, the camera image of the k-th frame can be represented as 3×W×H, where W is the number of pixels in the width dimension, H is the number of pixels in the height dimension, and 3 is the number of color channels. Sampling is performed at a sampling rate S to obtain a 3×N×M dimensional image, where N = W / S and M = H / S. Figure 3 As shown, the white box represents the sampled pixels. Taking the width dimension as an example, assuming W = 90 and S = 10, the width dimension coordinates of the sampled points are S × i + S / 2, i = 0 ~ (N-1), N = 9, that is, 5, 15, 25, 35, 45, 55, 65, 75, 85, a total of 9 points. The height dimension coordinates of the sampled points can be calculated in the same way, which will not be elaborated here.

[0085] The sampled 3D image data is then recombined into a 2D feature matrix F[k][i,j], where i = 1 ~ X, j = 1 ~ Y, to facilitate feature calculation. The dimensions of X are 3, and the dimensions of Y are N × M.

[0086] By adopting the above technical solution, the image storage space is reduced by S×S ​​times after sampling, which can significantly reduce storage and computation pressure.

[0087] In step S122, for each target image corresponding to the first fused feature map, the first similarity feature information is determined, and the second similarity feature information is determined.

[0088] In some embodiments, occlusion detection can be performed on the first fused feature map to obtain the first similarity feature information of the target image.

[0089] Optionally, a first difference can be determined based on the feature matrix of the first fused feature map and a preset black-grayscale feature matrix. Then, based on the first modulus and the first average value of the first difference, occlusion detection is performed on the first fused feature map to obtain the first similarity feature information of the target image.

[0090] For example, based on the first modulus and the first average value, occlusion detection can be performed on the first fused feature map using a discriminative feature detection formula to obtain the first similarity feature information of the target image. The discriminative feature detection formula may include:

[0091] T ＝ sum(F[K][i , j]-B[i , j]) / X / Y;

[0092] Where T represents the first similarity feature information, F[k][i,j] represents the feature matrix of the first fused feature map, B[i,j] represents the black grayscale feature matrix, X represents the number of horizontal pixels in the first fused feature map, and Y represents the number of vertical pixels in the first fused feature map.

[0093] By adopting the above technical solution, the first similarity feature information between the target image captured by the shooting device and the image captured when the shooting device is completely blocked can be determined while reducing storage and computing pressure. Based on the first similarity feature information, it can be determined whether the shooting device was completely blocked when capturing the target image.

[0094] In other embodiments, a second fusion feature map of the adjacent target images may also be determined.

[0095] In this step, the same steps can be used to uniformly sample the adjacent target images to obtain a sampled feature map. Then, the second fused feature map corresponding to the adjacent target images can be obtained by recombining the data from the sampled feature map.

[0096] Furthermore, based on the first fusion feature map and the second fusion feature map, a second similarity feature information between the target image and its neighboring target images can be determined.

[0097] Optionally, a second difference can be determined based on the feature matrix of the first fused feature map and the feature matrix of the second fused feature map; then, a second modulus and a second average value of the second difference can be determined; and based on the second modulus and the second average value, a second similarity feature information between the target image and its neighboring target images can be determined.

[0098] For example, based on the second modulus and the second average value, similarity feature detection can be performed on the first fused feature map of the target image and the second fused feature map of the adjacent target image using a similarity feature detection formula to obtain the second similarity feature information between the target image and the adjacent target images. The similarity feature detection formula may include:

[0099] Q＝ sum(F[K][i , j]-F[KP][i , j]) / X / Y;

[0100] Where Q represents the second similarity feature information, F[k][i,j] represents the feature matrix of the first fused feature map, F[kP][i,j] represents the feature matrix of the second fused feature map, X represents the number of horizontal pixels in the first fused feature map, and Y represents the number of vertical pixels in the first fused feature map.

[0101] By adopting the above technical solution, the second similarity feature information between the target image and the adjacent target image in the previous frame can be determined while reducing storage and computing pressure, and the similarity between the target image and the adjacent target image can be determined based on the second similarity feature information.

[0102] In some embodiments, if the first similarity feature information is less than or equal to a first preset threshold and the second similarity feature information is less than or equal to a second preset threshold, it can be determined that the shooting device is completely occluded when shooting the target image.

[0103] The first and second preset thresholds can be set based on experience and results, and are usually set to smaller values, such as Q≤2 and T≤5.

[0104] Figure 4 This is a block diagram illustrating a camera detection device according to an exemplary embodiment. (Refer to...) Figure 4 The camera detection device 200 includes an acquisition module 201, a first determination module 202, and a second determination module 203.

[0105] The acquisition module 201 is configured to continuously capture images of the target object using the imaging device to obtain multiple target images;

[0106] The first determining module 202 is configured to determine, for each target image, a first similarity feature information of the target image and a second similarity feature information of the target image and its neighboring target images. The first similarity feature information is used to characterize the similarity between the target image captured by the capturing device and an image captured by the capturing device when it is completely occluded. The second similarity feature information is used to characterize the similarity between the target image and its neighboring target images.

[0107] The second determining module 203 is configured to determine the occlusion detection result of the shooting device based on the first similarity feature information and the second similarity feature information. The occlusion detection result is used to characterize whether the shooting device is completely occluded when shooting the target image.

[0108] Figure 5 It is based on Figure 4 The illustrated embodiment shows a block diagram of a first determining module, as follows: Figure 5 As shown, the first determining module 202 includes:

[0109] The sampling submodule 2021 is configured to perform feature sampling on the target image to obtain the first fused feature map corresponding to the target image;

[0110] The determination submodule 2022 is configured to determine the first similarity feature information and the second similarity feature information for each target image corresponding to the first fused feature map.

[0111] Optionally, the sampling submodule 2021 is configured to uniformly sample the target image to obtain a sampled feature map; and to reconstruct the data from the sampled feature map to obtain a first fused feature map corresponding to the target image.

[0112] Optionally, the determining submodule 2022 is configured to perform occlusion detection on the first fused feature map to obtain the first similarity feature information of the target image.

[0113] Optionally, the determining submodule 2022 is configured to determine a first difference based on the feature matrix of the first fused feature map and a preset black-grayscale feature matrix; determine a first modulus and a first average value of the first difference; and perform occlusion detection on the first fused feature map based on the first modulus and the first average value to obtain the first similarity feature information of the target image.

[0114] Optionally, the determining submodule 2022 is configured to determine a second fusion feature map of the adjacent target image; and determine second similarity feature information between the target image and its adjacent target images based on the first fusion feature map and the second fusion feature map.

[0115] Optionally, the determining submodule 2022 is configured to determine a second difference based on the feature matrix of the first fused feature map and the feature matrix of the second fused feature map; determine a second modulus and a second average value of the second difference; and determine second similarity feature information between the target image and its neighboring target images based on the second modulus and the second average value.

[0116] Optionally, the second determining module 203 is configured to determine that the shooting device is completely occluded when shooting the target image if the first similarity feature information is less than or equal to the first preset threshold and the second similarity feature information is less than or equal to the second preset threshold.

[0117] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0118] Using the above-described device, multiple target images are obtained by continuously capturing images of a target object. For each target image, a first similarity feature information is determined, and a second similarity feature information is determined between the target image and its neighboring target images. The first similarity feature information is used to characterize the similarity between the target image captured by the capturing device and an image captured when the capturing device is completely occluded. The second similarity feature information is used to characterize the similarity between the target image and its neighboring target images. Based on the first and second similarity feature information, an occlusion detection result of the capturing device is determined. The occlusion detection result is used to characterize whether the capturing device was completely occluded when capturing the target image. In this way, for any target image acquired, based on the first similarity feature determined with the image captured when the shooting device is completely occluded, the second similarity feature with the dynamic target image of the preceding and following frames can be further considered. Static detection and dynamic detection are fused during detection, taking into account both the static features of the current frame and the dynamic features of the adjacent periods. This avoids the problem that a single detection algorithm cannot effectively distinguish between completely black occluded images and nighttime images. It can detect the occlusion of the vehicle camera lens in real time and accurately, thus improving the accuracy of image occlusion detection.

[0119] Figure 6 This is a block diagram illustrating a vehicle 300 according to an exemplary embodiment. For example, vehicle 300 may be a hybrid vehicle, a non-hybrid vehicle, an electric vehicle, a fuel cell vehicle, or other types of vehicle. Vehicle 300 may be an autonomous vehicle, a semi-autonomous vehicle, or a non-autonomous vehicle.

[0120] Reference Figure 6 The vehicle 300 may include various subsystems, such as an infotainment system 310, a perception system 320, a decision control system 330, a drive system 340, and a computing platform 350. The vehicle 300 may also include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and component of the vehicle 300 can be interconnected via wired or wireless means.

[0121] In some embodiments, the infotainment system 310 may include a communication system, an entertainment system, and a navigation system, etc.

[0122] The perception system 320 may include several sensors for sensing information about the environment surrounding the vehicle 300. For example, the perception system 320 may include a global positioning system (which may be GPS, BeiDou, or other positioning systems), an inertial measurement unit (IMU), lidar, millimeter-wave radar, ultrasonic radar, and a camera device.

[0123] The decision control system 330 may include a computing system, a vehicle controller, a steering system, a throttle, and a braking system.

[0124] The drive system 340 may include components that provide powered motion to the vehicle 300. In one embodiment, the drive system 340 may include an engine, an energy source, a transmission system, and wheels. The engine may be one or a combination of internal combustion engines, electric motors, and compressed air engines. The engine is capable of converting energy provided by the energy source into mechanical energy.

[0125] Some or all of the functions of the vehicle 300 are controlled by a computing platform 350. The computing platform 350 may include at least one processor 351 and a memory 352, the processor 351 being able to execute instructions 353 stored in the memory 352.

[0126] Processor 351 can be any conventional processor, such as a commercially available CPU. The processor may also include, for example, a Graphics Processing Unit (GPU), a Field Programmable Gate Array (FPGA), a System on Chip (SOC), an Application Specific Integrated Circuit (ASIC), or a combination thereof.

[0127] The memory 352 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0128] In addition to instruction 353, memory 352 can also store data, such as road maps, route information, vehicle position, direction, speed, and other data. The data stored in memory 352 can be used by computing platform 350.

[0129] In this embodiment of the disclosure, the processor 351 may execute instruction 353 to complete all or part of the steps of the above-described occlusion detection method.

[0130] This disclosure also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the occlusion detection method provided in this disclosure.

[0131] In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable device, the computer program having a code portion for performing the occlusion detection method described above when executed by the programmable device.

[0132] Furthermore, the term “exemplary” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “exemplary” is not necessarily to be construed as advantageous compared to other aspects or designs. Rather, the use of the term “exemplary” is intended to present the concept in a concrete manner. As used herein, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise specified or clear from the context, “X applies A or B” is intended to mean any of the natural inclusive arrangements. That is, “X applies A or B” satisfies any of the foregoing instances if X applies A; X applies B; or both X applies A and B. Additionally, unless otherwise specified or clear from the context to refer to the singular form, the articles “a” and “an” as used in this application and the appended claims are generally understood to mean “one or more.”

[0133] Similarly, although this disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art upon reading and understanding the specification and drawings. This disclosure includes all such modifications and variations and is limited only by the scope of the claims. In particular, with respect to the various functions performed by the components described above (e.g., elements, resources, etc.), unless otherwise indicated, the terminology used to describe such components is intended to correspond to any component (functionally equivalent) that performs the specific function of the described component, even if structurally not equivalent to the disclosed structure. Furthermore, although specific features of this disclosure may have been disclosed with respect to only one of several implementations, such features may be combined with one or more other features of other implementations, as may be desired and advantageous to any given or particular application. Moreover, with regard to the terms “comprising,” “owning,” “having,” “having,” or variations thereof as used in the detailed description or claims, such terms are intended to be inclusive in a manner similar to the term “including.”

[0134] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

[0135] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

[0136] It should be understood that, unless otherwise specifically indicated, features of various embodiments of this disclosure described herein can be combined with each other. As used herein, the term “and / or” includes any one of the relevant listed items and any combination of any two or more; similarly, “at least one of…” includes any one of the relevant listed items and any combination of any two or more.

[0137] It should be understood that, unless otherwise expressly specified and limited, the terms "joining," "attaching," "installing," "connecting," "linking," "fixing," etc., used in the embodiments of this disclosure should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection, an electrical connection, or a connection that allows communication between them; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise expressly limited. Those skilled in the art can understand the specific meaning of the above terms herein based on the specific circumstances.

[0138] Although terms such as “first,” “second,” and “third” may be used herein to describe various components, parts, regions, layers, or sections, these components, parts, regions, layers, or sections are not limited to these terms. Rather, these terms are used only to distinguish one component, part, region, layer, or section from another. Therefore, without departing from the teachings of the examples described herein, the first component, part, region, layer, or section mentioned in the examples may also be referred to as the second component, part, region, layer, or section. Furthermore, the terms “first” and “second” are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as “first” or “second” may explicitly or implicitly include at least one of that feature. In the description herein, “a plurality” means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. An occlusion detection method, characterized in that, include: Multiple target images are obtained by continuously capturing images of the target object using an imaging device; For each target image, a first similarity feature information of the target image is determined, and a second similarity feature information of the target image and its neighboring target images is determined. The first similarity feature information is used to characterize the similarity between the target image captured by the shooting device and the image captured by the shooting device when it is completely occluded. The second similarity feature information is used to characterize the similarity between the target image and the neighboring target images. Based on the first similarity feature information and the second similarity feature information, the occlusion detection result of the shooting device is determined, and the occlusion detection result is used to characterize whether the shooting device is completely occluded when shooting the target image.

2. The method according to claim 1, characterized in that, For each target image, determining the first similarity feature information of the target image and determining the second similarity feature information between the target image and its neighboring target images includes: Feature sampling is performed on the target image to obtain a first fused feature map corresponding to the target image; For each target image, the first similarity feature information and the second similarity feature information are determined from the first fused feature map.

3. The method according to claim 2, characterized in that, The step of performing feature sampling on the target image to obtain the first fused feature map corresponding to the target image includes: The target image is uniformly sampled to obtain a sampled feature map; By recombining the sampled feature map, a first fused feature map corresponding to the target image is obtained.

4. The method according to claim 2, characterized in that, Determining the first similarity feature information includes: Occlusion detection is performed on the first fused feature map to obtain the first similarity feature information of the target image.

5. The method according to claim 4, characterized in that, The step of performing occlusion detection on the first fused feature map to obtain the first similarity feature information of the target image includes: The first difference is determined based on the feature matrix of the first fused feature map and the preset black-grayscale feature matrix; Determine the first modulus and the first average value of the first difference; Based on the first modulus and the first average value, occlusion detection is performed on the first fused feature map to obtain the first similarity feature information of the target image.

6. The method according to claim 2, characterized in that, Determining the second similarity feature information includes: Determine the second fusion feature map of the adjacent target images; Based on the first fusion feature map and the second fusion feature map, the second similarity feature information of the target image and the neighboring target images of the target image is determined.

7. The method according to claim 6, characterized in that, The step of determining the second similarity feature information between the target image and its neighboring target images based on the first fusion feature map and the second fusion feature map includes: The second difference is determined based on the feature matrix of the first fused feature map and the feature matrix of the second fused feature map; Determine the second modulus and the second average of the second difference; Based on the second modulus and the second average value, the second similarity feature information between the target image and its neighboring target images is determined.

8. The method according to any one of claims 1-7, characterized in that, The step of determining the occlusion detection result of the shooting device based on the first similarity feature information and the second similarity feature information includes: If the first similarity feature information is less than or equal to the first preset threshold, and the second similarity feature information is less than or equal to the second preset threshold, it is determined that the shooting device is completely occluded when shooting the target image.

9. A camera detection device, characterized in that, include: The acquisition module is configured to continuously capture images of the target object using a shooting device to obtain multiple target images; The first determining module is configured to, for each target image, determine first similarity feature information of the target image and second similarity feature information of the target image and its neighboring target images. The first similarity feature information is used to characterize the similarity between the target image captured by the shooting device and an image captured by the shooting device when it is completely occluded. The second similarity feature information is used to characterize the similarity between the target image and its neighboring target images. The second determining module is configured to determine the occlusion detection result of the shooting device based on the first similarity feature information and the second similarity feature information, wherein the occlusion detection result is used to characterize whether the shooting device is completely occluded when shooting the target image.

10. A vehicle, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured as follows: Multiple target images are obtained by continuously capturing images of the target object using an imaging device; For each target image, a first similarity feature information of the target image is determined, and a second similarity feature information of the target image and its neighboring target images is determined. The first similarity feature information is used to characterize the similarity between the target image captured by the shooting device and the image captured by the shooting device when it is completely occluded. The second similarity feature information is used to characterize the similarity between the target image and the neighboring target images. Based on the first similarity feature information and the second similarity feature information, the occlusion detection result of the shooting device is determined, and the occlusion detection result is used to characterize whether the shooting device is completely occluded when shooting the target image.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1-8.

12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-8.

Images