Image sensor protection method and device, camera module and storage medium

Abstract

The invention provides an image sensor protection method and device, a camera module and a storage medium, and relates to the technical field of cameras, and the method comprises the steps: obtaining at least three first abnormal bright regions in an image sensor, and the at least three first abnormal bright regions at least have a time sequence relation; based on the target spatial-temporal features corresponding to the at least three first abnormal bright regions, determining a first predicted bright region in the image sensor; adjusting the light intensity of the first predicted bright area based on a first preset mapping relation; the first preset mapping relation comprises a mapping relation between the regional brightness and the light transmittance. The invention can weaken the intensity of light irradiated to the image sensor by strong light and prevent the image sensor from being damaged.

Description

Technical Field

[0001] The present invention relates to the technical field of cameras, and in particular, to an image sensor protection method, device, camera module, and storage medium. Background Art

[0002] The image sensor of a camera mainly receives light through photosensitive elements and converts the light into an electrical signal for imaging. The laser emitted by a lidar is a high-intensity light beam. If it directly irradiates on the image sensor of the camera, due to the excessive intensity of the received light, it will cause the charge of the photosensitive element to saturate, and may even burn out the circuit, resulting in damage to the image sensor. For example, when the camera takes an image after shooting a vehicle-mounted lidar, there are two straight lines, one horizontal and one vertical, in the image, which is a manifestation of the image sensor being burned out by the lidar. Therefore, how to reduce the damage of strong light such as laser to the image sensor is an urgent problem to be solved at present. Summary of the Invention

[0003] The present invention provides an image sensor protection method, device, camera module, and storage medium, which are used to solve the defect of how to reduce the damage of strong light such as laser to the image sensor in the prior art, reduce the light intensity of the strong light irradiating on the image sensor, and prevent damage to the image sensor.

[0004] The present invention provides an image sensor protection method, including:

[0005] Obtain at least three first abnormally bright areas in the image sensor, and there is at least a temporal relationship among the at least three first abnormally bright areas;

[0006] Determine a first predicted bright area in the image sensor based on the target spatio-temporal characteristics corresponding to each of the at least three first abnormally bright areas;

[0007] Adjust the light intensity of the first predicted bright area based on a first preset mapping relationship; the first preset mapping relationship includes the mapping relationship between the regional brightness and the light transmittance.

[0008] According to the image sensor protection method provided by the present invention, the adjusting the light intensity of the first predicted bright area based on the first preset mapping relationship includes:

[0009] Obtain the regional coordinates corresponding to at least two dimming mechanisms;

[0010] Match the spatial prediction positions of the first predicted bright area with the regional coordinates corresponding to each of the dimming mechanisms respectively, and determine the target dimming mechanism corresponding to the first predicted bright area, where the target dimming mechanism is disposed on the image sensor;

[0011] Match the brightness distribution prediction feature of the first predicted bright area with the regional brightness in the first preset mapping relationship to determine the target light transmittance corresponding to the target dimming mechanism;

[0012] Adjust the light intensity of the first predicted bright area based on the target light transmittance corresponding to the target dimming mechanism.

[0013] According to the image sensor protection method provided by the present invention, the adjusting the light intensity of the first predicted bright area based on the target light transmittance corresponding to the target dimming mechanism includes:

[0014] Based on a second preset mapping relationship and the target light transmittance corresponding to the target dimming mechanism, determine the target voltage value corresponding to the target dimming mechanism, where the target voltage value is used to adjust the light intensity of the first predicted bright area corresponding to the target dimming mechanism; the second preset mapping relationship includes the mapping relationship between the light transmittance and the voltage value.

[0015] According to the image sensor protection method provided by the present invention, the determining the first predicted bright area in the image sensor based on the target spatio-temporal features corresponding to each of the at least three first abnormal bright areas includes:

[0016] Perform clustering based on the target spatio-temporal features of the at least three first abnormal bright areas to determine at least one clustering set;

[0017] Based on the sub-target spatio-temporal features in each of the clustering sets, determine the sub-first predicted bright areas corresponding to each of the clustering sets;

[0018] Based on all the sub-first predicted bright areas, determine the first predicted bright area in the image sensor.

[0019] According to the image sensor protection method provided by the present invention, the target spatio-temporal features at least include a target spatial position, a target brightness distribution feature, and a target time feature;

[0020] The performing clustering based on the target spatio-temporal features of the at least three first abnormal bright areas to determine at least one clustering set includes:

[0021] S1. Determine at least one first initial clustering set among the at least three first abnormal bright areas; each of the first initial clustering sets includes at least two first abnormal bright areas with the smallest target time feature and having a time sequence relationship;

[0022] S2. Based on the first abnormal bright areas with the same target time feature, construct at least three second initial clustering sets;

[0023] S3. For each of the second initial clustering sets, based on the target spatial positions, target brightness distribution characteristics, and target time characteristics corresponding to the first abnormal bright regions in each of the second initial clustering sets, construct an initial feature matrix corresponding to the second initial clustering set; each column feature type in the initial feature matrix is the same, and each row feature belongs to the same first abnormal bright region;

[0024] S4. Based on the initial feature matrices corresponding to the first target abnormal bright region and the second target abnormal bright region respectively, and the at least one first initial clustering set, determine the distance interval difference, brightness distribution similarity difference, and time interval difference between the first target abnormal bright region and the second target abnormal bright region; the first target abnormal bright region is any one of the first abnormal bright regions whose belonging first initial clustering set has not been determined, and the second target abnormal bright region is a first abnormal bright region whose belonging first initial clustering set has been determined; the first target abnormal bright region and the second target abnormal bright region are adjacent, and the target time characteristic of the second target abnormal bright region is less than the target time characteristic of the first target abnormal bright region;

[0025] S5. When the distance interval difference is less than or equal to a first preset threshold, the brightness distribution similarity difference is greater than or equal to a second preset threshold, and the time interval difference is less than or equal to a third preset threshold, add the first target abnormal bright region to the first initial clustering set corresponding to the second target abnormal bright region;

[0026] S6. Repeat steps S2 to S5, and stop the iteration when the stop condition is reached, and determine the at least one clustering set.

[0027] According to the image sensor protection method provided by the present invention, the sub-target spatio-temporal characteristics include sub-target spatial position, sub-target brightness distribution characteristics, and sub-target time characteristics;

[0028] The determining the sub-first predicted bright regions corresponding to the clustering sets based on the sub-target spatio-temporal characteristics in the clustering sets includes:

[0029] For each of the clustering sets, based on the change trend of the sub-target spatial positions of all the sub-first abnormal bright regions in the clustering set, determine the sub-spatial prediction position of the sub-first predicted bright region corresponding to the clustering set;

[0030] Based on the change trend of the sub-target brightness distribution characteristics of all the sub-first abnormal bright regions in the clustering set, determine the sub-brightness distribution prediction characteristics of the sub-first predicted bright region corresponding to the clustering set;

[0031] Based on the change trend of the sub-target time characteristics of all sub-first abnormally bright areas in the clustering set, determine the sub-predicted time of the sub-first predicted bright area corresponding to the clustering set, where the sub-predicted time is used to indicate adjusting the light intensity of the first predicted bright area before the sub-predicted time.

[0032] According to the image sensor protection method provided by the present invention, after adjusting the light intensity of the first predicted bright area, the method further includes:

[0033] Obtain at least one second abnormally bright area in the image sensor;

[0034] In the case where the at least one second abnormally bright area does not match the first predicted bright area, update the light transmittance of the dimming mechanism corresponding to each of the at least one second abnormally bright area and the first predicted bright area to adjust the light intensity corresponding to each of the at least one second abnormally bright area and the first predicted bright area.

[0035] The present invention also provides an image sensor protection device, including:

[0036] An acquisition module, configured to acquire at least three first abnormally bright areas in the image sensor, where there is at least a timing relationship among the at least three first abnormally bright areas;

[0037] A determination module, configured to determine a first predicted bright area in the image sensor based on the target spatio-temporal characteristics corresponding to each of the at least three first abnormally bright areas;

[0038] An adjustment module, configured to adjust the light intensity of the first predicted bright area based on a first preset mapping relationship; the first preset mapping relationship includes a mapping relationship between regional brightness and light transmittance.

[0039] The present invention also provides a camera module, including at least two dimming mechanisms, an image sensor, a memory, a processor, and a computer program stored on the memory and executable on the processor;

[0040] Each of the dimming mechanisms includes oppositely arranged conductive layer blocks and a liquid crystal dimming film, the oppositely arranged conductive layer blocks are connected to the processor, and the liquid crystal dimming film is disposed between the oppositely arranged conductive layer blocks;

[0041] All the dimming mechanisms are spliced and disposed on the image sensor, and the image sensor is connected to the processor;

[0042] When the processor executes the program, it implements the image sensor protection method as described in any one of the above.

[0043] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the image sensor protection method described in any one of the above is implemented.

[0044] The image sensor protection method, device, camera module and storage medium provided by the present invention extract the target spatio-temporal features corresponding to at least three first abnormal bright areas with a timing relationship in the image sensor, predict the first predicted bright area where the lidar may perform the next strong light scan, and match according to the area brightness in the first preset mapping relationship with the first predicted bright area. Before the lidar performs a strong light scan on the first predicted bright area, the light transmittance of the first predicted bright area is adjusted in advance, thereby weakening the light intensity of the strong light rays of the lidar irradiating the first predicted bright area, protecting the image sensor in real time, and preventing damage to the image sensor. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0046] Figure 1 is a schematic flowchart of the image sensor protection method provided by an embodiment of the present invention;

[0047] Figure 2 is a schematic diagram of the scanning trajectory line of the M-line scanning method provided by an embodiment of the present invention;

[0048] Figure 3 is a schematic diagram of the structure of at least two dimming mechanisms provided by an embodiment of the present invention;

[0049] Figure 4 is a schematic diagram of the second abnormal bright area provided by an embodiment of the present invention;

[0050] Figure 5 is a schematic diagram of the structure of the image sensor protection device provided by an embodiment of the present invention;

[0051] Figure 6 is a schematic diagram of the structure of the camera module provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0052] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0053] Regarding the problem of how to reduce the damage of strong light such as laser to an image sensor, in the prior art, on the one hand, an anti-laser filter can be added in front of the camera lens, which can effectively reduce the damage of the laser to the image sensor. However, the ability of the anti-laser filter to reduce the laser intensity is fixed. Due to different laser models, different anti-laser filters need to be configured in front of the lens according to the laser type, and the versatility is poor. On the other hand, an image sensor with an anti-laser damage function can be used. An automatic light intensity adjustment circuit or a special material that can withstand high-intensity light is adopted in the image sensor, but the cost is high.

[0054] Therefore, in view of the above problems, an embodiment of the present invention provides a method for protecting an image sensor. Figure 1 It is a schematic flow chart of the method for protecting an image sensor provided by an embodiment of the present invention. As Figure 1 shown, the method includes:

[0055] Step 110, obtain at least three first abnormally bright areas in the image sensor, and there is at least a timing relationship among the at least three first abnormally bright areas.

[0056] Optionally, the image sensor uses the photoelectric conversion function of a photoelectric device to convert the optical signal on the photosensitive element into an electrical signal proportional to the optical signal for imaging. The image sensor may include a CMOS (Complementary Metal-Oxide-Semiconductor) sensor or a CCD (Charge Coupled Device) sensor. The embodiments of the present invention do not limit this.

[0057] It should be noted that the first abnormally bright area is generated when the high-intensity light beam emitted by the lidar irradiates the image sensor. When the lidar irradiates the image sensor for a long time, it affects indicators such as the unit charge transfer rate and charge collection efficiency in the image sensor, resulting in charge saturation or even overflow, thereby causing damage to the image sensor. The damage to the image sensor may include spot damage, line damage, or cross-shaped line-plane damage, etc. In addition, the damage situation of the image sensor is related to the irradiation duration of the lidar. Short-time laser irradiation will generate a first abnormally bright area with strong light characteristics, and long-time laser irradiation will cause damage to the image sensor.

[0058] Optionally, the models of the above lidar may include AT128 or XT16, etc. The power consumption of the AT128 lidar is 18W, and the single-point power within 1 second is 0.117mW; the power consumption of the XT16 lidar is 9W, and the single-point power within 1 second is 0.281mW. To ensure the safe use of the image sensor, the CIPA (Camera & Imaging Products Association) image sensor safety standard recommends using a lidar with a laser power not exceeding 50mW; the ASIS (American Society for Testing and Materials) image sensor safety standard recommends using a lidar with a laser power not exceeding 10mW; the ISO (International Organization for Standardization) image sensor safety standard recommends using a lidar with a laser power not exceeding 1mW.

[0059] Optionally, the scanning method of the above lidar may include a single-line scanning method or a multi-line scanning method. When the scanning method of the lidar is the single-line scanning method, the image sensor can detect at least three first abnormally bright areas on a single scanning trajectory line, and the appearance order of the at least three first abnormally bright areas has a sequence, and they are arranged in sequence along the scanning direction of the lidar. When the scanning method of the lidar is the multi-line scanning method, there are at least three first abnormally bright areas on each scanning trajectory line. For example, Figure 2 is a schematic diagram of the scanning trajectory line of the M-line scanning method provided by an embodiment of the present invention. As Figure 2 shown, when the lidar is in the M-line scanning method, the image sensor can simultaneously detect M first abnormally bright areas in the target direction perpendicular to the scanning direction of the lidar, but the M first abnormally bright areas appear simultaneously, and at least three first abnormally bright areas can be detected in the scanning direction of the lidar. The appearance order of the at least three first abnormally bright areas has a sequence, and M is an integer greater than 1.

[0060] Step 120: Determine a first predicted bright area in the image sensor based on the target spatio-temporal characteristics corresponding to the at least three first abnormally bright areas.

[0061] Specifically, after obtaining at least three first abnormally bright areas with a timing relationship, the target spatio-temporal characteristics corresponding to each first abnormally bright area can be extracted, the scanning rule of the lidar can be analyzed and determined, and then according to this scanning rule, the first predicted bright area that may appear during lidar scanning can be predicted, that is, the next abnormally bright area with strong light characteristics that may be scanned during laser scanning.

[0062] Further, determining the first predicted bright area in the image sensor based on the target spatio-temporal features respectively corresponding to the at least three first abnormally bright areas includes:

[0063] Performing clustering on the target spatio-temporal features of the at least three first abnormally bright areas to determine at least one clustering set;

[0064] Determining the sub-first predicted bright areas corresponding to the respective clustering sets based on the sub-target spatio-temporal features in the respective clustering sets;

[0065] Determining the first predicted bright area in the image sensor based on all the sub-first predicted bright areas.

[0066] Specifically, after extracting the target spatio-temporal features corresponding to each first abnormally bright area, using a clustering algorithm, clustering all the first abnormally bright areas according to all the target spatio-temporal features to obtain at least one clustering set. Each clustering set represents a scanning trajectory line of the lidar. At least three sub-first abnormally bright areas included in each clustering set are abnormally bright areas that are on the same scanning trajectory line, have a temporal sequence relationship, and are distributed along the scanning direction. When the lidar is in a single-line scanning mode, one clustering set is obtained after clustering. When the lidar is in an M-line scanning mode, M clustering sets are obtained after clustering. For example, as Figure 2 shown, when the lidar is in an M-line scanning mode, M clustering sets are obtained after clustering. Each clustering set constitutes a scanning trajectory line, and each clustering set includes multiple sub-first abnormally bright areas distributed along the scanning direction. After clustering, for each clustering set, extract the sub-target spatio-temporal features of each sub-first abnormally bright area in each clustering set. According to all the sub-target spatio-temporal features in the same clustering set, determine the scanning rule corresponding to this scanning trajectory line, and then perform prediction according to this scanning rule to obtain the sub-first predicted bright area. This sub-first predicted bright area can be understood as the next abnormally bright area that may be scanned by the lidar when scanning this scanning trajectory line. For example, as Figure 2 shown, taking the first scanning trajectory line as an example, respectively extract the sub-target spatio-temporal features corresponding to each sub-first abnormally bright area from sub-first abnormally bright area 1 to sub-first abnormally bright area 4 on the first scanning trajectory line. According to the 4 sub-target spatio-temporal features, the sub-first predicted bright area 5 on the first scanning trajectory line can be predicted. After the sub-first predicted bright area is predicted for each scanning trajectory line, all the sub-first predicted bright areas constitute the first predicted bright area, that is, this first predicted bright area includes at least one sub-first predicted bright area. It should be noted that this sub-target spatio-temporal feature is the above-mentioned target spatio-temporal feature. The target spatio-temporal feature is the feature corresponding to each first abnormally bright area before clustering, while the sub-target spatio-temporal feature is the feature corresponding to each sub-first abnormally bright area in each clustering set after clustering. This sub-first abnormally bright area belongs to the first abnormally bright area.

[0067] Furthermore, the target spatio-temporal features at least include a target spatial position, a target brightness distribution feature, and a target time feature;

[0068] Performing clustering based on the target spatio-temporal features of the at least three first abnormally bright regions to determine at least one clustering set, including:

[0069] S1. Determine at least one first initial clustering set among the at least three first abnormally bright regions; each of the first initial clustering sets includes at least two first abnormally bright regions with the smallest target time feature and having a temporal relationship;

[0070] S2. Based on each of the first abnormally bright regions with the same target time feature, construct at least three second initial clustering sets;

[0071] S3. For each of the second initial clustering sets, based on the target spatial position, the target brightness distribution feature, and the target time feature corresponding to each of the first abnormally bright regions in the second initial clustering set, construct an initial feature matrix corresponding to the second initial clustering set; each column feature type in the initial feature matrix is the same, and each row feature belongs to the same first abnormally bright region;

[0072] S4. Based on the initial feature matrices corresponding to the first target abnormally bright region and the second target abnormally bright region respectively, and the at least one first initial clustering set, determine the distance interval difference, the brightness distribution similarity difference, and the time interval difference between the first target abnormally bright region and the second target abnormally bright region; the first target abnormally bright region is any one of the first abnormally bright regions whose belonging first initial clustering set has not been determined, and the second target abnormally bright region is a first abnormally bright region whose belonging first initial clustering set has been determined; the first target abnormally bright region is adjacent to the second target abnormally bright region, and the target time feature of the second target abnormally bright region is less than the target time feature of the first target abnormally bright region;

[0073] S5. In the case where the distance interval difference is less than or equal to a first preset threshold, the brightness distribution similarity difference is greater than or equal to a second preset threshold, and the time interval difference is less than or equal to a third preset threshold, add the first target abnormally bright region to the first initial clustering set corresponding to the second target abnormally bright region;

[0074] S6. Repeat steps S2 to S5, and stop iterating when a stop condition is reached, and determine the at least one clustering set.

[0075] Exemplarily, such as Figure 2As shown, taking the example of obtaining 4M first abnormally bright areas where the target spatio-temporal features include the target spatial position, the target brightness distribution feature, and the target time feature, first, when the scanning mode of the lidar is the M-line scanning mode, M first initial clustering sets are determined. Each first initial clustering set includes two first abnormally bright areas with the smallest target time feature and having a temporal relationship. For example, the first initial clustering set corresponding to the first row includes the first abnormally bright area 1 and the first abnormally bright area 2 with the earliest sorted target time feature. According to the target spatial positions corresponding to the first abnormally bright area 1 and the first abnormally bright area 2 respectively, the reference distance interval between the first abnormally bright area 1 and the first abnormally bright area 2 can be determined; according to the target brightness distribution features corresponding to the first abnormally bright area 1 and the first abnormally bright area 2 respectively, the reference brightness distribution similarity between the first abnormally bright area 1 and the first abnormally bright area 2 can be determined; according to the target time features corresponding to the first abnormally bright area 1 and the first abnormally bright area 2 respectively, the reference time interval between the first abnormally bright area 1 and the first abnormally bright area 2 can be determined. At the same time, according to the first abnormally bright areas with the same target time feature, 4 second initial clustering sets are constructed. Each second initial clustering set includes Figure 2 the M first abnormally bright areas shown in the same column in

[0076] After that, for each second initial clustering set, the target spatial position, the target brightness distribution feature, and the target time feature of each first abnormally bright area in the second initial clustering set are respectively extracted, and an initial feature matrix corresponding to the second initial clustering set is constructed. The initial feature matrix is shown in Equation (1), and Equation (1) is:

[0077]

[0078] where Q k represents the initial feature matrix corresponding to the k-th second initial clustering set, that is, the initial feature matrix corresponding to the k-th column second initial clustering set. i represents the i-th row first abnormally bright area, and 1 ≤ i ≤ M. j represents the j-th target spatio-temporal feature, j = 1, 2, 3. When j = 1, x ki1 represents the target spatial position corresponding to the i-th row first abnormally bright area in the initial feature matrix corresponding to the k-th second initial clustering set. When j = 2, x ki2 represents the target brightness distribution feature corresponding to the i-th row first abnormally bright area in the initial feature matrix corresponding to the k-th second initial clustering set. When j = 3, x ki3 represents the target time feature corresponding to the i-th row first abnormally bright area in the initial feature matrix corresponding to the k-th second initial clustering set.

[0079] After determining the initial feature matrices corresponding to the four second initial clustering sets, taking i = 1 as an example, where the first target abnormal bright area is the first abnormal bright area 3 in the first row of the third second initial clustering set, and the second target abnormal bright area is the first abnormal bright area 2 in the first row of the second second initial clustering set, the differences corresponding to each type of feature are determined according to the target spatio-temporal features at the same position and of the same type in the initial feature matrix Q2 and the initial feature matrix Q3. Specifically, it includes: according to the target spatial position x in the initial feature matrix Q2 211 and the target spatial position x in the initial feature matrix Q3 311 , the distance interval between the first abnormal bright area 3 and the first abnormal bright area 2 is determined. Then, the absolute value of the difference between the distance interval and the reference distance interval corresponding to the first abnormal bright area 2 is calculated to determine the distance interval difference corresponding to the first abnormal bright area 3. According to the target brightness distribution feature x in the initial feature matrix Q2 212 and the target brightness distribution feature x in the initial feature matrix Q3 312 , the brightness distribution similarity between the first abnormal bright area 3 and the first abnormal bright area 2 is determined. Then, the absolute value of the difference between the brightness distribution similarity and the reference brightness distribution similarity corresponding to the first abnormal bright area 2 is calculated to determine the brightness distribution similarity difference corresponding to the first abnormal bright area 3. According to the target time feature x in the initial feature matrix Q2 213 and the target time feature x in the initial feature matrix Q3 313 , the time interval between the first abnormal bright area 3 and the first abnormal bright area 2 is determined. Then, the absolute value of the difference between the time interval and the reference time interval corresponding to the first abnormal bright area 2 is calculated to determine the time interval difference corresponding to the first abnormal bright area 3. After determining the distance interval difference, the brightness distribution similarity difference, and the time interval difference, the distance interval difference is respectively compared with the first preset threshold, the brightness distribution similarity difference is compared with the second preset threshold, and the time interval difference is compared with the third preset threshold. If the distance interval difference is less than or equal to the first preset threshold, the brightness distribution similarity difference is greater than or equal to the second preset threshold, and the time interval difference is less than or equal to the third preset threshold, the first abnormal bright area 3 can be added to the first initial clustering set corresponding to the first abnormal bright area 2. Then, continue to judge the first abnormal bright area 4 and the first abnormal bright area 3 until there is no new first abnormal bright area, or the new first abnormal bright area exceeds the detection range of the image sensor, and then stop the iteration, and the updated M clustering sets can be obtained.

[0080] It should be noted that the target time feature can be understood as the time stamp when the first abnormal bright area appears.

[0081] In addition, the target spatio-temporal feature may further include a target shape feature, which is used to represent the contour of the first abnormally bright area. During clustering, in addition to determining the difference in distance interval, the difference in brightness distribution similarity, and the difference in time interval between the first target abnormally bright area and the second target abnormally bright area, the target shape feature of the first target abnormally bright area and the shape feature of the second target abnormally bright area may also be extracted. According to the target shape feature and the shape feature, the shape similarity between the first target abnormally bright area and the second target abnormally bright area is determined, and the difference between the shape similarity and the reference shape similarity in the corresponding first initial clustering set is calculated to determine the shape similarity difference, and the shape similarity difference is compared with a fourth preset threshold. When the distance interval difference is less than or equal to the first preset threshold, the brightness distribution similarity difference is greater than or equal to the second preset threshold, the time interval difference is less than or equal to the third preset threshold, and the shape similarity difference is greater than or equal to the fourth preset threshold, the first target abnormally bright area can be classified into the first initial clustering set corresponding to the second target abnormally bright area. The target spatio-temporal feature may further include other features, which are not limited in the embodiments of the present invention.

[0082] In addition, after determining all the abnormally bright areas, the target time features corresponding to each abnormally bright area can be extracted respectively, the number of the same target time features is determined, and according to this number, the scanning mode of the lidar can be determined, and then the number of the first initial clustering sets during clustering can be determined. For example, if the number of the same target time features is 3, it can be determined that the scanning mode of the lidar is a 3-line scanning mode. When the lidar scans, 3 scanning trajectory lines can be obtained correspondingly, and then 3 first initial clustering sets can be created during clustering.

[0083] Furthermore, the sub-target spatio-temporal feature includes a sub-target spatial position, a sub-target brightness distribution feature, and a sub-target time feature;

[0084] Determining the corresponding sub-first predicted bright area for each clustering set based on the sub-target spatio-temporal features in each clustering set includes:

[0085] For each clustering set, based on the change trend of the sub-target spatial positions of all the sub-first abnormally bright areas in the clustering set, determine the sub-spatial prediction position of the sub-first predicted bright area corresponding to the clustering set;

[0086] Based on the change trend of the sub-target brightness distribution features of all the sub-first abnormally bright areas in the clustering set, determine the sub-brightness distribution prediction feature of the sub-first predicted bright area corresponding to the clustering set;

[0087] Based on the change trend of the sub-target time features of all sub-first abnormal bright areas in the clustering set, determine the sub-predicted time of the sub-first predicted bright area corresponding to the clustering set, where the sub-predicted time is used to indicate adjusting the light intensity of the first predicted bright area before the sub-predicted time.

[0088] Specifically, after clustering to obtain at least one clustering set, the corresponding number of scan trajectory lines are obtained. For each clustering set, the sub-target spatial positions of all sub-first abnormal bright areas can be extracted, and according to the change trend of the sub-target spatial positions, the sub-spatial prediction position of the sub-first predicted bright area corresponding to the clustering set can be predicted along the scan direction corresponding to the clustering set. Then, the sub-target brightness distribution features of all sub-first abnormal bright areas are extracted, and according to the change trend of the sub-target brightness distribution features, the sub-brightness distribution prediction features of the sub-first predicted bright area corresponding to the sub-spatial prediction position in the clustering set are predicted. The change trend of the sub-target time features of all sub-first abnormal bright areas is extracted, and the sub-predicted time corresponding to the sub-spatial prediction position in the clustering set is predicted. This sub-predicted time can be understood as the scan time when the lidar scans to this sub-first predicted bright area.

[0089] Exemplarily, such as Figure 2As shown, taking the clustering set corresponding to the first scan trajectory line as an example, the sub-target spatial positions corresponding to each of the first sub-abnormally bright regions 1 to 4 on the first scan trajectory line are extracted respectively. By determining the distance intervals between adjacent two first sub-abnormally bright regions, the change trends of the three distance intervals can be that the three distance intervals are equal or the change amplitudes between the three distance intervals are relatively small, etc. According to the change trends of the three distance intervals, the sub-spatial prediction position of the first sub-predicted bright region 5 on the first scan trajectory line can be predicted. Then, the sub-target brightness distribution characteristics corresponding to each of the first sub-abnormally bright regions 1 to 4 are extracted respectively, and the brightness distribution similarity corresponding to adjacent two first sub-abnormally bright regions is determined. The change trends of the three brightness distribution similarities can be that the three brightness distribution similarities are the same or the change amplitudes of the three brightness distribution similarities are relatively small, etc. According to this change trend, the sub-target brightness distribution characteristics corresponding to the first sub-predicted bright region 5 can be predicted. In addition, the sub-target time characteristics corresponding to each of the first sub-abnormally bright regions 1 to 4 on the first scan trajectory line can be extracted respectively. By determining the time intervals between adjacent two first sub-abnormally bright regions, the change trends of the three time intervals can be that the three time intervals are equal or the change amplitudes between the three time intervals are relatively small, etc. According to the change trends of the three time intervals, the sub-predicted time of the first sub-predicted bright region 5 on the first scan trajectory line can be predicted. In addition, the sub-target shape characteristics corresponding to each of the first sub-abnormally bright regions 1 to 4 can be extracted respectively, and the shape similarity corresponding to adjacent two first sub-abnormally bright regions is determined. The change trends of the three shape similarities can be that the three shape similarities are the same or the change amplitudes of the three shape similarities are relatively small, etc. According to this change trend, the sub-target shape characteristics corresponding to the first sub-predicted bright region 5 can be predicted.

[0090] Step 130: Adjust the light intensity of the first predicted bright region based on the first preset mapping relationship; the first preset mapping relationship includes the mapping relationship between the regional brightness and the light transmittance.

[0091] Specifically, after predicting the first predicted bright region, the spatio-temporal prediction characteristics of the first predicted bright region can be obtained and matched with the regional brightness in the first preset mapping relationship, so as to determine the target light transmittance corresponding to the first predicted bright region, and then adjust the light intensity of the strong light of the lidar irradiating the first predicted bright region to weaken the light intensity of the strong light of the lidar irradiating the first predicted bright region, preventing damage to the image sensor.

[0092] Optionally, the first preset mapping relationship can be stored in the memory or memory card of the camera. Before adjusting the light intensity of the first predicted bright area, the processor can obtain the first preset mapping relationship. The memory card can include an SD (Secure Digital) card, a microSD card, etc., and the embodiments of the present invention do not limit this.

[0093] Further, adjusting the light intensity of the first predicted bright area based on the first preset mapping relationship includes:

[0094] Obtain the area coordinates corresponding to at least two dimming mechanisms

[0095] Match the spatial prediction position of the first predicted bright area with the area coordinates corresponding to each dimming mechanism respectively to determine the target dimming mechanism corresponding to the first predicted bright area, and the target dimming mechanism is arranged on the image sensor;

[0096] Match the brightness distribution prediction feature of the first predicted bright area with the area brightness in the first preset mapping relationship to determine the target light transmittance corresponding to the target dimming mechanism;

[0097] Adjust the light intensity of the first predicted bright area based on the target light transmittance corresponding to the target dimming mechanism.

[0098] It should be noted that Figure 3 is a schematic structural diagram of at least two dimming mechanisms provided by the embodiments of the present invention, as Figure 3As shown, before adjusting the light intensity of the first predicted bright area, at least two dimming mechanisms need to be set on the image sensor. All the dimming mechanisms are spliced and arranged in front of the image sensor. The strong light of the lidar needs to pass through the dimming mechanism first and then irradiate onto the image sensor. Each dimming mechanism may include a relatively established conductive layer block A and a conductive layer block B. Both the conductive layer block A and the conductive layer block B are in a transparent state, and the conductive layer block A and the conductive layer block B are respectively connected to the positive and negative electrodes of the power supply to apply a corresponding voltage between the conductive layer block A and the conductive layer block B. A liquid crystal dimming film is laid between the conductive layer block A and the conductive layer block. The liquid crystal dimming film, the conductive layer block A, and the conductive layer block B have the same size. After the conductive layer block A and the conductive layer block B corresponding to both sides of the liquid crystal dimming film are powered on, the liquid crystal molecules in the liquid crystal dimming film will be arranged neatly, making the liquid crystal dimming film in a transparent state, so that the strong light of the lidar can pass through the dimming mechanism. After the endpoints of the conductive layer block A and the conductive layer block B corresponding to both sides of the liquid crystal dimming film, the liquid crystal molecules in the liquid crystal dimming film are in an irregular scattered state, making the liquid crystal dimming film in a fogged state, affecting the penetration ability of the strong light of the lidar to the dimming mechanism. In the embodiment of the present invention, the size of each dimming mechanism is not limited. The smaller the size of the dimming mechanism, the higher the accuracy of adjusting the light intensity of the first predicted bright area in the embodiment of the present invention. In addition, adding a liquid crystal dimming film in the same dimming mechanism does not affect the adjustment of the voltage value between the conductive layer block A and the conductive layer block B.

[0099] Optionally, in addition to the Figure 3 structure shown, it may also include a protective lens or a protective film with adjustable brightness, etc., to meet the needs of the strong light of different models of lidar irradiating the image sensor, thereby improving the versatility. Moreover, the above-mentioned dimming mechanism, protective lens or protective film, etc. all have low costs. While protecting the image sensor, it avoids a large increase in the cost of the camera device.

[0100] After predicting the first predicted bright area, the corresponding spatial prediction position of the first predicted bright area can be extracted, and the spatial prediction position is matched with the area coordinates corresponding to each dimming mechanism to determine the target dimming mechanism that meets the spatial prediction position. The number of the target dimming mechanisms can be one or more, as long as it is ensured that the target dimming mechanism can completely cover the first predicted bright area. After determining the target dimming mechanism, the brightness distribution prediction feature of the first predicted bright area is matched with the area brightness in the first preset mapping relationship, and the transmittance corresponding to the matched area brightness is determined as the target transmittance of the target dimming mechanism. Before the sub-prediction time, according to the target transmittance, the light passing through the target dimming mechanism is adjusted, and the light intensity irradiating the first predicted bright area is weakened before the sub-prediction time to avoid damage to the first predicted bright area.

[0101] Optionally, a coordinate system can be constructed according to the image sensor, and the area coordinates corresponding to each dimming mechanism can be determined respectively. The area coordinates can include the upper left corner coordinates and the lower right corner coordinates corresponding to the dimming mechanism.

[0102] Further, adjusting the light intensity of the first predicted bright area based on the target light transmittance corresponding to the target dimming mechanism includes:

[0103] Based on a second preset mapping relationship and the target light transmittance corresponding to the target dimming mechanism, determine the target voltage value corresponding to the target dimming mechanism. The target voltage value is used to adjust the light intensity of the first predicted bright area corresponding to the target dimming mechanism; the second preset mapping relationship includes the mapping relationship between the light transmittance and the voltage value.

[0104] Specifically, after determining the target light transmittance corresponding to the target dimming mechanism, the target light transmittance can be matched with the light transmittances in the second preset mapping relationship, and the voltage value corresponding to the light transmittance that is the same as the target light transmittance is determined as the target voltage value corresponding to the target dimming mechanism. After applying this target voltage value between the conductive layer block A and the conductive layer block B of the target dimming mechanism, the liquid crystal molecules in the liquid crystal dimming film of the target dimming mechanism can be arranged neatly in a certain proportion, so that the light transmittance of the liquid crystal dimming film is the target light transmittance. Before the lidar irradiates the first predicted bright area, the target light transmittance of the target dimming mechanism corresponding to the first predicted bright area is adjusted in advance to weaken the light intensity of the strong light of the lidar irradiating the first predicted bright area, protect the image sensor, and prevent the image sensor from being damaged due to strong light irradiation.

[0105] It should be noted that the second preset mapping relationship includes voltage values corresponding to different light transmittances. The larger the voltage value, the larger the light transmittance corresponding to the liquid crystal dimming film.

[0106] Further, after adjusting the light intensity of the first predicted bright area, the method further includes:

[0107] Obtain at least one second abnormally bright area in the image sensor;

[0108] In the case where the at least one second abnormally bright area does not match the first predicted bright area, update the light transmittances of the dimming mechanisms corresponding to the at least one second abnormally bright area and the first predicted bright area respectively, so as to adjust the light intensities of the at least one second abnormally bright area and the first predicted bright area respectively.

[0109] Exemplarily, taking the scanning mode of the lidar as a single-line scanning mode and the first predicted bright area as the sub-first predicted bright area 5 as an example, after adjusting the light intensity of the sub-first predicted bright area 5, at least one second abnormal bright area C in the image sensor is continuously detected in real time. The spatial position of the second abnormal bright area C and the sub-spatial prediction position of the sub-first predicted bright area 5 are extracted, and the spatial position is matched with the sub-spatial prediction position. If the spatial position does not match the sub-spatial prediction position, the target dimming mechanism can be re-determined according to the spatial position of the second abnormal bright area C, and the target light transmittance of the target dimming mechanism is updated according to the brightness distribution characteristics of the second abnormal bright area C. At the same time, the light transmittance of the dimming mechanism corresponding to the previously predicted sub-first predicted bright area 5 is updated. For example, the dimming mechanism corresponding to the sub-first predicted bright area 5 is powered off to save energy and reduce consumption.

[0110] In addition, if it is detected that the lidar scan exceeds the range of the image sensor, all dimming mechanisms can be adjusted to a transparent state to cancel the protection of the image sensor. In addition, after adjusting the light intensity of the first predicted bright area, if no strong light scan is detected in the first predicted bright area within a preset time period, the target dimming mechanism can be adjusted to a transparent state to cancel the protection of the first predicted bright area in the image sensor.

[0111] The image sensor protection method provided by the embodiments of the present invention extracts the target spatio-temporal characteristics corresponding to each first abnormal bright area through at least three first abnormal bright areas with a timing relationship in the image sensor, predicts the first predicted bright area where the lidar may perform the next strong light scan, and matches according to the regional brightness in the first preset mapping relationship with the first predicted bright area. Before the lidar performs a strong light scan on the first predicted bright area, the light transmittance of the first predicted bright area is adjusted in advance, thereby weakening the light intensity of the strong light of the lidar irradiating the first predicted bright area, protecting the image sensor in real time, and preventing damage to the image sensor. In addition, according to the laser power of different models of lidar, the light transmittance of the first predicted bright area can be adjusted to protect the image sensor in real time and improve the versatility.

[0112] The image sensor protection device provided by the present invention will be described below. The image sensor protection device described below can be correspondingly referred to the image sensor protection method described above.

[0113] The embodiments of the present invention further provide an image sensor protection device, Figure 5 which is a schematic structural diagram of the image sensor protection device provided by the embodiments of the present invention. As Figure 5 shown, the image sensor protection device 500 includes: an acquisition module 510, a determination module 520, and an adjustment module 530, where:

[0114] An acquisition module 510, configured to acquire at least three first abnormally bright areas in an image sensor, where there is at least a timing relationship among the at least three first abnormally bright areas;

[0115] A determination module 520, configured to determine a first predicted bright area in the image sensor based on the target spatio-temporal features corresponding to each of the at least three first abnormally bright areas;

[0116] An adjustment module 530, configured to adjust the light intensity of the first predicted bright area based on a first preset mapping relationship; the first preset mapping relationship includes a mapping relationship between regional brightness and light transmittance.

[0117] The image sensor protection device provided by the embodiment of the present invention extracts the target spatio-temporal features corresponding to each of the at least three first abnormally bright areas with a timing relationship in the image sensor, predicts the first predicted bright area where the lidar may perform the next strong light scan, and matches the first predicted bright area with the regional brightness in the first preset mapping relationship. Before the lidar performs a strong light scan on the first predicted bright area, the light transmittance of the first predicted bright area is adjusted in advance, thereby weakening the light intensity of the strong light of the lidar irradiating the first predicted bright area, protecting the image sensor in real time, and preventing damage to the image sensor. In addition, the light transmittance of the first predicted bright area can be adjusted according to the laser power of different models of lidars to protect the image sensor in real time and improve versatility.

[0118] Optionally, the determination module 520 is specifically configured to:

[0119] Perform clustering based on the target spatio-temporal features of the at least three first abnormally bright areas to determine at least one clustering set;

[0120] Determine a sub-first predicted bright area corresponding to each clustering set based on the sub-target spatio-temporal features in each clustering set;

[0121] Determine the first predicted bright area in the image sensor based on all sub-first predicted bright areas.

[0122] Optionally, the target spatio-temporal features at least include a target spatial position, a target brightness distribution feature, and a target time feature.

[0123] Optionally, the determination module 520 is specifically configured to:

[0124] S1. Determine at least one first initial clustering set among the at least three first abnormally bright areas; each first initial clustering set includes at least two first abnormally bright areas with the smallest target time feature and having a timing relationship;

[0125] S2. Construct at least three second initial clustering sets based on the first abnormally bright areas with the same target time feature;

[0126] S3. For each of the second initial clustering sets, based on the target spatial positions, target brightness distribution characteristics, and target time characteristics corresponding to each of the first abnormally bright regions in the second initial clustering set, construct an initial feature matrix corresponding to the second initial clustering set; each column feature type in the initial feature matrix is the same, and each row feature belongs to the same first abnormally bright region.

[0127] S4. Based on the initial feature matrices corresponding to the first target abnormally bright region and the second target abnormally bright region respectively, and the at least one first initial clustering set, determine the distance interval difference, brightness distribution similarity difference, and time interval difference between the first target abnormally bright region and the second target abnormally bright region; the first target abnormally bright region is any one of the first abnormally bright regions whose belonging first initial clustering set has not been determined, and the second target abnormally bright region is a first abnormally bright region whose belonging first initial clustering set has been determined; the first target abnormally bright region and the second target abnormally bright region are adjacent, and the target time characteristic of the second target abnormally bright region is less than the target time characteristic of the first target abnormally bright region.

[0128] S5. When the distance interval difference is less than or equal to a first preset threshold, the brightness distribution similarity difference is greater than or equal to a second preset threshold, and the time interval difference is less than or equal to a third preset threshold, add the first target abnormally bright region to the first initial clustering set corresponding to the second target abnormally bright region.

[0129] S6. Repeat steps S2 to S5, stop the iteration when the stop condition is reached, and determine the at least one clustering set.

[0130] Optionally, the sub-goal spatio-temporal characteristics include sub-goal spatial position, sub-goal brightness distribution characteristics, and sub-goal time characteristics.

[0131] Optionally, the determining module 520 is specifically configured to:

[0132] For each of the clustering sets, based on the change trend of the sub-goal spatial positions of all the sub-first abnormally bright regions in the clustering set, determine the sub-spatial prediction position of the sub-first predicted bright region corresponding to the clustering set;

[0133] Based on the change trend of the sub-goal brightness distribution characteristics of all the sub-first abnormally bright regions in the clustering set, determine the sub-brightness distribution prediction characteristics of the sub-first predicted bright region corresponding to the clustering set;

[0134] Based on the change trend of the sub-object time characteristics of all sub-first abnormally bright areas in the clustering set, determine the sub-predicted time of the sub-first predicted bright area corresponding to the clustering set, where the sub-predicted time is used to indicate adjusting the light intensity of the first predicted bright area before the sub-predicted time.

[0135] Optionally, the adjustment module 530 is specifically configured to:

[0136] Obtain the area coordinates corresponding to at least two dimming mechanisms;

[0137] Match the spatial prediction position of the first predicted bright area with the area coordinates corresponding to each dimming mechanism respectively to determine the target dimming mechanism corresponding to the first predicted bright area, where the target dimming mechanism is disposed on the image sensor;

[0138] Match the brightness distribution prediction feature of the first predicted bright area with the area brightness in the first preset mapping relationship to determine the target light transmittance corresponding to the target dimming mechanism;

[0139] Adjust the light intensity of the first predicted bright area based on the target light transmittance corresponding to the target dimming mechanism.

[0140] Optionally, the adjustment module 530 is specifically configured to:

[0141] Based on the second preset mapping relationship and the target light transmittance corresponding to the target dimming mechanism, determine the target voltage value corresponding to the target dimming mechanism, where the target voltage value is used to adjust the light intensity of the first predicted bright area corresponding to the target dimming mechanism; the second preset mapping relationship includes the mapping relationship between the light transmittance and the voltage value.

[0142] Optionally, the image sensor protection device further includes an update module, and the update module is specifically configured to:

[0143] Obtain at least one second abnormally bright area in the image sensor;

[0144] In the case that the at least one second abnormally bright area does not match the first predicted bright area, update the light transmittances of the dimming mechanisms corresponding to the at least one second abnormally bright area and the first predicted bright area respectively to adjust the light intensities of the at least one second abnormally bright area and the first predicted bright area respectively.

[0145] Figure 6 It is a schematic structural diagram of a camera module provided by an embodiment of the present invention, as Figure 6As shown in the figure, the electronic device may include: a processor 610, a communications interface 620, a memory 630, a communication bus 640, at least two dimming mechanisms 650, and an image sensor 660, where:

[0146] Each of the dimming mechanisms 650 includes a pair of oppositely arranged conductive layer blocks and a liquid crystal dimming film. The oppositely arranged conductive layer blocks are connected to the processor, and the liquid crystal dimming film is disposed between the oppositely arranged conductive layer blocks;

[0147] All the dimming mechanisms 650 are spliced and disposed on the image sensor 660, and the image sensor is connected to the processor;

[0148] The processor 610, the communications interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call the logic instructions in the memory 630 to execute an image sensor protection method, which includes:

[0149] Obtain at least three first abnormally bright areas in the image sensor 660, and there is at least a timing relationship among the at least three first abnormally bright areas;

[0150] Based on the target spatio-temporal characteristics corresponding to each of the at least three first abnormally bright areas, determine a first predicted bright area in the image sensor 660;

[0151] Based on a first preset mapping relationship, adjust the light intensity of the first predicted bright area; the first preset mapping relationship includes the mapping relationship between the area brightness and the transmittance.

[0152] In addition, when the logic instructions in the above-mentioned memory 630 are implemented in the form of software functional units and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk, or an optical disc that can store program codes.

[0153] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the image sensor protection method provided by each of the above methods, and the method includes:

[0154] Obtain at least three first abnormally bright areas in the image sensor, and there is at least a timing relationship among the at least three first abnormally bright areas;

[0155] Based on the target spatio-temporal characteristics corresponding to each of the at least three first abnormally bright areas, determine the first predicted bright area in the image sensor;

[0156] Based on a first preset mapping relationship, adjust the light intensity of the first predicted bright area; the first preset mapping relationship includes the mapping relationship between regional brightness and light transmittance.

[0157] In another aspect, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is implemented to execute the image sensor protection method provided by each of the above methods, and the method includes:

[0158] Obtain at least three first abnormally bright areas in the image sensor, and there is at least a timing relationship among the at least three first abnormally bright areas;

[0159] Based on the target spatio-temporal characteristics corresponding to each of the at least three first abnormally bright areas, determine the first predicted bright area in the image sensor;

[0160] Based on a first preset mapping relationship, adjust the light intensity of the first predicted bright area; the first preset mapping relationship includes the mapping relationship between regional brightness and light transmittance.

[0161] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative labor.

[0162] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image sensor protection method, characterized in that, Including: Obtain at least three first abnormally bright areas in the image sensor, and there is at least a temporal relationship among the at least three first abnormally bright areas; Determine a first predicted bright area in the image sensor based on the target spatio-temporal characteristics corresponding to each of the at least three first abnormally bright areas; Adjust the light intensity of the first predicted bright area based on a first preset mapping relationship; the first preset mapping relationship includes the mapping relationship between regional brightness and light transmittance.

2. The image sensor protection method according to claim 1, wherein The adjusting the light intensity of the first predicted bright area based on the first preset mapping relationship includes: Obtain the regional coordinates corresponding to at least two dimming mechanisms; Match the spatial prediction positions of the first predicted bright area with the regional coordinates corresponding to each of the dimming mechanisms to determine the target dimming mechanism corresponding to the first predicted bright area, and the target dimming mechanism is arranged on the image sensor; Match the brightness distribution prediction feature of the first predicted bright area with the regional brightness in the first preset mapping relationship to determine the target light transmittance corresponding to the target dimming mechanism; Adjust the light intensity of the first predicted bright area based on the target light transmittance corresponding to the target dimming mechanism.

3. The image sensor protection method according to claim 2, wherein The adjusting the light intensity of the first predicted bright area based on the target light transmittance corresponding to the target dimming mechanism includes: Determine the target voltage value corresponding to the target dimming mechanism based on a second preset mapping relationship and the target light transmittance corresponding to the target dimming mechanism, and the target voltage value is used to adjust the light intensity of the first predicted bright area corresponding to the target dimming mechanism; the second preset mapping relationship includes the mapping relationship between light transmittance and voltage value.

4. The method for protecting an image sensor according to any one of claims 1-3, characterized in that, The determining a first predicted bright area in the image sensor based on the target spatio-temporal characteristics corresponding to each of the at least three first abnormally bright areas includes: Perform clustering based on the target spatio-temporal characteristics of the at least three first abnormally bright areas to determine at least one clustering set; Determine the sub-first predicted bright areas corresponding to each of the clustering sets based on the sub-target spatio-temporal characteristics in each of the clustering sets; Determine the first predicted bright area in the image sensor based on all the sub-first predicted bright areas.

5. The image sensor protection method according to claim 4, wherein The target spatio-temporal characteristics at least include target spatial position, target brightness distribution feature, and target time feature; The performing clustering based on the target spatio-temporal characteristics of the at least three first abnormally bright areas to determine at least one clustering set includes: S1. Determine at least one first initial clustering set among the at least three first abnormally bright areas; each of the first initial clustering sets includes at least two first abnormally bright areas with the smallest target time feature and having a temporal relationship; S2. Construct at least three second initial clustering sets based on the first abnormally bright areas with the same target time feature; S3. For each of the second initial clustering sets, construct an initial feature matrix corresponding to the second initial clustering set based on the target spatial position, target brightness distribution feature, and target time feature corresponding to each of the first abnormally bright areas in the second initial clustering set; each column feature type in the initial feature matrix is the same, and each row feature belongs to the same first abnormally bright area; S4. Determine the distance interval difference, brightness distribution similarity difference, and time interval difference between the first target abnormal bright area and the second target abnormal bright area based on the respective initial feature matrices corresponding to the first target abnormal bright area and the second target abnormal bright area, and the at least one first initial clustering set. The first target abnormal bright area is any first abnormal bright area that has not been determined to belong to the first initial clustering set, and the second target abnormal bright area is a first abnormal bright area that has been determined to belong to the first initial clustering set. The first target abnormal bright area and the second target abnormal bright area are adjacent, and the target time feature of the second target abnormal bright area is less than the target time feature of the first target abnormal bright area. S5. When the distance interval difference is less than or equal to a first preset threshold, the brightness distribution similarity difference is greater than or equal to a second preset threshold, and the time interval difference is less than or equal to a third preset threshold, add the first target abnormal bright area to the first initial clustering set corresponding to the second target abnormal bright area. S6. Repeat steps S2 to S5, stop iterating when the stop condition is reached, and determine the at least one clustering set.

6. The image sensor protection method according to claim 5, wherein, The sub-goal spatio-temporal feature includes sub-goal spatial position, sub-goal brightness distribution feature, and sub-goal time feature. The determining the sub-first predicted bright area corresponding to each clustering set based on the sub-goal spatio-temporal features in each clustering set includes: For each clustering set, determine the sub-spatial predicted position of the sub-first predicted bright area corresponding to the clustering set based on the change trend of the sub-goal spatial positions of all sub-first abnormal bright areas in the clustering set. Determine the sub-brightness distribution predicted feature of the sub-first predicted bright area corresponding to the clustering set based on the change trend of the sub-goal brightness distribution features of all sub-first abnormal bright areas in the clustering set. Determine the sub-predicted time of the sub-first predicted bright area corresponding to the clustering set based on the change trend of the sub-goal time features of all sub-first abnormal bright areas in the clustering set. The sub-predicted time is used to indicate adjusting the light intensity of the first predicted bright area before the sub-predicted time.

7. The method for protecting an image sensor according to any one of claims 1-3, characterized in that, After adjusting the light intensity of the first predicted bright area, the method further includes: Obtain at least one second abnormal bright area in the image sensor. When the at least one second abnormal bright area does not match the first predicted bright area, update the light transmittance of the dimming mechanism corresponding to the at least one second abnormal bright area and the first predicted bright area respectively to adjust the light intensity corresponding to the at least one second abnormal bright area and the first predicted bright area respectively.

8. An image sensor protection device, characterized in that, Includes: An acquisition module, configured to acquire at least three first abnormal bright areas in the image sensor, and there is at least a temporal relationship among the at least three first abnormal bright areas. A determination module, configured to determine the first predicted bright area in the image sensor based on the respective target spatio-temporal features corresponding to the at least three first abnormal bright areas. An adjustment module, configured to adjust the light intensity of the first predicted bright area based on a first preset mapping relationship. The first preset mapping relationship includes the mapping relationship between the regional brightness and the light transmittance.

9. A camera module, comprising at least two dimming mechanisms, an image sensor, a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that each of the dimming mechanisms includes oppositely arranged conductive layer blocks and a liquid crystal dimming film, the oppositely arranged conductive layer blocks are connected to the processor, and the liquid crystal dimming film is disposed between the oppositely arranged conductive layer blocks; all the dimming mechanisms are spliced and disposed on the image sensor, and the image sensor is connected to the processor; when the processor executes the program, the image sensor protection method described in any one of claims 1-7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the image sensor protection method described in any one of claims 1-7 is implemented.

Images