Camera

Abstract

The invention provides a camera, and relates to the technical field of sensor equipment. The camera comprises a light sensing module used for generating pixel data, and the pixel data comprises an electric signal value of each pixel point in a current frame and identification information of each pixel group; the filtering module is used for determining abnormal pixel points of the current frame through a preset standard electric signal numerical range based on the pixel data, and determining an abnormal pixel group of the current frame through a preset identification verification rule; the image processing module is configured to perform credibility judgment on the non-abnormal pixel group according to abnormal pixel points contained in the non-abnormal pixel group passing the identification verification rule; judging the abnormal pixel group which does not pass the identification verification rule as an untrusted pixel group; and detecting whether the current frame image generated based on the pixel data is credible or not based on the credibility judgment result of all the pixel groups. According to the invention, it is ensured that only the high-quality and credible current frame image can be used for decision making, and misjudgment caused by image abnormity is avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of sensor devices, in particular to a camera. BACKGROUND

[0002] In the field of autonomous driving, cameras are a key component of autonomous driving systems, responsible for providing environmental image information to decision-making systems. A camera is composed of numerous pixel points, each of which can sense light signals and convert them into electrical signals, and then into digital signals to form an image.

[0003] However, during the manufacture or use of the camera, due to various factors, the electrical signals of some pixel points may become unstable, which not only affects the generation quality of the entire image, but also may mislead the autonomous driving system, thereby affecting its safety. SUMMARY

[0004] Therefore, an embodiment of the present application provides a camera.

[0005] In a first aspect, an embodiment of the present application provides a camera, comprising: a light sensing module having a pixel array, configured to generate pixel data, the pixel data comprising an electrical signal value of each pixel point in a current frame and identification information of each pixel group, the pixel group comprising a pixel row and / or a pixel column; a filtering module configured to determine abnormal pixel points in the current frame based on the pixel data through a preset standard electrical signal value range, and determine abnormal pixel groups in the current frame through a preset identification checking rule, wherein the identification checking rule comprises at least one of the following: a rule for verifying that the identification information of each pixel group is unique; a rule for verifying that the identification information of all pixel groups is continuous; a rule for verifying that the identification information of each pixel group conforms to a predetermined format specification; a rule for verifying that the number of identification information of all pixel groups matches an expected total number; an image processing module configured to perform credibility determination on a non-abnormal pixel group containing abnormal pixel points in the non-abnormal pixel group through the identification checking rule; determine an abnormal pixel group that does not pass the identification checking rule as an untrustworthy pixel group; and based on the credibility determination results of all pixel groups, detect whether a current frame image generated based on the pixel data is trustworthy.

[0006] In combination with the first aspect, in some implementations of the first aspect, the image processing module is further configured to: determine a first ratio value of the number of abnormal pixel points in the non-abnormal pixel group to the total number of pixel points in the non-abnormal pixel group, and if the first ratio value is greater than a first proportion threshold, determine that the non-abnormal pixel group is an untrustworthy pixel group; determine a second ratio value of the number of untrustworthy pixel groups in the current frame to the total number of pixel groups, and if the second ratio value is greater than a second proportion threshold, determine that the current frame image generated based on the pixel data is untrustworthy.

[0007] With reference to the first aspect, in some implementations of the first aspect, the image processing module is further configured to dynamically adjust the second proportion threshold based on the light intensity data of the current environment, wherein the second proportion threshold increases as the light intensity data increases; and / or the image processing module is further configured to set different second proportion thresholds for the region of interest and the non-region of interest in the current frame, wherein the second proportion threshold for the region of interest is smaller than the second proportion threshold for the non-region of interest.

[0008] With reference to the first aspect, in some implementations of the first aspect, the second proportion threshold is determined based on the minimum light intensity data of the current environment, the maximum light intensity data of the current environment, and a normalization coefficient.

[0009] With reference to the first aspect, in some implementations of the first aspect, the image processing module is further configured to, for each abnormal pixel point of the current frame, determine whether the abnormal pixel point satisfies a first optimization condition based on the number of abnormal pixel points within a target range of the abnormal pixel point; and if the abnormal pixel point satisfies the first optimization condition, optimize the abnormal pixel point to obtain a normal pixel point corresponding to the abnormal pixel point; wherein the abnormal pixel points included in the non-abnormal pixel group refer to abnormal pixel points in the non-abnormal pixel group that do not satisfy the first optimization condition; preferably, the first optimization condition comprises that the number of abnormal pixel points within the target range is less than a first number threshold.

[0010] With reference to the first aspect, in some implementations of the first aspect, the image processing module is further configured to, if the abnormal pixel point satisfies the first optimization condition, determine an abnormal persistence feature of each abnormal pixel point of the current frame, the abnormal persistence feature being used to represent a behavior pattern of the abnormal pixel point deviating from the standard electrical signal value range in consecutive multiple frames; determine whether each abnormal pixel point satisfies a second optimization condition based on the abnormal persistence feature of each abnormal pixel point; and if the abnormal pixel point satisfies the second optimization condition, optimize the electrical signal value of the abnormal pixel point.

[0011] With reference to the first aspect, in some implementations of the first aspect, the second optimization condition comprises that the electrical signal value of the abnormal pixel point in consecutive multiple frames is not within the standard electrical signal value range; and the image processing module is further configured to, if the abnormal pixel point satisfies the second optimization condition, determine an electrical signal statistical value of the pixel points within a target range centered on the abnormal pixel point; and replace the electrical signal value of the abnormal pixel point with the electrical signal statistical value.

[0012] In some implementations of the first aspect, the second optimization condition comprises that the abnormal pixel point has a frequency of abnormality in the continuous multiple frames that is lower than a preset frequency threshold, and the abnormality indicates that the electrical signal value is out of the standard electrical signal value range; the image processing module is further configured to: if the abnormal pixel point satisfies the second optimization condition, obtain the electrical signal values of the abnormal pixel point in the continuous multiple frames; obtain a weighted average electrical signal value based on the electrical signal values of the abnormal pixel point in the continuous multiple frames of the current frame and respective weight coefficients, wherein the weight coefficients are proportional to time correlation of the corresponding frame data; and replace the electrical signal value of the abnormal pixel point with the weighted average electrical signal value; and preferably, the weight coefficients are determined based on a time sequence of the frame data and a decay factor, wherein the decay factor is used to control a decay speed of the weight coefficients with respect to time of the corresponding frame data.

[0013] In some implementations of the first aspect, the image processing module is further configured to: obtain a working temperature of the light sensing module and / or the filtering module; and if the working temperature of the light sensing module and / or the filtering module is greater than a preset temperature threshold, determine that the current frame image generated based on the pixel data is not trustworthy.

[0014] In some implementations of the first aspect, the image processing module is further configured to: if it is determined that the current frame image generated based on the pixel data is not trustworthy, add an untrustworthy label to the current frame image; and send the current frame image and the untrustworthy label corresponding to the current frame image to a data receiving system, so as to facilitate the data receiving system to determine whether to use the current frame image based on the untrustworthy label, or to facilitate the data receiving system to execute a corresponding functional safety policy based on the untrustworthy label, wherein the functional safety policy refers to measures formulated to ensure that the autonomous driving system can still operate safely when the camera is abnormal.

[0015] In the second aspect, the present application provides an image detection method, comprising: generating pixel data, wherein the pixel data comprises an electrical signal value of each pixel point in a current frame and identification information of each pixel group, and the pixel group comprises a pixel row and / or a pixel column; determining an abnormal pixel point of the current frame based on the pixel data through a preset standard electrical signal value range, and determining an abnormal pixel group of the current frame through a preset identification verification rule, wherein the identification verification rule comprises at least one of the following: a rule for verifying that the identification information of each pixel group is unique; a rule for verifying that the identification information of all pixel groups is continuous; a rule for verifying that the identification information of each pixel group conforms to a predetermined format specification; a rule for verifying that the number of the identification information of all pixel groups matches an expected total number; performing a trustworthiness determination on a non-abnormal pixel group containing an abnormal pixel point in the non-abnormal pixel group through the identification verification rule; determining an abnormal pixel group that does not pass the identification verification rule as an untrustworthy pixel group; and detecting whether a current frame image generated based on the pixel data is trustworthy based on a trustworthiness determination result of all pixel groups.

[0016] In a third aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program for executing the image detection method in the second aspect.

[0017] In a fourth aspect, an embodiment of the present application provides an electronic device, which comprises: a processor; a memory for storing processor-executable instructions; and the processor is configured to execute the image detection method in the second aspect.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, which comprises instructions for causing an electronic device to implement the image detection method in the second aspect when the instructions are executed on the electronic device.

[0019] In the embodiment, the pixel data generated by the light sensing module not only contains the electrical signal value, but also contains the identification information of the pixel group, which provides double dimensions for the positioning of the abnormal pixel point. That is, the filtering module can not only quickly screen the abnormal pixel point based on the preset standard electrical signal numerical range, but also identify the abnormal pixel group through the identification checking rule. Then, for the non-abnormal pixel group passing the identification checking rule, the credibility is not directly adopted, but is further determined based on the possible abnormal pixel point inside, which improves the accuracy and reliability of the detection. For the abnormal pixel group not passing the identification checking rule, it is directly determined as an untrusted pixel group, which avoids the deeper data processing error caused by the identification information error, and ensures the integrity and reliability of the image. Finally, the credibility determination results of all pixel groups are integrated to detect whether the current frame image is credible, which ensures that only high-quality and credible current frame image is used for decision-making, avoids the misjudgment caused by image abnormality, and enhances the safety of automatic driving. BRIEF DESCRIPTION OF DRAWINGS

[0020] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description of embodiments of the present application, taken in conjunction with the accompanying drawings. The drawings provided in the disclosure serve to provide a further understanding of the embodiments of the present application, constitute a part of the specification and are used to explain the present application together with the embodiments of the present application, but do not constitute a limitation on the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0021] Figure 1 Fig. 1 shows a structural schematic diagram of a camera according to an embodiment of the present application.

[0022] Figure 2 Fig. 2 shows a flowchart of an image detection method according to an embodiment of the present application.

[0023] Figure 3Fig. 1 shows a structural schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be clearly and completely described with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0025] Figure 1 Fig. 2 shows a structural schematic diagram of a camera according to an embodiment of the present application. As shown in Fig. 2, the camera comprises a light sensing module 110, a filtering module 120 and an image processing module 130. Figure 1 The light sensing module 110 has a pixel array, which exemplarily comprises m rows and n columns of pixel points.

[0026] Optionally, the light sensing module 110 is configured to generate pixel data.

[0027] The pixel data comprises an electrical signal value of each pixel point in a current frame. Exemplarily, when light irradiates on a pixel point, a photosensitive element in the light sensing module 110 converts light intensity information into an electrical signal value. In addition, the pixel data further comprises identification information of each pixel group, which is used to distinguish and identify different pixel groups. Exemplarily, the identification information is a digital code, which can quickly locate a pixel group in subsequent processing. In some embodiments, a pixel group comprises a pixel row and / or a pixel column, that is, a pixel group can be a pixel row, a pixel column or a combination of both.

[0028] Optionally, the filtering module 120 is configured to determine an abnormal pixel point of the current frame based on the pixel data through a preset standard electrical signal value range, and determine an abnormal pixel group of the current frame through a preset identification checking rule.

[0029] The standard electrical signal value range refers to a range of electrical signal values preset according to normal working conditions of the camera, which is mainly used to distinguish normal pixel points and abnormal pixel points. Optionally, if an electrical signal value of a pixel point is lower than a lower limit of the standard electrical signal value range or higher than an upper limit of the standard electrical signal value range, it indicates that the pixel point may have a fault or be interfered, and thus is determined as an abnormal pixel point. If the electrical signal value of the pixel point is within the standard electrical signal value range, the pixel point is a normal pixel point.

[0030] Specifically, the identification checking rule includes at least one of the following: a rule for verifying that the identification information of each pixel group is unique, a rule for verifying that the identification information of all pixel groups is continuous, a rule for verifying that the identification information of each pixel group conforms to a predetermined format specification, and a rule for verifying that the number of identification information of all pixel groups matches an expected total number. The rule for verifying that the identification information of each pixel group is unique is used to check whether there is repeated identification information (for example, two Z600 columns appear in the same frame), and if duplication is found, the corresponding pixel group is marked as an abnormal pixel group. The rule for verifying that the identification information of all pixel groups is continuous is used to check whether the arrangement of the identification information conforms to a predetermined order (for example, the column number should be continuously increased from Z001 to Z300), and if the identification information jumps or is out of order, the corresponding pixel group is marked as an abnormal pixel group. The rule for verifying that the identification information of each pixel group conforms to a predetermined format specification is used to check whether the format of the identification information conforms to the specification (for example, the column number should be Z+3 digits, and the length should be fixed as 4 characters), and if the format of the identification information is incorrect, the corresponding pixel group is marked as an abnormal pixel group. The rule for verifying that the number of identification information of all pixel groups matches an expected total number is used to confirm that all pixel groups have identification information, and there is no missing or null value, and if the identification information of a certain pixel group is missing, the corresponding pixel group is marked as an abnormal pixel group.

[0031] Optionally, the image processing module 130 is configured to perform a credibility determination on the non-abnormal pixel group containing the abnormal pixel points passing the identification checking rule, and determine the abnormal pixel group not passing the identification checking rule as an untrustworthy pixel group, and detect whether the current frame image generated based on the pixel data is trustworthy based on the credibility determination results of all pixel groups.

[0032] For example, the credibility determination is performed on the non-abnormal pixel group containing the abnormal pixel points passing the identification checking rule based on the number of abnormal pixel points. Further, whether the current frame image generated based on the pixel data is trustworthy is detected based on the number of pixel groups passing the credibility determination. For example, if the number of abnormal pixel points contained in the non-abnormal pixel group is greater than a first number threshold, it is determined that the non-abnormal pixel group is untrustworthy, if the number of pixel groups passing the credibility determination is greater than a second number threshold, it is determined that the current frame image generated based on the pixel data is trustworthy. Otherwise, it is untrustworthy.

[0033] It should be noted that the image is trustworthy, that is, the current frame image is defect-free, or the defect degree is within the allowable range, and can be safely used by other systems (such as an automatic driving system or a decision system).

[0034] In this embodiment, the pixel data generated by the light sensing module not only contains the electrical signal value, but also contains the identification information of the pixel group, providing a double dimension for the positioning of abnormal pixel points. That is, the filtering module can not only quickly screen abnormal pixel points based on the preset standard electrical signal numerical range, but also identify abnormal pixel groups through identification checking rules. Then, for non-abnormal pixel groups that pass the identification checking rules, they are not directly adopted, but further based on the possible existence of abnormal pixel points in them, their credibility is determined, improving the accuracy and reliability of the detection. For abnormal pixel groups that do not pass the identification checking rules, they are directly determined as untrusted pixel groups, avoiding deeper data processing errors that may be caused by identification information errors, ensuring the integrity and reliability of the image. Finally, the credibility determination results of all pixel groups are integrated to detect whether the current frame image is credible, and then ensure that only high-quality and credible current frame images are used for decision-making, avoiding misjudgment caused by image abnormalities, thereby enhancing the safety of autonomous driving.

[0035] In addition, Figure 1 Embodiments focus on the basic composition and function of the camera, including the cooperative work of the light sensing module 110, the filtering module 120 and the image processing module 130, and how they work together on pixel data to detect the credibility of the current frame image. Next, further explore how the image processing module 130 more carefully assesses and determines the credibility of the current frame image in some specific implementation ways. Specifically, a first ratio value of the number of abnormal pixel points in the non-abnormal pixel group to the total number of pixel points in the non-abnormal pixel group is determined, and if the first ratio value is greater than a first proportion threshold, the non-abnormal pixel group is determined as an untrusted pixel group; a second ratio value of the number of untrusted pixel groups in the current frame to the total number of pixel groups is determined, and if the second ratio value is greater than a second proportion threshold, the current frame image generated based on the pixel data is determined as untrusted.

[0036] For example, in a camera containing a 640(row)×480(column) pixel array, assuming that a certain non-abnormal pixel group (such as the 5th row) has 480 pixel points, of which 50 abnormal pixel points are detected. If the first proportion threshold is set to 5%, then 50 / 480≈10.4%, which exceeds 5%, so the 5th row is determined as an untrusted pixel group.

[0037] If there are 10 untrusted pixel groups in the current frame, and the total number of pixel groups is 640, then the second ratio value is 10 / 640≈1.56%. If the second proportion threshold is set to 2%, then 1.56% is less than 2%, and finally the current frame image is determined as trusted.

[0038] In the embodiment, for the non-anomalous pixel group passing the identification of the checking rule, a first ratio of the number of anomalous pixel points to the total number of pixel points in the pixel group is calculated and compared with a first ratio threshold. If the first ratio exceeds the threshold, the non-anomalous pixel group is determined as an untrusted pixel group. This step can identify the pixel group with normal identification information but actually having a large number of anomalous pixel points, thereby improving the detection accuracy of image anomalies. Then, a second ratio of the number of untrusted pixel groups in the current frame to the total number of pixel groups is calculated and compared with a second ratio threshold. If the second ratio exceeds the threshold, the current frame image is determined as untrusted. This step forms a comprehensive judgment on the trustworthiness of the entire frame image, ensuring that only high-quality and highly-trustworthy images can pass the detection.

[0039] To further improve the intelligence and adaptability of the camera, the image processing module 130 also has the ability to dynamically adjust the second ratio threshold according to the light intensity data of the environment, and the flexibility to set differentiated thresholds for different regions. Specifically, in some implementations, the image processing module 130 is also configured to dynamically adjust the second ratio threshold based on the light intensity data of the current environment, wherein the second ratio threshold increases as the light intensity data increases.

[0040] It can be understood that in an environment with strong light (i.e., large light intensity data), the light signal received by the light sensing module 110 is stronger, and the electrical signal value of the pixel points is generally higher, so the influence of anomalous pixel points on the generated image is relatively small. Therefore, the second ratio threshold can be appropriately increased to allow more images with a certain proportion of anomalous pixel groups to be determined as trusted.

[0041] Conversely, in a weak light environment (i.e., small light intensity data), the light signal received by the light sensing module 110 is relatively small, and at this time the electrical signal generated by the pixel points is relatively weak and is more easily affected by various interference factors, resulting in a relatively increased number of anomalous pixel points. In order to ensure the quality and reliability of the generated image, the second ratio threshold can be appropriately reduced. In this way, when evaluating the trustworthiness of the generated image, even if the proportion of untrusted pixel groups is small, it will be easier to determine that the current frame image is untrusted, thereby avoiding subsequent decision errors due to the use of low-quality images. For example, in the scenario of night driving or in a tunnel, the requirement for image quality of the autonomous driving system will be higher. By reducing the second ratio threshold, anomalous conditions in the image can be more sensitively detected to ensure driving safety.

[0042] The strategy of dynamically adjusting the second proportion threshold based on the light intensity data enables the camera to automatically optimize the image quality evaluation standard according to the light conditions of the environment. On the one hand, in weak light, the decline in image quality caused by insufficient light is avoided to affect the decision of the automatic driving system. On the other hand, in strong light, more images with a certain proportion of abnormal pixel groups are allowed to be judged as credible, thereby improving the adaptability of the camera to different lighting environments and reducing misjudgment and missed judgment caused by environmental changes under the premise of ensuring image quality.

[0043] In some embodiments, the second proportion threshold is determined based on the minimum light intensity data of the current environment, the maximum light intensity data of the current environment, and a normalization coefficient. The formula can be expressed as: the second proportion threshold T2(L) = T 2,min +(T 2,max -T 2,min )×f(L), where T 2,min represents the minimum light intensity data of the current environment, T 2,max represents the maximum light intensity data of the current environment, and f(L) represents the normalization light intensity function, that is, the normalization coefficient.

[0044] The difference between the maximum light intensity data and the minimum light intensity data reflects the contrast and fluctuation degree of the environmental light. The larger the difference, the more complex or unstable the environmental light, and at this time a normalization coefficient is multiplied to calculate the reference fault tolerance corresponding to the noise level of the current environment. Then the minimum light intensity data is added to ensure that the calculated second proportion threshold is always anchored above the basic level of the environmental light intensity.

[0045] It can be seen that the second proportion threshold increases with the increase of the minimum light intensity data of the current environment. In addition, in a good and stable lighting environment, the difference in light intensity is small, and the second proportion threshold is low, so a more stringent credibility judgment is performed. Conversely, in an environment with severe changes in light, the difference in light intensity increases, and the second proportion threshold increases, avoiding the inevitable noise caused by environmental interference from being misjudged as an image being unreliable, effectively preventing the excessive discard of reliable images in harsh conditions.

[0046] In other implementations, the image processing module is further configured to set different second proportion thresholds for the region of interest and the non-region of interest in the current frame image.

[0047] Specifically, the region of interest refers to the part of the image that has important information and plays a key role in subsequent decision-making and analysis, such as the regions of road signs, traffic lights, pedestrians, and vehicles in the automatic driving scenario. The non-region of interest is the relatively less important part of the image, which has less impact on overall understanding and decision-making.

[0048] In addition, the second proportion threshold of the region of interest is smaller than the second proportion threshold of the non-region of interest. That is, in the region of interest, a relatively small proportion of untrusted pixel groups will lead to the generated image being determined as untrusted, thereby triggering further processing of the entire image or directly discarding. Conversely, for the non-region of interest, since its influence on the decision is smaller, a higher second proportion threshold can be set, which helps to reduce unnecessary image processing operations and improve processing efficiency. For example, in the automatic driving scene, for the region of interest of the road sign, the second proportion threshold is set to 1%. For the non-region of interest, such as the sky or the background in the distance, the second proportion threshold is set to 5%.

[0049] For example, in the current frame image, region A in the image is determined as the region of interest, and region B is determined as the non-region of interest. Wherein, the second proportion threshold corresponding to region A is a, and the second proportion threshold corresponding to region B is b, a is smaller than b. Then, the image processing module 130 performs the feasibility determination for region A and region B respectively. Specifically, the ratio of the number of untrusted pixel groups in region A to the total number of pixel groups in the region is calculated, and it is determined whether the ratio is greater than the second proportion threshold a. At the same time, the ratio of the number of untrusted pixel groups in the non-region of interest to the total number of pixel groups in the region is calculated, and it is determined whether the ratio is greater than the second proportion threshold b.

[0050] In one example, when the ratio of the number of untrusted pixel groups in the region of interest to the total number of pixel groups in the region is smaller than the second proportion threshold corresponding to the region, the current frame image is determined as trusted regardless of the judgment result of the non-region of interest; or when the ratio of the number of untrusted pixel groups in the region of interest to the total number of pixel groups in the region is greater than or equal to the second proportion threshold corresponding to the region, the current frame image is determined as untrusted regardless of the judgment result of the non-region of interest. This judgment strategy ensures that the data reliability of the region of interest has the highest priority.

[0051] In another example, the current frame image is determined as trusted only when the ratio of the number of untrusted pixel groups in the region of interest to the total number of pixel groups in the region and the ratio of the number of untrusted pixel groups in the non-region of interest to the total number of pixel groups in the region are both smaller than the second proportion threshold corresponding to the region respectively, otherwise, it is untrusted.

[0052] This strategy of setting different second proportion thresholds for the region of interest and the non-region of interest further optimizes the accuracy and resource allocation of image quality evaluation. By setting a lower second proportion threshold for the region of interest, the camera can more strictly control the image quality of the key region. At the same time, by setting a relatively high second proportion threshold for the non-region of interest, it can reduce the excessive processing of the camera on these regions, and reasonably tolerate a certain proportion of image abnormalities without affecting the overall decision.

[0053] In some embodiments, in order to improve the image quality, the image processing module 130 is further capable of optimizing the abnormal pixel points. Specifically, for each abnormal pixel point of the current frame, based on the number of abnormal pixel points in the target range of the abnormal pixel point, it is determined whether the abnormal pixel point satisfies a first optimization condition; if the abnormal pixel point satisfies the first optimization condition, the abnormal pixel point is optimized to obtain a normal pixel point corresponding to the abnormal pixel point.

[0054] The target range refers to a pixel region within a certain range centered on the abnormal pixel point. Optionally, the target range can be fixed, such as a radius region of a pixel point centered on the abnormal pixel point, or a pixel region of b x c centered on the abnormal pixel point, where b and c can be the same or different. In addition, the target range can also be a dynamically adjusted range.

[0055] In some embodiments, by checking the number of other abnormal pixel points in the target range around the abnormal pixel point, it is determined whether the abnormal pixel point of the current frame satisfies the first optimization condition. If so, the abnormal pixel point is optimized to obtain a normal pixel point corresponding to the abnormal pixel point. Accordingly, the abnormal pixel points contained in the non-abnormal pixel group refer to the abnormal pixel points in the non-abnormal pixel group that do not satisfy the first optimization condition.

[0056] For example, in a camera containing a 640 (row) x 480 (column) pixel array, assuming that there are 5 abnormal pixel points in a non-abnormal pixel group (such as the 10th row). The image processing module 130 will check whether these abnormal pixel points satisfy the first optimization condition one by one. If 3 of them satisfy the first optimization condition, they will be further optimized to normal pixel points. The remaining 2 abnormal pixel points do not satisfy the first optimization condition, and they belong to the abnormal pixel points in the non-abnormal pixel group.

[0057] In some embodiments, the first optimization condition includes that the number of abnormal pixel points in the target range is less than a first number threshold.

[0058] The first number threshold is a preset value for determining whether the number of abnormal pixel points in the target range is small enough. If the number of abnormal pixel points in the target range is less than the first number threshold, it is determined that the abnormal pixel point satisfies the first optimization condition. Optionally, the first number threshold is set based on the performance of the camera, the image quality requirement, and the actual application scenario.

[0059] If the number of abnormal pixels in the target range is small, it indicates that the surrounding environment of the abnormal pixel in the current frame is relatively normal, and the abnormal pixel can be repaired to a normal pixel through some optimization algorithm. For example, in a camera containing a 640 (row) x 480 (column) pixel array, it is assumed that a certain abnormal pixel in the current frame is located at the middle position of the pixel matrix. Optionally, a 3x3 target range is set with the abnormal pixel as the center, that is, 9 pixel points including the abnormal pixel. If, in the target range, all the pixel points except the center abnormal pixel are normal, the number of abnormal pixels in the target range is 1. If the first number threshold is set to 3, then 1 is less than 3, and it is determined that the abnormal pixel meets the optimization condition and can be optimized.

[0060] By judging whether the abnormal pixel in the current frame meets the first optimization condition through the number of abnormal pixels in the target range, it can avoid invalid optimization of the area where the surrounding environment already has a large number of abnormal pixels. By optimizing the abnormal pixels that meet the first optimization condition, it can effectively reduce noise and distortion, improve the overall quality of the current frame image, and enhance driving safety. At the same time, by distinguishing between optimizable and non-optimizable abnormal pixels and optimizing only the abnormal pixels that meet the first optimization condition, it avoids indiscriminate processing and reduces unnecessary computational overhead, improving the utilization efficiency of computing resources. Moreover, the first number threshold can be flexibly adjusted according to actual conditions to enhance the adaptability and reliability of the camera. In actual applications, such as autonomous driving scenarios, this optimization strategy helps to reduce noise and distortion in images and improve the overall quality of images, thereby ensuring accurate perception and decision-making of the environment by the autonomous driving system.

[0061] In order to further improve the accuracy and robustness of abnormal pixel optimization, after completing the first round of screening through the first optimization condition, an optional progressive optimization strategy is provided in the embodiments of the present application, which is implemented as follows. If the abnormal pixel meets the first optimization condition, the abnormal persistence feature of each abnormal pixel in the current frame is determined; based on the abnormal persistence feature of each abnormal pixel, it is judged whether each abnormal pixel meets the second optimization condition; if the abnormal pixel meets the second optimization condition, the electrical signal value of the abnormal pixel is optimized.

[0062] The abnormal persistence feature is used to represent the behavior pattern of the abnormal pixel deviating from the standard electrical signal value range in consecutive multiple frames. This feature reveals the persistence and regularity of the abnormal pixel from the time sequence dimension, further ensuring that the optimization operation is only applied to abnormal pixels that meet a specific behavior pattern and need to be processed.

[0063] Exemplarily, first, the abnormal pixel points satisfying the first optimization condition are tracked and recorded in the continuous multiple frames of images, and then the abnormal persistence characteristics of the abnormal pixel points are calculated through a time sequence analysis algorithm. Subsequently, whether the abnormal pixel points satisfy the second optimization condition is determined based on the characteristics of each abnormal pixel point.

[0064] The second optimization condition in the scheme ensures that the signal optimization operation is only performed on abnormal points that meet specific persistence and regularity, so that the subsequent electrical signal value optimization is targeted, and the optimization operation is only performed on the minimum pixel set that actually has problems and needs to be processed, thereby effectively improving the image quality while maximizing the preservation of the original details of the current frame image.

[0065] Next, based on different manifestations of the second optimization condition, two optimization strategies are specifically introduced.

[0066] In one implementation, the second optimization condition includes that the electrical signal values of the abnormal pixel points in the continuous multiple frames are all not within the standard electrical signal value range. At this time, if the abnormal pixel points satisfy the second optimization condition, the electrical signal statistics of the pixel points in the target range centered on the abnormal pixel points are determined, and the electrical signal values of the abnormal pixel points are replaced by the electrical signal statistics.

[0067] The electrical signal values of the abnormal pixel points in the current frame in the continuous multiple frames are all not within the standard electrical signal value range, indicating that the abnormal pixel points may have persistent abnormal problems. In order to repair such abnormalities, the electrical signal statistics of other pixel points in the target range centered on the abnormal pixel points are determined. Optionally, the electrical signal statistics include the median, mean or other statistics of the electrical signals of the other pixel points in the target range, to represent the normal electrical signal level of the other pixel points in the target range. Then, the electrical signal values of the original abnormal pixel points are replaced by the electrical signal statistics, thereby realizing optimization of the abnormal pixel points.

[0068] For example, assuming that the electrical signal values of a certain abnormal pixel point in the continuous 5 frames all exceed the standard electrical signal value range, it is determined that the abnormal pixel point is persistent. Then, taking the 8 pixel points in the 3x3 region around the abnormal pixel point as the center, the median of the electrical signal values of these pixel points is calculated as the electrical signal statistics. Finally, the electrical signal values of the abnormal pixel points are replaced by the electrical signal statistics, and the optimization is completed.

[0069] By checking the performance of the abnormal pixel points in the continuous multiple frames, persistent abnormal pixel points are repaired in a targeted manner, which can effectively improve the quality of the generated current frame image. Replacing the electrical signal values of the abnormal pixel points with the electrical signal statistics of the surrounding pixel points can preserve the local consistency of the generated current frame image and reduce false positives.

[0070] In another implementation, the second optimization condition includes that the abnormal pixel point of the current frame has a frequency of abnormality in the continuous multiple frames that is lower than a preset frequency threshold. At this time, if the abnormal pixel point satisfies the second optimization condition, the electrical signal values of the abnormal pixel point in the continuous multiple frames are obtained, and the abnormality indicates that the electrical signal values are out of the standard electrical signal value range; a weighted average electrical signal value is obtained based on the electrical signal values of the abnormal pixel point of the current frame in the continuous multiple frames and respective weight coefficients; and the electrical signal values of the abnormal pixel point are replaced by the weighted average electrical signal value.

[0071] Specifically, the abnormal pixel point of the current frame has a frequency of abnormality in the continuous multiple frames that is lower than a preset frequency threshold, indicating that the abnormality of the pixel point is likely to be sporadic rather than a persistent fault. In order to repair such an abnormal pixel point, the electrical signal values of the abnormal pixel point in the continuous multiple frames are obtained, and it is recorded which frames have electrical signal values of the pixel point out of the standard electrical signal value range. Then, a weighted average electrical signal value is calculated according to the electrical signal values of the pixel point in the continuous multiple frames and respective weight coefficients.

[0072] In some embodiments, the weight coefficient is determined based on the time sequence of the frame data and a decay factor, where the decay factor is used to control the decay speed of the weight coefficient with respect to the time of the corresponding frame data. Optionally, the formula of the weight coefficient can be represented as: w k = e -λk , λ > 0, where k represents the time sequence of certain frame data, and λ represents the decay factor used to control the decay speed of the weight coefficient with respect to the time of the corresponding frame data.

[0073] As can be seen from the formula, the weight coefficient is proportional to the time sequence of the corresponding frame data, that is, the closer the time to the frame data of the current frame, the greater the weight coefficient, which can ensure that the repaired electrical signal value better reflects the actual situation. Finally, the electrical signal values of the abnormal pixel point in the current frame are replaced by the weighted average electrical signal value, thereby realizing the optimization of the abnormal pixel point.

[0074] For example, suppose the abnormal pixel in the current frame is in frame 8. This pixel appeared abnormally in frames 3 and 5 of the previous 7 consecutive frames, for a total frequency of 3 times. If the preset frequency threshold is 4 times, then the frequency of this abnormal pixel appearing abnormally is lower than the preset frequency threshold. Then, the electrical signal values ​​of this abnormal pixel in frames 1 to 7 are obtained, assuming they are 105, 110, 108, 112, 106, 107, and 110 respectively (the standard electrical signal value range is assumed to be 100 to 103), and the weight coefficients are 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, and 0.7 respectively based on time correlation (the most recent frame has the highest weight). The weighted average electrical signal value is calculated as follows: (105×0.1+110×0.2+108×0.3+112×0.4+106×0.5+107×0.6+110×0.7) / (0.1+0.2+0.3+0.4+0.5+0.6+0.7)≈108.54. Finally, this weighted average value of 108.54 is used to replace the electrical signal values ​​of abnormal pixels in the current frame, completing the optimization.

[0075] By examining the frequency of anomalous pixels appearing across multiple consecutive frames, sporadic anomalous pixels can be addressed in a targeted manner. Then, by employing a weighted averaging algorithm combined with temporal correlation weights, the optimized electrical signal values ​​can be ensured to more closely reflect real-world scenarios. This method is particularly effective in complex and dynamic environments, such as autonomous driving scenarios, where it can quickly adapt to changes in lighting and temporary sensor interference.

[0076] remove Figure 1 In the detection method described in this embodiment, the image processing module 130 can also monitor the operating temperature of the light sensing module 110 and / or the filtering module 120, and determine that the current frame image is unreliable when the temperature is too high. Specifically, the image processing module 130 obtains the operating temperature of the light sensing module 110 and / or the filtering module 120; if the operating temperature of the light sensing module 110 and / or the filtering module 120 is greater than a preset temperature threshold, it determines that the current frame image generated based on pixel data is unreliable.

[0077] The light sensing module 110 and / or the filtering module 120 can monitor their working temperatures in real time during operation, and send their working temperatures to the image processing module 130. The image processing module 130 compares the working temperatures of the light sensing module 110 and / or the filtering module 120 with preset temperature thresholds. Optionally, the temperature thresholds are set according to the temperature range of the light sensing module 110 and / or the filtering module 120 when they are working normally, so as to ensure that the two modules can work in a stable state. If the working temperatures of the light sensing module 110 and / or the filtering module 120 exceed the preset temperature thresholds, it is considered that the current working environment or hardware state will affect the accuracy of the generated current frame image, and thus it is determined that the current frame image generated based on the pixel data is untrusted.

[0078] For example, in a high-temperature environment, an autonomous vehicle travels in hot weather, and the light sensing module and the filtering module of the camera can have performance degradation due to overheating. If the temperatures of the two modules exceed the preset temperature thresholds, such as 50℃, the modules can not work normally, resulting in errors in the generated current frame image. Therefore, the image processing module determines that the current frame image is untrusted.

[0079] In a high-temperature environment, the performance of electronic components can be degraded. Therefore, by monitoring the working temperatures of the light sensing module and / or the filtering module in real time and setting corresponding temperature thresholds, the image processing module can discover potential problems in time before the hardware is affected, and avoid using untrusted current frame images. Not only the hardware is protected from high-temperature damage, but also the accuracy and trustworthiness of the current frame image are ensured, thereby improving the stability and safety of the entire camera.

[0080] In order to enhance the functionality of the camera, the image processing module 130 also has the ability to mark and transmit untrusted images, so that the subsequent system can make corresponding decisions or take necessary safety measures according to the information. Specifically, the image processing module 130 adds an untrusted label to the current frame image if it determines that the current frame image generated based on the pixel data is untrusted; and sends the current frame image and the untrusted label corresponding to the current frame image to a data receiving system, so that the data receiving system can determine whether to use the current frame image based on the untrusted label, or execute a corresponding functional safety strategy based on the untrusted label.

[0081] The untrusted label is identification information for indicating that the generated current frame image has quality problems. After receiving the information (including the current frame image and the untrusted label corresponding to the current frame image), the data receiving system can decide whether to use the current frame image according to the untrusted label, or trigger a corresponding functional safety strategy based on the untrusted label.

[0082] Optionally, the data receiving system can be an image analysis system for further processing and analysis of the current frame image, such as target detection, recognition, and tracking; or a decision-making system for making critical decisions based on the current frame image or other sensor data, such as path planning, speed control, and obstacle avoidance; or a display system for displaying the current frame image to a user, such as a driver. The functional safety strategy refers to measures implemented to ensure the safe operation of the autonomous driving system even when the camera malfunctions. Optionally, the functional safety strategy may include switching to a backup camera, limiting the functions of the autonomous driving system, triggering an alarm, or executing a safe stop, etc.

[0083] In this embodiment, an untrusted tag is added to the untrusted image by the image processing module and transmitted to the data receiving system, enabling real-time monitoring and management of image quality and effectively improving the intelligence and reliability of the camera. Then, the data receiving system can promptly determine the usability of the untrusted image based on the untrusted tag, avoiding misjudgments and incorrect decisions caused by using low-quality images, thus enhancing the safety of the autonomous driving system.

[0084] The embodiments corresponding to the camera in this application have been described in detail above. The following will be combined with… Figure 2 This application describes in detail embodiments of the image detection method. It should be understood that the descriptions of the image detection method embodiments correspond to the descriptions of the camera embodiments; therefore, any parts not described in detail can be referred to the preceding embodiments corresponding to the camera.

[0085] Figure 2 The diagram shown is a schematic flowchart of an image detection method provided in an embodiment of this application. Exemplarily, this method is applied to the camera in the aforementioned embodiment. Figure 2 As shown, the method includes the following steps.

[0086] Step S210: Generate pixel data.

[0087] Pixel data includes the electrical signal value of each pixel in the current frame. For example, when light shines on a pixel, light intensity information is converted into an electrical signal value. Additionally, the pixel data includes identification information for each pixel group, used to distinguish and identify different pixel groups. For example, the identification information is a numerical code that enables rapid location of pixel groups in subsequent processing. In some embodiments, a pixel group includes pixel rows and / or pixel columns; that is, a pixel group can be a pixel row, a pixel column, or a combination of both.

[0088] Step S220: Based on pixel data, determine abnormal pixel points in the current frame through a preset standard electrical signal value range, and determine abnormal pixel groups in the current frame through a preset identification verification rule.

[0089] The standard electrical signal value range refers to a range of electrical signal values preset according to normal working conditions of the camera, and is mainly used to distinguish normal pixel points and abnormal pixel points. Optionally, if the electrical signal value of a pixel point is lower than the lower limit of the standard electrical signal value range or higher than the upper limit of the standard electrical signal value range, it indicates that the pixel point may have a fault or be interfered, and is thus determined as an abnormal pixel point. If the electrical signal value of a pixel point is within the standard electrical signal value range, the pixel point is a normal pixel point.

[0090] Specifically, the identification verification rule includes at least one of the following: a rule for verifying that the identification information of each pixel group is unique, a rule for verifying that the identification information of all pixel groups is continuous, a rule for verifying that the identification information of each pixel group conforms to a predetermined format specification, and a rule for verifying that the number of identification information of all pixel groups matches an expected total number. The rule for verifying that the identification information of each pixel group is unique is used to check whether there is repeated identification information (for example, two Z600 columns appear in the same frame), and if repetition is found, the corresponding pixel group is marked as an abnormal pixel group. The rule for verifying that the identification information of all pixel groups is continuous is used to check whether the arrangement of the identification information conforms to a preset order (for example, the column number should be continuously increased from Z001 to Z300), and if the identification information jumps or is out of order, the corresponding pixel group is marked as an abnormal pixel group. The rule for verifying that the identification information of each pixel group conforms to a predetermined format specification is used to check whether the format of the identification information conforms to the specification (for example, the column number should be Z+3 digits, and the length should be fixed as 4 characters), and if the format of the identification information is incorrect, the corresponding pixel group is marked as an abnormal pixel group. The rule for verifying that the number of identification information of all pixel groups matches an expected total number is used to confirm that all pixel groups have identification information, and there is no missing or null value, and if the identification information of a certain pixel group is missing, the corresponding pixel group is marked as an abnormal pixel group.

[0091] In step S230, the non-abnormal pixel group passing the identification verification rule is subjected to a credibility determination on the abnormal pixel points contained in the non-abnormal pixel group; the abnormal pixel group not passing the identification verification rule is determined as an untrustworthy pixel group; and based on the credibility determination results of all pixel groups, it is detected whether the current frame image generated based on the pixel data is trustworthy.

[0092] For example, the number of abnormal pixel points contained in the non-abnormal pixel group passing the identification verification rule is subjected to a credibility determination on the non-abnormal pixel group. Further, based on the number of pixel groups passing the credibility determination, it is detected whether the current frame image generated based on the pixel data is trustworthy. For example, if the number of abnormal pixel points contained in the non-abnormal pixel group is greater than a first number threshold, it is determined that the non-abnormal pixel group is untrustworthy; if the number of pixel groups passing the credibility determination is greater than a second number threshold, it is determined that the current frame image generated based on the pixel data is trustworthy. Otherwise, it is untrustworthy.

[0093] It should be noted that the image credibility refers to that the current frame image is defect-free, or the defect degree thereof is within an allowable range, and can be safely used by other systems (such as an automatic driving system or a decision system).

[0094] In the embodiment, the pixel data not only contains the electrical signal value, but also contains the identification information of the pixel group, thereby providing a double dimension for positioning of the abnormal pixel point. That is, the abnormal pixel point can be quickly screened based on the preset standard electrical signal value range, or the abnormal pixel group can be identified through the identification checking rule. Then, for the non-abnormal pixel group passing the identification checking rule, the credibility is not directly adopted, but is further determined based on the abnormal pixel point possibly existing in the non-abnormal pixel group, thereby improving the accuracy and reliability of detection. For the abnormal pixel group not passing the identification checking rule, the abnormal pixel group is directly determined as an untrustworthy pixel group, thereby avoiding a deeper data processing error possibly caused by identification information error, and ensuring the integrity and reliability of the image. Finally, the credibility determination results of all the pixel groups are comprehensively considered to determine whether the current frame image is credible, and then only the high-quality and credible current frame image is used for decision, thereby avoiding misjudgment caused by image abnormality, and enhancing the safety of automatic driving.

[0095] The electronic device according to the embodiments of the present application will be described below with reference to Figure 3 . Figure 3 As shown in the structural schematic diagram of the electronic device provided by an exemplary embodiment of the present application.

[0096] As shown in Figure 3 , the electronic device 30 includes one or more processors 301 and a memory 302.

[0097] The processor 301 can be a central processing unit (CPU) or other forms of processing units having data processing capability and / or instruction execution capability, and can control other components in the electronic device 30 to perform desired functions.

[0098] The memory 302 can include one or more computer program products that can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory, for example, can include random access memory (RAM), cache memory, and / or the like. The non-volatile memory, for example, can include read-only memory (ROM), hard disk drives, solid-state drives, and / or the like. The computer-readable storage media can store one or more computer program instructions executable by the processor 301 to implement the image detection method of various embodiments of the present application described above and / or other desired functions. The computer-readable storage media can also store various contents such as pixel data, standard electrical signal value ranges, abnormal pixel points, and abnormal pixel groups.

[0099] In one example, the electronic device 30 can further include an input device 303 and an output device 304, which are interconnected through a bus system and / or other forms of connection mechanisms (not shown).

[0100] The input device 303 can include, for example, a keyboard, a mouse, and / or the like.

[0101] The output device 304 can output various information including pixel data, standard electrical signal value ranges, abnormal pixel points, and abnormal pixel groups to the outside. The output device 304 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and / or the like.

[0102] Of course, in order to simplify, Figure 3 Only some of the components of the electronic device 30 related to the present application are shown in FIG. 3, and components such as buses, input / output interfaces, and / or the like are omitted. In addition, the electronic device 30 can include any other appropriate components according to the specific application.

[0103] In addition to the above-described method and device, an embodiment of the present application can be a computer program product including computer program instructions that, when executed by a processor, cause the processor to perform the steps of the image detection method according to various embodiments of the present application described above in the specification.

[0104] The computer program product can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. The embodiments of the present application are not limited by the programming languages made use of, and the use of "software" should be taken to mean any programmable entity whatsoever.

[0105] In addition, an embodiment of the present application can also be a computer readable storage medium, having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform the steps described above in the image detection method according to various embodiments of the present application.

[0106] The computer readable storage medium can be any combination of one or more computer readable medium(s). The computer readable medium can be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0107] The above describes the basic principles of the present application in combination with specific embodiments, but it should be noted that the advantages, benefits, effects and the like mentioned in the present application are only examples and are not limiting, and these advantages, benefits, effects and the like cannot be considered as mandatory for each embodiment of the present application. In addition, the above specific details are only for the purpose of example and understanding, and are not limiting, and the above details do not limit the present application to the specific details described above.

[0108] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0109] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0110] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0111] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A camera, characterized in that, include: A photosensitive module with a pixel array is configured to generate pixel data, the pixel data including the electrical signal value of each pixel in the current frame and the identification information of each pixel group, the pixel group including pixel rows and / or pixel columns; The filtering module is configured to determine abnormal pixels in the current frame based on the pixel data using a preset standard electrical signal value range, and to determine abnormal pixel groups in the current frame using preset identifier verification rules. The identifier verification rules include at least one of the following: a rule verifying the uniqueness of the identifier information for each pixel group; a rule verifying the continuity of the identifier information for all pixel groups; a rule verifying that the identifier information for each pixel group conforms to a predetermined format specification; and a rule verifying that the number of identifier information for all pixel groups matches the expected total number. The image processing module is configured to: determine the credibility of the non-abnormal pixel group containing abnormal pixels that pass the identification verification rule; determine the abnormal pixel group that fails the identification verification rule as an untrusted pixel group; and detect whether the current frame image generated based on the pixel data is trustworthy based on the credibility determination results of all pixel groups.

2. The camera according to claim 1, characterized in that, The image processing module is also configured to: Determine a first ratio between the number of abnormal pixels in the non-abnormal pixel group and the total number of pixels in the non-abnormal pixel group. If the first ratio is greater than a first ratio threshold, then determine that the non-abnormal pixel group is an untrusted pixel group. A second ratio is determined between the number of untrusted pixel groups in the current frame and the total number of pixel groups. If the second ratio is greater than a second ratio threshold, the current frame image generated based on the pixel data is determined to be untrusted.

3. The camera according to claim 2, characterized in that, The image processing module is further configured to dynamically adjust the second proportional threshold based on the light intensity data of the current environment, wherein the second proportional threshold increases as the light intensity data increases; and / or, The image processing module is further configured to set different second ratio thresholds for the region of interest and the region of non-interest in the current frame image, wherein the second ratio threshold for the region of interest is smaller than the second ratio threshold for the region of non-interest.

4. The camera according to claim 3, characterized in that, The second proportional threshold is determined based on the minimum light intensity data of the current environment, the maximum light intensity data of the current environment, and the normalization coefficient.

5. The camera according to claim 1, characterized in that, The image processing module is also configured to: For each abnormal pixel in the current frame, based on the number of abnormal pixels within the target range of the abnormal pixel, it is determined whether the abnormal pixel satisfies the first optimization condition; If the abnormal pixel meets the first optimization condition, then the abnormal pixel is optimized to obtain the normal pixel corresponding to the abnormal pixel. Wherein, the abnormal pixels included in the non-abnormal pixel group refer to the abnormal pixels in the non-abnormal pixel group that do not meet the first optimization condition; Preferably, the first optimization condition includes the number of abnormal pixels within the target range being less than a first quantity threshold.

6. The camera according to claim 5, characterized in that, The image processing module is also configured to: If the abnormal pixel satisfies the first optimization condition, then the abnormal persistence feature of each abnormal pixel in the current frame is determined. The abnormal persistence feature is used to characterize the behavior pattern of the abnormal pixel deviating from the standard electrical signal value range in multiple consecutive frames. Based on the abnormal persistence characteristics of each abnormal pixel, determine whether each abnormal pixel satisfies the second optimization condition; If the abnormal pixel meets the second optimization condition, then the electrical signal value of the abnormal pixel is optimized.

7. The camera according to claim 6, characterized in that, The second optimization condition includes that the electrical signal values ​​of the abnormal pixels in multiple consecutive frames are not within the range of the standard electrical signal values; the image processing module is further configured to: If the abnormal pixel meets the second optimization condition, then the electrical signal statistics of the pixels within the target range centered on the abnormal pixel are determined; The electrical signal value of the abnormal pixel is replaced with the electrical signal statistics.

8. The camera according to claim 6, characterized in that, The second optimization condition includes that the frequency of abnormal pixels appearing in multiple consecutive frames is lower than a preset frequency threshold, wherein the occurrence of abnormality indicates that the electrical signal value exceeds the standard electrical signal value range; the image processing module is further configured to: If the abnormal pixel satisfies the second optimization condition, then the electrical signal value of the abnormal pixel in multiple consecutive frames is obtained; Based on the electrical signal values ​​of the abnormal pixels in the current frame and their respective weight coefficients in multiple consecutive frames, a weighted average electrical signal value is obtained, wherein the weight coefficients are proportional to the temporal correlation of the corresponding frame data. The electrical signal value of the abnormal pixel is replaced with the weighted average electrical signal value; Preferably, the weighting coefficient is determined based on the timing of the frame data and the decay factor, wherein the decay factor is used to control the decay rate of the weighting coefficient as the corresponding frame data decays over time.

9. The camera according to claim 1, characterized in that, The image processing module is also configured to: Obtain the operating temperature of the light sensing module and / or the filtering module; If the operating temperature of the light sensing module and / or the filtering module is greater than a preset temperature threshold, then the current frame image generated based on the pixel data is determined to be unreliable.

10. The camera according to claim 1, characterized in that, The image processing module is also configured to: If it is determined that the current frame image generated based on the pixel data is untrustworthy, then an untrustworthy tag is added to the current frame image; The current frame image and the corresponding untrusted tag are sent to the data receiving system so that the data receiving system can determine whether to use the current frame image based on the untrusted tag. Alternatively, the data receiving system can execute a corresponding functional safety policy based on the untrusted tag. The functional safety policy refers to measures designed to ensure that the autonomous driving system can still operate safely when the camera malfunctions.

Images