Image recognition method and device, electronic equipment and vehicle

Abstract

The invention provides an image recognition method and device, electronic equipment and a vehicle, and is applied to the technical field of image processing, and the method comprises the steps: adjusting the luminous intensity of a light supplementing light source of a camera device from a first luminous intensity to a second luminous intensity, the first luminous intensity being a luminous intensity corresponding to normal exposure, and the second luminous intensity being a luminous intensity corresponding to normal exposure; the second luminous intensity is the luminous intensity corresponding to abnormal exposure; collecting an environment image under the second luminous intensity through a camera device, and judging whether the environment image meets a preset shielding condition or not; and in response to determining that the environment image meets the preset shielding condition, determining that the camera device is shielded. The condition that the camera device is shielded can be recognized more accurately, so that a driver can be correspondingly prompted, the state and behavior of the driver are monitored in real time, accidents caused by fatigue or distraction of the driver are reduced, and the overall road traffic safety is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image recognition method, device, electronic equipment and vehicle. Background Art

[0002] Driver monitoring systems use cameras and sensors to monitor the driver's status and behavior in real time, such as fatigue and distraction, to prevent potential safety risks. However, the effectiveness of driver monitoring systems may be affected by many factors. For example, the inability to accurately identify situations where the camera is blocked may result in the inability to correctly monitor and judge the driver's fatigue or distraction status, increasing the risk of traffic accidents. Summary of the invention

[0003] In view of this, the purpose of the present application is to propose an image recognition method, device, electronic equipment and vehicle to accurately identify the situation where the camera device is blocked to avoid increasing the risk of traffic accidents.

[0004] Based on the above objectives, the present application provides an image recognition method, comprising:

[0005] Adjusting the luminous intensity of the fill light source of the camera device from a first luminous intensity to a second luminous intensity, wherein the first luminous intensity is the luminous intensity corresponding to normal exposure, and the second luminous intensity is the luminous intensity corresponding to abnormal exposure;

[0006] Capturing an environment image at a second luminous intensity by means of a camera device, and determining whether the environment image meets a preset shielding condition;

[0007] In response to determining that the environment image meets a preset blocking condition, it is determined that the camera device is blocked.

[0008] Based on the same inventive concept, the present application also provides an image recognition device, comprising:

[0009] an adjusting module, configured to adjust the luminous intensity of the fill light source of the camera device from a first luminous intensity to a second luminous intensity, wherein the first luminous intensity is the luminous intensity corresponding to normal exposure, and the second luminous intensity is the luminous intensity corresponding to abnormal exposure;

[0010] The judgment module is configured to collect an environmental image at a second luminous intensity through a camera device, and judge whether the environmental image meets a preset shielding condition; in response to determining that the environmental image meets the preset shielding condition, determine that the camera device is blocked.

[0011] Based on the same inventive concept, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method as described above when executing the computer program.

[0012] Based on the same inventive concept, the present application also provides a vehicle, which includes the above-mentioned electronic device.

[0013] From the above, it can be seen that the image recognition method, device, electronic device and vehicle provided by the present application, wherein the method includes: by adjusting the luminous intensity of the fill light source of the camera device from a first luminous intensity to a second luminous intensity, wherein the first luminous intensity is the luminous intensity corresponding to normal exposure, and the second luminous intensity is the luminous intensity corresponding to abnormal exposure; by the camera device collecting the environmental image at the second luminous intensity, it is possible to provide a more accurate environmental image for subsequent judgment of the light reflection conditions of obstructions with different reflective properties; judging whether the environmental image meets the preset obstruction conditions, when some obstructions with better reflective properties are close to the camera device, the light of the fill light source of the camera device is strongly reflected by the obstruction. When some obstructions with poor reflective performance are close to the camera device, the light of the fill light source of the camera device is scattered or partially reflected, and the image collected by the camera device appears to be completely black, completely gray or partially overexposed. Therefore, in response to determining that the environmental image meets the preset obstruction condition, it is determined that the camera device is obstructed; the environmental image collected by the camera device under the second luminous intensity is used to accurately identify the obstruction of the camera device, so as to give corresponding prompts to the driver, such as removing the obstruction to monitor the driver's status and behavior in real time, reduce accidents caused by driver fatigue or distraction, and improve overall road traffic safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0015] Figure 1 This is a flow chart of the image recognition method according to an embodiment of the present application;

[0016] Figure 2 This is a flowchart of an image recognition method according to another embodiment of the present application;

[0017] Figure 3 A schematic diagram of an image recognition device according to an embodiment of the present application;

[0018] Figure 4 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0019] Figure 5 A schematic diagram of an image being overexposed in the image recognition method according to an embodiment of the present application;

[0020] Figure 6 A schematic diagram of underexposure of an image in the image recognition method according to an embodiment of the present application;

[0021] Figure 7 A schematic diagram of a preset area in the image recognition method of an embodiment of the present application;

[0022] Figure 8 A schematic diagram of a frame group image in the image recognition method according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0024] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0025] In the related art, with the advancement of technology, various advanced driver assistance systems have been developed and applied to commercial and private vehicles to improve road safety and reduce traffic accidents. Among them, the driver monitoring system, as an important component of the advanced driver assistance system, uses cameras and sensors to monitor the driver's status and behavior, such as fatigue, distraction, etc. in real time, thereby preventing potential safety risks. However, the effectiveness of these systems may be affected by many factors. One of the main problems is that the camera may be blocked, which will seriously affect the function of the system. Camera occlusion can be divided into several situations: one is that the driver intentionally or accidentally blocks the camera with his hands or other objects; the second is the internal design of the vehicle, such as the camera blocking slide, which may block the camera's line of sight at an inappropriate time; the third is that the external environmental factors (such as mud splashing) during the vehicle's driving process cause the camera to be blocked. These occlusion situations will cause the monitoring system to be unable to accurately capture the driver's facial expressions and eye dynamics, and thus cannot correctly judge the driver's fatigue or distraction state. If the monitoring system fails to issue warnings in a timely and accurate manner, the risk of traffic accidents may increase.

[0026] Based on the above problems, the applicant found that: when some obstructions with good reflective properties are close to the camera device, if the camera device is provided with a fill light source (the fill light source can be an infrared lamp, the light emitted by the infrared lamp is invisible to the human eye, but the camera device can capture it), the light of the fill light source illuminates the obstruction, the light is strongly reflected by the obstruction, and the image captured by the camera device shows an overall or large area overexposure phenomenon (such as Figure 5 As shown in the figure, when some obstructions with poor reflective properties are close to the camera device, the light from the fill light source of the camera device is scattered or partially reflected, and the image collected by the camera device appears completely black, completely gray, or partially overexposed due to underexposure (as shown in the figure). Figure 6 As shown), if the camera device is not provided with a fill light source, the camera device cannot accurately capture the exposure image of the high-reflection or low-reflection obstruction, and thus cannot effectively identify whether the obstruction exists. By adjusting the luminous intensity of the fill light source of the camera device from a first luminous intensity to a second luminous intensity (the first luminous intensity is the luminous intensity corresponding to normal exposure, and the second luminous intensity is the luminous intensity corresponding to abnormal exposure), the camera device captures an environmental image at the second luminous intensity, and determines whether the environmental image meets the preset obstruction condition; in response to determining that the environmental image meets the preset obstruction condition, it is determined that the camera device is obstructed. By accurately identifying the situation in which the camera device is obstructed through the environmental image captured by the camera device at the second luminous intensity, the driver can be prompted to remove the obstruction, thereby monitoring the driver's status and behavior in real time, reducing accidents caused by driver fatigue or distraction, and improving overall road traffic safety.

[0027] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0028] The present application provides an image recognition method, such as Figure 1 As shown, in some embodiments, the method is executed by a vehicle controller or a data processor independently provided with the vehicle controller, and the subsequent embodiments are all illustrated by taking the vehicle controller as an example; the method includes S101-S103.

[0029] S101, adjusting the luminous intensity of the fill light source of the camera device from a first luminous intensity to a second luminous intensity, wherein the first luminous intensity is the luminous intensity corresponding to normal exposure, and the second luminous intensity is the luminous intensity corresponding to abnormal exposure;

[0030] In this step, the "luminous intensity corresponding to normal exposure" refers to the luminous intensity of the fill light source of the camera device without determining whether the camera device is blocked. It should be understood that the luminous intensity is not a fixed value, and the fill light source can be automatically adjusted according to the lighting conditions.

[0031] In this step, the "luminous intensity corresponding to abnormal exposure" refers to the luminous intensity of the fill light source when determining whether the camera device is blocked. It should be understood that the luminous intensity is compared with the luminous intensity when the camera device is not blocked under the same lighting conditions. In addition, the luminous intensity is usually stronger or weaker than the luminous intensity corresponding to normal exposure.

[0032] The luminous intensity of the fill light source of the camera device is adjusted from the first luminous intensity to the second luminous intensity. The environmental image collected by the camera device at the luminous intensity corresponding to the abnormal exposure can provide more accurate data support for the subsequent judgment of the light reflection situation of the obstruction.

[0033] S102, collecting an environment image at a second luminous intensity through a camera device, and determining whether the environment image meets a preset shielding condition;

[0034] In specific implementation, under the second luminous intensity, that is, after being adjusted to the luminous intensity corresponding to abnormal exposure, the environmental image collected by the camera device will display the environmental visual effect under the adjusted lighting conditions to determine whether the environmental image meets the preset shielding conditions. The principle is that when some obstructions with good reflective performance are close to the camera device, the light of the fill light source of the camera device is strongly reflected by the obstruction, and the environmental image collected by the camera device presents the phenomenon of overall or large-area overexposure. When some obstructions with poor reflective performance are close to the camera device, the light of the fill light source of the camera device is scattered or partially reflected, and the image collected by the camera device presents the situation of full black, full gray or partial overexposure. Therefore, when the environmental image meets the above-mentioned situation of large-area overexposure or full black, full gray and partial overexposure, it can be determined that the camera device is blocked by the obstruction. In the present application, the environmental image is collected by the camera device under the second luminous intensity, and the second luminous intensity is the luminous intensity corresponding to abnormal exposure. The luminous intensity corresponding to abnormal exposure is usually stronger or weaker than normal exposure, that is, underexposure and overexposure. Specifically, by capturing an environmental image under underexposure, if there is no camera device blocking it, the environmental image will appear overall dark. However, if the environmental image still appears overall or overexposed over a large area, it is possible to accurately determine that the camera is blocked by an obstruction, thus avoiding misjudgment due to excessive luminous intensity of the fill light source. By capturing an environmental image under overexposure, if there is no camera device blocking it, the environmental image will appear large-area highlight areas or overall whiteness. However, if the environmental image still appears completely black, completely gray, or partially overexposed, it is possible to accurately determine that the camera is blocked by an obstruction, thus avoiding misjudgment due to excessively low luminous intensity of the fill light source.

[0035] Therefore, it can be understood that the second luminous intensity may include a first sub-luminous intensity and a second sub-luminous intensity, and the first sub-luminous intensity is a luminous intensity corresponding to underexposure, and the second sub-luminous intensity is a luminous intensity corresponding to overexposure, then the environmental image captured by the camera device at the second luminous intensity includes the first environmental image captured at the first sub-luminous intensity, and the second environmental image captured at the second sub-luminous intensity. Correspondingly, determining that the environmental image meets the preset shielding condition includes: if it is determined that the first environmental image does not meet the preset shielding condition, but it is not possible to fully determine whether the camera device is blocked at this time (because there may be an obstruction with poor reflective performance, the first environmental image obtained at the first sub-luminous intensity will not show an overall or large-area overexposure phenomenon), therefore determining whether the second environmental image meets the preset shielding condition; if it is determined that the second environmental image meets the preset shielding condition, it indicates that the second environmental image captured by the camera device presents a situation of all black, all gray or partial overexposure, and it is determined that the environmental image meets the preset shielding condition and the camera device is blocked. For example, the camera device captures 30 frames of images every second, among which frames 1-4 are images obtained by fill-lighting the fill-light source at a luminous intensity corresponding to underexposure (first sub-luminous intensity), and frames 5-8 are images obtained by fill-lighting the fill-light source at a luminous intensity corresponding to overexposure (second sub-luminous intensity). Then, the 4th frame of image 1-4 can be selected as the first environmental image, and the 8th frame of image 5-8 can be selected as the second environmental image. At this time, the grayscale values ​​of multiple sampling points in the first environment image are determined, and the sampling points with grayscale values ​​greater than or equal to the preset grayscale values ​​are determined as target sampling points. If the number of target sampling points is less than the preset number, it is determined that the first environment image does not meet the preset occlusion condition (if the number of target sampling points is greater than or equal to the preset number, it is determined that the first environment image meets the preset occlusion condition). Therefore, it is determined whether the second environment image meets the preset occlusion condition, that is, the grayscale values ​​of all pixels in the preset area of ​​the second environment image are obtained, and the grayscale value variance of all pixels is determined. If it is determined that the grayscale value variance is less than or equal to the preset grayscale value variance, it is determined that the second environment image meets the preset occlusion condition (if it is determined that the grayscale value variance is greater than the preset grayscale value variance, it is determined that the second environment image does not meet the preset occlusion condition), it is determined that the environment image meets the preset occlusion condition, and it is determined that the camera device is blocked. If it is determined that the first environment image meets the preset occlusion condition, it is not necessary to judge whether the second environment image meets the preset occlusion condition.It should be noted that since the second environmental image is collected at a luminous intensity corresponding to overexposure, there is a situation where reflective phenomena in the environment interfere with the judgment result. If it is mistakenly judged that there is an obstruction, the next step of judgment will not be performed at this time, resulting in a decrease in the accuracy of the judgment result. Compared with the first environmental image, which is collected at a luminous intensity corresponding to underexposure, there is no interference from reflective phenomena in the environment. Therefore, it is more accurate to first judge whether there is an obstruction through the first environmental image than to first judge whether there is an obstruction through the second environmental image. Judging whether there is an obstruction through the second environmental image is a supplementary judgment when it is judged that there is no obstruction through the first environmental image, which can improve the accuracy of the overall judgment result.

[0036] In addition, since 9-30 frames of images are images obtained by fill-lighting the fill-light source at the luminous intensity corresponding to normal exposure, when it is determined whether the camera device is blocked in the next second, the camera device continues to collect 30 frames of images. At this time, the luminous intensity of the fill-light source of the camera device is again adjusted from the first luminous intensity (the luminous intensity corresponding to normal exposure) to the second luminous intensity (that is, the luminous intensity corresponding to abnormal exposure, 1-4 frames of images are the luminous intensity corresponding to underexposure, and 5-8 frames of images are the luminous intensity corresponding to overexposure), and the camera device again collects the environmental image at the second luminous intensity, and determines whether the environmental image meets the preset blocking condition to determine whether the camera device is blocked, that is, the process of determining whether the camera device is blocked is performed once per second.

[0037] The above description only uses the image captured by the camera device within one second to determine whether the camera device is blocked. It is also possible to determine whether the camera device is blocked by the results determined every second within a certain period of time. For example, within 5 seconds, if the judgment result every second is that the first environmental image does not meet the preset blocking condition, and it is determined that the second environmental image meets the preset blocking condition, that is, for the judgment result every second, the judgment results for each frame (the 4th frame and the 8th frame as described above) are the same, then it is determined that the camera device is blocked, otherwise the camera device is not blocked, so that it can more accurately determine whether the camera device is blocked to avoid misjudgment.

[0038] S103: In response to determining that the environment image meets a preset blocking condition, determine that the camera device is blocked.

[0039] In specific implementation, when the environmental image analysis shows that the preset occlusion conditions are met, such as when the environmental image in the figure meets the conditions of large-area overexposure or all black, all gray, and partial overexposure, it indicates that the camera is blocked by some object. At this time, it is confirmed that the camera device is blocked. After confirming that the camera is blocked, corresponding measures can be taken, including but not limited to issuing visual or sound warnings to the driver, prompting the driver that the camera device is blocked and needs to be dealt with immediately to ensure driving safety. Ensure that the camera device can work effectively at critical moments, and promptly discover and solve problems that may affect driving safety. By promptly identifying and responding to camera occlusion, accidents that may be caused by failure to properly monitor the driver's status are avoided.

[0040] In this embodiment, camera occlusion may cause the driver monitoring system to be unable to accurately capture the driver's status, such as fatigue or distraction. By accurately detecting and identifying occlusion and responding in time, it can be ensured that the camera device can work effectively at critical moments, thereby ensuring that the driver's behavior and status can be effectively monitored and traffic accidents that may be caused by monitoring failure can be prevented.

[0041] In some embodiments, Figure 2 As shown, the abnormal exposure includes underexposure and overexposure; and the determining whether the environmental image meets the preset shielding condition includes S201 and S202.

[0042] S201, in response to the second luminous intensity being a luminous intensity corresponding to underexposure, determining grayscale values ​​of a plurality of sampling points in the environment image;

[0043] Underexposure means that during the normal image acquisition process of the camera device, the reflection brightness of the object is low due to the dark lighting conditions of the fill light source, so that the brightness of the image collected by the camera device is low. The luminous intensity corresponding to underexposure is usually weaker than the luminous intensity corresponding to normal exposure. When the second luminous intensity is the luminous intensity corresponding to underexposure, it is necessary to detect whether the environmental image still shows the phenomenon of overall or large-area overexposure (because under normal circumstances, if the second luminous intensity is the luminous intensity corresponding to underexposure, the environmental image collected by the camera device will usually show a large area of ​​gray, but if the environmental image still shows overall or large-area overexposure, it can be determined that the camera device is blocked by an obstruction with good reflective performance), so the grayscale values ​​of multiple sampling points in the environmental image are determined; the grayscale value is the brightness of the image pixel. The grayscale value of the pixel point of the environmental image is large, indicating that the pixel point is in the state of overexposure, and the grayscale value of the pixel point of the environmental image is small, indicating that the pixel point is in the state of underexposure.

[0044] S202: In response to determining that the number of target sampling points is greater than or equal to a preset number, determine that the environment image meets a preset occlusion condition; wherein the target sampling point is a sampling point whose grayscale value is greater than or equal to a preset grayscale value.

[0045] In a specific implementation, when the grayscale value of the sampling point is greater than or equal to the preset grayscale value (exemplarily, the preset grayscale value can be set to 230), it can be determined that the sampling point is in an overexposed state. At this time, the sampling point is determined as a target sampling point, and the target sampling points are counted. When the number of target sampling points is greater than or equal to a preset number (exemplarily, the preset number can be set to 5% of the number of pixels in the environmental image), it indicates that the environmental image meets the condition of large-area overexposure, and it is determined that the environmental image meets the preset shielding condition.

[0046] In this embodiment, by accurately measuring the grayscale values ​​of multiple sampling points in the environmental image and comparing them with the preset grayscale value threshold, it is possible to more accurately determine whether there is occlusion, reduce the possibility of misjudgment, detect occlusion in a timely manner, and ensure the effectiveness of the monitoring system. By timely detecting and processing occlusion, the continued effectiveness of the monitoring system can be ensured, thereby maintaining driving safety.

[0047] In some embodiments, determining the grayscale values ​​of a plurality of sampling points in the environment image includes:

[0048] A sampling image is determined in a preset detection area of ​​the environmental image by a preset sampling frame, and grayscale values ​​of a plurality of sampling points in the sampling image are determined.

[0049] In specific implementation, the detection area can be preset according to the position and viewing angle of the camera. The preset detection area is the part of the environment image that is most likely to be blocked, such as the front area facing the camera or the area near the driver's face. In the preset detection area, a preset sampling frame is defined. The preset sampling frame is a small area in the environment image, and the state of the entire environment image is evaluated by analyzing the grayscale values ​​of multiple sampling points in the preset sampling frame. Specifically, the preset sampling frame can be slid from the starting position with a preset step length in the preset detection area of ​​the environment image, and the sampling image of the preset sampling frame at the position is obtained, and the grayscale values ​​of multiple sampling points in the sampling image are determined. If the grayscale value of the sampling point in the sampling image is greater than or equal to the preset grayscale value, the sampling point is determined as the target sampling point, and the target sampling points are counted. If the number of target sampling points in the sampling image is less than the preset number, it indicates that the detection result of the current sampling image cannot determine whether the environment image meets the situation of large-area overexposure. At this time, the preset sampling frame slides to the next position with a preset step length in the preset detection area of ​​the environment image, and the sampling image of the preset sampling frame at the position is obtained again. Repeat the above steps. If it is determined that the number of target sampling points in the sampled image is greater than or equal to the preset number, it indicates that the environmental image meets the condition of large-area overexposure, and it is determined that the environmental image meets the preset occlusion condition. At this time, the preset sampling frame stops sliding; if the preset sampling frame traverses the preset detection area of ​​the environmental image and still has not determined a sampled image in which the number of target sampling points is greater than or equal to the preset number, it indicates that the environmental image is normal, and the camera device is not blocked at this time.

[0050] In this embodiment, by using the sampling frame detection within the preset detection area, the specific area where the occlusion occurs can be more accurately located. The position and size of the sampling frame are adjusted according to the specific position and viewing angle of the camera, which can work effectively in a variety of different application scenarios. By checking each sampled image and calculating the number of target sampling points, the situation of misjudging occlusion due to accidental high gray value points can be reduced. By ensuring that the camera is not blocked, the monitoring system can continue to operate and effectively monitor the driver's status, thereby improving overall road safety.

[0051] In some embodiments, the abnormal exposure includes underexposure and overexposure; and the determining whether the environment image meets the preset shielding condition includes:

[0052] In response to the second luminous intensity being a luminous intensity corresponding to overexposure, obtaining grayscale values ​​of all pixels in a preset area in the environment image, and determining a grayscale value variance of all pixels;

[0053] Overexposure means that during the normal image acquisition process of the camera device, the reflection brightness of the object is higher due to the brighter lighting conditions of the fill light source, so that the brightness of the image collected by the camera device is higher. The luminous intensity corresponding to overexposure is usually stronger than the luminous intensity corresponding to normal exposure. When the second luminous intensity is the luminous intensity corresponding to overexposure, it is necessary to detect whether the environmental image still appears to be completely black, completely gray, or partially overexposed (because under normal circumstances, if the second luminous intensity is the luminous intensity corresponding to overexposure, the environmental image captured by the camera device will usually present a large area of ​​highlight area or even completely white, but if the environmental image still appears to be completely black, completely gray, or partially bright, it can be determined that the camera device is blocked by an obstruction with poor reflective performance). Therefore, the grayscale values ​​of all pixels in a preset area of ​​the environmental image are obtained, and the grayscale value variance of all pixels is determined. Because when the environmental image appears to be completely black, completely gray, or partially overexposed, the grayscale value variance of all pixels in the preset area is small, and the camera device can normally capture the in-vehicle environmental image when it is not blocked. For the normally captured in-vehicle environmental image, the grayscale value variance of all pixels in the preset area is large. Therefore, by determining the grayscale value variance of all pixels, it can be determined whether the camera device is blocked. At the same time, the preset area can be set to an area where the grayscale value variance of pixels is usually large, such as Figure 7 Shown are area A1 (vehicle A-pillar) and area A2 (vehicle roof).

[0054] In response to determining that the gray value variance is less than or equal to a preset gray value variance, it is determined that the environment image meets a preset occlusion condition.

[0055] In specific implementation, the gray value variance is a statistical measure used to measure the dispersion of pixel brightness in the environmental image. Under normal circumstances without occlusion, the brightness in the environmental image should show a certain change, which reflects the uneven distribution of light in the natural environment. A higher gray value variance usually indicates that there are rich brightness changes in the environmental image, while a lower variance value may indicate that the image brightness is too consistent, such as when the environmental image appears completely black, completely gray, or partially overexposed. When it is determined that the gray value variance is less than or equal to the preset gray value variance (exemplarily, the preset gray value variance can be set to the gray value variance of the area corresponding to the A-pillar and the ceiling of the vehicle in the normal environmental image), it indicates that the brightness in the environmental image is too uniform, which is usually related to the presence of an obstruction in front of the camera, so it is determined that the environmental image meets the preset occlusion condition.

[0056] In this embodiment, the gray value variance provides a method for quantifying the brightness distribution of the environment image. By comparing the gray value variance and the preset gray value variance, it is possible to accurately determine whether the environment image has abnormally uniform brightness due to occlusion, reduce the uncertainty of relying on subjective judgment, improve the objectivity and accuracy of detection, and quickly identify the gray value abnormality caused by occlusion, so that timely response can be made, such as warning the driver or taking treatment measures, maintaining continuous monitoring of the driver's status, and providing a more solid guarantee for driving safety.

[0057] In some embodiments, the abnormal exposure includes underexposure and overexposure, the second luminous intensity includes a first sub-luminous intensity and a second sub-luminous intensity, and the first sub-luminous intensity is a luminous intensity corresponding to underexposure, and the second sub-luminous intensity is a luminous intensity corresponding to overexposure, and the environmental image includes a first environmental image and a second environmental image; and collecting the environmental image at the second luminous intensity by a camera device includes:

[0058] Capturing a first environment image at a first sub-luminous intensity and a second environment image at a second sub-luminous intensity by a camera device;

[0059] In a specific implementation, the second luminous intensity includes a first sub-luminous intensity and a second sub-luminous intensity, and the first sub-luminous intensity is a luminous intensity corresponding to underexposure. The second sub-luminous intensity is a luminous intensity corresponding to overexposure. When the luminous intensity of the fill light source is adjusted to the first sub-luminous intensity and the second sub-luminous intensity, the camera device collects a first environment image at the first sub-luminous intensity, and collects a second environment image at the second sub-luminous intensity, so as to fully judge the shielding condition of the camera device through the first environment image and the second environment image, that is, to provide reliable and complete image data for detecting shielding objects with better reflective performance and shielding objects with poor reflective performance.

[0060] Accordingly, determining that the environment image meets the preset occlusion condition includes:

[0061] In response to determining that the first environment image does not meet the preset occlusion condition, determining whether the second environment image meets the preset occlusion condition;

[0062] In a specific implementation, when the first environment image is collected by the camera device under the first sub-luminous intensity, the grayscale values ​​of multiple sampling points in the first environment image are determined. If the number of target sampling points (target sampling points are sampling points with grayscale values ​​greater than or equal to the preset grayscale value) is greater than or equal to the preset number, it is determined that the first environment image meets the preset occlusion condition. If the number of target sampling points is less than the preset number, it is determined that the first environment image does not meet the preset occlusion condition, that is, the environment image does not show an overall or large-area overexposure phenomenon. At this time, it is determined whether the second environment image meets the preset occlusion condition, and the occlusion condition of the camera device is comprehensively judged to avoid misjudgment or missed judgment.

[0063] In response to determining that the second environment image meets the preset occlusion condition, it is determined that the environment image meets the preset occlusion condition.

[0064] In a specific implementation, when the second environment image is captured by the camera device at the second sub-luminous intensity, the grayscale values ​​of all pixels in the preset area of ​​the environment image are obtained, and the grayscale value variance of all pixels is determined. In response to determining that the grayscale value variance is less than or equal to the preset grayscale value variance, that is, the second environment image presents a situation of full black, full gray, or partial overexposure, it is determined that the environment image meets the preset occlusion condition.

[0065] In this embodiment, by collecting images under two different luminous intensities, the environmental conditions can be evaluated more comprehensively, and occlusions can be detected under different lighting conditions, thereby reducing misjudgments caused by a single lighting condition. The analysis of the first environmental image and the second environmental image can help identify different types of occlusions, such as occlusions with better reflective properties and occlusions with poor reflective properties, to more accurately determine whether there is an occlusion, ensure the reliability of the judgment, and reduce misjudgments and missed judgments that may be caused by single image analysis.

[0066] In some embodiments, collecting the first environment image at the first sub-luminous intensity by a camera device includes:

[0067] Collecting a first frame group image at a first sub-luminous intensity by a camera device, determining an average grayscale value of all pixels of each frame image in the first frame group image, and determining an image with a minimum average grayscale value as a first environment image;

[0068] In specific implementation, under the first sub-luminous intensity, the camera device will continuously collect multiple frames of images to form an image frame group, that is, the first frame group image, and calculate the average grayscale value of all pixels in each frame of the image. For example, the camera device collects 30 frames of images per second, wherein the images of frames 1-4 are the first frame group images obtained by the fill light source under the first sub-luminous intensity (luminous intensity corresponding to underexposure), and the average grayscale value of all pixels in each frame of the image in frames 1-4 is calculated. The grayscale value reflects the brightness of the image, and the average grayscale value provides a quantitative indicator to evaluate the overall brightness of each frame of the image. In the first frame group of images, the image with the smallest average grayscale value will be selected as the first environmental image. Because it is necessary to collect environmental images under underexposure, if the environmental image at this time shows the phenomenon of overall or large-area overexposure, it can be accurately judged that the camera device is blocked by the obstruction, and for the accuracy of the judgment, the image with the smallest average grayscale value is determined as the first environmental image. At this time, if the first environmental image still shows the phenomenon of overall or large-area overexposure, it can be more accurately judged that the camera device is blocked by the obstruction, reducing the possibility of misjudgment.

[0069] Collecting a second environment image at a second sub-luminous intensity by a camera device includes:

[0070] A second frame group image is collected at a second sub-luminous intensity by a camera device, an average grayscale value of all pixels of each frame image in the second frame group image is determined, and an image with the largest average grayscale value is determined as the second environment image.

[0071] In specific implementation, under the second sub-luminous intensity, the camera device will continuously collect multiple frames of images to form an image frame group, namely, the second frame group image, and calculate the average grayscale value of all pixels in each frame of the image. For example, the camera device collects 30 frames of images per second, of which 5-8 frames are the second frame group images obtained by the fill light source under the second sub-luminous intensity (luminous intensity corresponding to overexposure), and the average grayscale value of all pixels in each frame of the image in 5-8 frames is calculated; the image with the largest average grayscale value is selected as the second environmental image. Because it is necessary to collect environmental images under overexposure, if the environmental image at this time appears to be completely black, completely gray, or partially overexposed, it can be accurately determined that the camera device is blocked by an obstruction. In order to ensure the accuracy of the judgment, the image with the largest average grayscale value is determined as the second environmental image. At this time, if the second environmental image still appears to be completely black, completely gray, or partially overexposed, it can be more accurately determined that the camera device is blocked by an obstruction, reducing the possibility of misjudgment.

[0072] In this embodiment, by collecting images under underexposure and overexposure conditions, it is possible to more comprehensively evaluate whether the camera is blocked. Selecting the image with the smallest average grayscale value and the image with the largest average grayscale value as representative images can ensure that the images are the most representative under the respective luminous intensity conditions, which helps to avoid misjudgment caused by improper lighting conditions.

[0073] In some embodiments, the collecting of the environment image at the second light intensity by the camera device includes:

[0074] The frame group images are collected by a camera device at the second luminous intensity, and the last frame image in the frame group images is determined as the environment image.

[0075] In specific implementation, under the second luminous intensity, the camera device will continuously capture multiple frames of images to form a frame group image. Since the camera device and the image processing system need to have enough time to adapt to the light, the last frame of image is selected as the environment image. Figure 8 As shown, for the first frame group image captured at the first sub-luminous intensity, the last frame image of the first frame group image is used as the environmental image, and for the second frame group image captured at the second sub-luminous intensity, the last frame image of the second frame group image is used as the environmental image, which can better reflect the stable response of the camera device to the light environment after the luminous intensity is adjusted. The last frame image provides more accurate visual information at the second luminous intensity, which helps to accurately judge the situation where the camera device is blocked.

[0076] In this embodiment, as time goes by, the camera device and the image processing system will gradually adapt to the current lighting conditions. Selecting the last frame of image as the environment image can better reflect the stable response of the camera device to the light environment after a series of adjustments. This indicates that the last frame of image is obtained after the camera device has fully adapted to the lighting conditions, thus providing more accurate and reliable visual information.

[0077] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0078] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0079] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides an image recognition device.

[0080] refer to Figure 3 , the image recognition device comprises:

[0081] An adjustment module 701, which is configured to adjust the luminous intensity of the fill light source of the camera device from a first luminous intensity to a second luminous intensity, wherein the first luminous intensity is the luminous intensity corresponding to normal exposure, and the second luminous intensity is the luminous intensity corresponding to abnormal exposure;

[0082] The judgment module 702 is configured to capture an environmental image at a second luminous intensity through a camera device, and judge whether the environmental image meets a preset occlusion condition; in response to determining that the environmental image meets the preset occlusion condition, determine that the camera device is blocked.

[0083] Further, the determination module 702 is specifically used for:

[0084] In response to the second luminous intensity being a luminous intensity corresponding to underexposure, determining grayscale values ​​of a plurality of sampling points in the environment image;

[0085] In response to determining that the number of target sampling points is greater than or equal to a preset number, determining that the environment image meets a preset occlusion condition;

[0086] The target sampling point is a sampling point whose grayscale value is greater than or equal to a preset grayscale value.

[0087] Furthermore, the judging module 702 is further configured to:

[0088] A sampling image is determined in a preset detection area of ​​the environmental image by a preset sampling frame, and grayscale values ​​of a plurality of sampling points in the sampling image are determined.

[0089] Furthermore, the judging module 702 is further configured to:

[0090] In response to the second luminous intensity being a luminous intensity corresponding to overexposure, obtaining grayscale values ​​of all pixels in a preset area in the environment image, and determining a grayscale value variance of all pixels;

[0091] In response to determining that the gray value variance is less than or equal to a preset gray value variance, it is determined that the environment image meets a preset occlusion condition.

[0092] Furthermore, the judging module 702 is further configured to:

[0093] Capturing a first environment image at a first sub-luminous intensity and a second environment image at a second sub-luminous intensity by a camera device;

[0094] Accordingly, determining that the environment image meets the preset occlusion condition includes:

[0095] In response to determining that the first environment image does not meet the preset occlusion condition, determining whether the second environment image meets the preset occlusion condition;

[0096] In response to determining that the second environment image meets the preset occlusion condition, it is determined that the environment image meets the preset occlusion condition.

[0097] Furthermore, the judging module 702 is further configured to:

[0098] Collecting a first frame group image at a first sub-luminous intensity by a camera device, determining an average grayscale value of all pixels of each frame image in the first frame group image, and determining an image with a minimum average grayscale value as a first environment image;

[0099] Collecting a second environment image at a second sub-luminous intensity by a camera device includes:

[0100] A second frame group image is collected at a second sub-luminous intensity by a camera device, an average grayscale value of all pixels of each frame image in the second frame group image is determined, and an image with the largest average grayscale value is determined as the second environment image.

[0101] Furthermore, the judging module 702 is further configured to:

[0102] The frame group images are collected by a camera device at the second luminous intensity, and the last frame image in the frame group images is determined as the environment image.

[0103] For the convenience of description, the above device is described in terms of functions divided into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0104] The device of the above embodiment is used to implement the corresponding image recognition method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0105] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the image recognition method described in any of the above embodiments is implemented.

[0106] Figure 4 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0107] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0108] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0109] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0110] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0111] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0112] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0113] The electronic device of the above embodiment is used to implement the corresponding image recognition method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0114] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the image recognition method described in any of the above embodiments.

[0115] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0116] The computer instructions stored in the storage medium of the above embodiments are used to enable the computer to execute the image recognition method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0117] Based on the same concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a computer program product, including computer program instructions. When the computer program instructions are run on a computer, the computer executes the method described in any of the above embodiments, which has the beneficial effects of the corresponding method embodiments and will not be repeated here.

[0118] It is understandable that before using the technical solutions of each embodiment of the present disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0119] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly remind the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can independently choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0120] As an optional but non-limiting implementation, in response to receiving the user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0121] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0122] A person skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0123] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0124] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0125] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the present application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the protection scope of the present application.

Claims

1. An image recognition method, characterized in that: include: Adjusting the luminous intensity of the fill light source of the camera device from a first luminous intensity to a second luminous intensity, wherein the first luminous intensity is the luminous intensity corresponding to normal exposure, and the second luminous intensity is the luminous intensity corresponding to abnormal exposure; Capturing an environment image at a second luminous intensity by means of a camera device, and determining whether the environment image meets a preset shielding condition; In response to determining that the environment image meets a preset blocking condition, it is determined that the camera device is blocked.

2. The image recognition method according to claim 1, characterized in that: The abnormal exposure includes underexposure and overexposure; and the determining whether the environment image meets the preset shielding condition includes: In response to the second luminous intensity being a luminous intensity corresponding to underexposure, determining grayscale values ​​of a plurality of sampling points in the environment image; In response to determining that the number of target sampling points is greater than or equal to a preset number, determining that the environment image meets a preset occlusion condition, The target sampling point is a sampling point whose grayscale value is greater than or equal to a preset grayscale value.

3. The image recognition method according to claim 2, characterized in that: The step of determining the grayscale values ​​of a plurality of sampling points in the environment image comprises: A sampling image is determined in a preset detection area of ​​the environmental image by a preset sampling frame, and grayscale values ​​of a plurality of sampling points in the sampling image are determined.

4. The image recognition method according to claim 1, characterized in that: The abnormal exposure includes underexposure and overexposure; and the determining whether the environment image meets the preset shielding condition includes: In response to the second luminous intensity being a luminous intensity corresponding to overexposure, obtaining grayscale values ​​of all pixels in a preset area in the environment image, and determining a grayscale value variance of all pixels; In response to determining that the gray value variance is less than or equal to a preset gray value variance, it is determined that the environment image meets a preset occlusion condition.

5. The image recognition method according to claim 1, characterized in that: The abnormal exposure includes underexposure and overexposure, the second luminous intensity includes a first sub-luminous intensity and a second sub-luminous intensity, and the first sub-luminous intensity is a luminous intensity corresponding to underexposure, and the second sub-luminous intensity is a luminous intensity corresponding to overexposure, and the environmental image includes a first environmental image and a second environmental image, The collecting of the environment image at the second luminous intensity by the camera device comprises: A camera device is used to capture a first environment image at a first sub-luminous intensity and a second environment image at a second sub-luminous intensity. Determining that the environment image meets the preset occlusion condition includes: In response to determining that the first environment image does not meet the preset occlusion condition, determining whether the second environment image meets the preset occlusion condition; In response to determining that the second environment image meets the preset occlusion condition, it is determined that the environment image meets the preset occlusion condition.

6. The image recognition method according to claim 5, characterized in that: The collecting of the first environment image at the first sub-luminous intensity by a camera device includes: The first frame group image is collected by a camera device at a first sub-luminous intensity, the average grayscale value of all pixels of each frame image in the first frame group image is determined, and the image with the smallest average grayscale value is determined as the first environment image, Collecting a second environment image at a second sub-luminous intensity by a camera device includes: A second frame group image is collected at a second sub-luminous intensity by a camera device, an average grayscale value of all pixels of each frame image in the second frame group image is determined, and an image with the largest average grayscale value is determined as the second environment image.

7. The image recognition method according to claim 1, characterized in that: The collecting of the environment image at the second luminous intensity by the camera device includes: The frame group images are collected by a camera device at the second luminous intensity, and the last frame image in the frame group images is determined as the environment image.

8. An image recognition device, characterized in that: include: an adjusting module, configured to adjust the luminous intensity of the fill light source of the camera device from a first luminous intensity to a second luminous intensity, wherein the first luminous intensity is the luminous intensity corresponding to normal exposure, and the second luminous intensity is the luminous intensity corresponding to abnormal exposure; A judgment module, configured to capture an environmental image at a second luminous intensity through a camera device, and judge whether the environmental image meets a preset shielding condition; In response to determining that the environment image meets a preset blocking condition, it is determined that the camera device is blocked.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

10. A vehicle, characterized in that: The vehicle comprises the electronic device as claimed in claim 9.

Images