Detection methods, devices and storage media

Abstract

This application provides a detection method, apparatus, and storage medium applicable to intelligent driving or autonomous driving fields. The method includes: acquiring single-frame or multi-frame environmental images from at least one image acquisition device, such as an in-vehicle camera; comprehensively analyzing the brightness variation patterns or image quality parameters of at least one target object in the single-frame or multi-frame environmental images to determine the state information of a first terminal. The state information of the first terminal includes whether the glass of the first terminal is fogged, or the weather conditions of the environment in which the first terminal is located. This detection process distinguishes between fogged glass and foggy weather conditions, improving the accuracy of environmental detection.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a detection method, apparatus and storage medium. Background Technology

[0002] With the rapid development of artificial intelligence, assisted driving and autonomous driving require the perception of the surrounding driving environment. To accurately perceive the driving environment, it's necessary to know various information such as pedestrians, vehicles, and lane markings along the driving path to ensure driving within a defined path and avoid collisions with other vehicles and pedestrians. The requirements for sensor perception vary depending on the scenario, road conditions, and function. As one of the most important sensors, cameras play a crucial role, performing tasks such as obstacle detection, lane marking detection, and road boundary detection.

[0003] A common phenomenon during vehicle operation is windshield fogging, especially in winter. Due to the large temperature difference between the interior and exterior, the windshield fogs up on the inside, similar to a foggy day outside. Cameras capture environmental images of both scenarios, but these images are unclear. Current technology cannot distinguish between these two scenarios, resulting in low accuracy in vehicle environmental detection. Summary of the Invention

[0004] This application provides a detection method, apparatus, and storage medium to distinguish between glass fogging and foggy weather conditions, thereby improving the accuracy of environmental monitoring.

[0005] In a first aspect, embodiments of this application provide a detection method, the method comprising: acquiring at least one frame of environmental image from at least one image acquisition device, the environmental image being used to present information about the environment in which a first terminal is located; determining state information of the first terminal based on the at least one frame of environmental image, the state information including at least one of the following: whether the glass of the first terminal is fogged up, or the weather conditions of the environment in which the first terminal is located.

[0006] In the above scheme, by acquiring one or more frames of environmental images captured by at least one image acquisition device, it is determined whether the first terminal has fogged glass or the weather conditions (including foggy weather) of the environment in which the first terminal is located, based on the single or multiple frames of environmental images. Through the above process, the distinction between fogged glass and foggy weather conditions is achieved, improving the accuracy of environmental detection.

[0007] Optionally, the weather condition includes dense fog, light fog, or normal. Specifically, it can be determined whether it is dense fog, light fog, or normal by describing the fog's envelope, particle size, density, visibility, etc. As an example, the lower the visibility value, the worse the visibility and the higher the fog concentration.

[0008] In one possible implementation, the state information includes whether the glass of the first terminal is fogged up, and the environment image presents an environment containing at least two target objects and the sky. The at least two target objects in the environment image can be understood as at least two detection points in the environment image, with each detection point corresponding to one or more pixels in the image.

[0009] Optionally, the state information of the first terminal is determined based on at least one frame of environmental image, including: determining the state information based on the brightness information of at least two target objects, the brightness information of the sky, and the depth information of at least two target objects in the first environmental image of at least one frame of environmental image.

[0010] Optionally, state information is determined based on the brightness information of at least two target objects in the first environmental image, the brightness information of the sky, and the depth information of at least two target objects. This includes determining that the glass of the first terminal is fogged up. In the first environmental image, there is at least one group of target objects among the at least two target objects. The difference in the extinction coefficients of any two target objects in each group is greater than a first threshold. The extinction coefficients are determined by the brightness information of the target objects, the brightness information of the sky, and the depth information of the target objects. The extinction coefficients are used to indicate the degree of brightness loss of the target objects in the atmosphere.

[0011] In this implementation, the state information of the first terminal is determined by comparing the extinction coefficients of at least two target objects in a single frame of an environmental image from an image acquisition device. Taking two target objects as an example, if the difference in the extinction coefficients of these two target objects is greater than a first threshold (i.e., the difference in extinction coefficients between two detection points in the image is significant), then the glass of the first terminal is determined to be fogged. The above process can effectively distinguish between glass fogging and foggy weather conditions, avoiding the misclassification of a foggy glass scene as a foggy weather scene.

[0012] In one possible implementation, the state information includes whether the glass of the first terminal is fogged up, and the environment presented in the environmental image includes at least one near-end target object. The near-end target object includes an object outside the first terminal that is less than a preset distance from the first terminal. For example, the near-end target object can be the front or rear hood of a vehicle, any object fixed to the hood, or the rearview mirrors on the left and right sides of the vehicle.

[0013] Optionally, determining the state information of the first terminal based on at least one frame of environmental image includes: determining the state information based on the sharpness value of at least one near-end target object in the first environmental image of at least one frame of environmental image; wherein the sharpness value of at least one near-end target object is determined by the grayscale value of the image block corresponding to at least one near-end target object.

[0014] Optionally, the status information is determined based on the sharpness value of at least one near-end target object in the first environmental image, including: determining that the glass of the first terminal is fogged up, wherein in the first environmental image, there is at least one near-end target object whose sharpness value is less than or equal to a preset sharpness threshold.

[0015] In this implementation, the state information of the first terminal is determined by analyzing the sharpness value of at least one near-end target object in a single frame of environmental image from an image acquisition device. Taking a near-end target object as an example, if the sharpness value corresponding to the near-end target object is less than or equal to a preset sharpness threshold (i.e., the near-end target object in the image is blurry), then the glass of the first terminal is determined to be fogged. It should be understood that glass fogging will cause near-end objects to be blurry, while foggy weather has little impact on the sharpness of near-end objects. The above process can effectively distinguish between glass fogging and foggy weather, avoiding the misclassification of a glass fogging scene as a foggy weather scene.

[0016] In one possible implementation, acquiring at least one frame of environmental image from at least one image acquisition device includes: acquiring multiple frames of environmental image from at least one image acquisition device. Determining the state information of the first terminal based on the environmental image includes: determining the state information of the first terminal based on the multiple frames of environmental image.

[0017] Optionally, the status information includes whether the glass of the first terminal is fogged up, and the environment image presented includes at least one target object.

[0018] Optionally, determining the state information of the first terminal based on multiple frames of environmental images includes: determining the state information based on the extinction coefficient or sharpness value of at least one target object in the multiple frames of environmental images.

[0019] The extinction coefficient of at least one target object in each frame of the environmental image is determined by the brightness information of at least one target object, the brightness information of the sky, and the depth information of at least one target object; the sharpness value of at least one target object in each frame of the environmental image is determined by the grayscale value of the image block corresponding to at least one target object.

[0020] In the first implementation, state information is determined based on the extinction coefficient of at least one target object in multiple frames of environmental images, including: determining that the glass of the first terminal is fogged, wherein the difference in the extinction coefficient of the same target object in any two frames of environmental images is less than or equal to a fourth threshold.

[0021] In this case, the fogging of the glass on the first terminal is determined by comparing the extinction coefficients of the same target object in multiple frames of environmental images from an image acquisition device. If the difference in the extinction coefficients of the same target object in any two frames of the multiple environmental images is less than or equal to the fourth threshold, it indicates that the extinction coefficients of the same target object remain basically unchanged in the multiple environmental images, and this feature corresponds to the fogging scene.

[0022] It should be understood that if the first terminal is in a foggy environment, as the first terminal moves, the distance between the same target object and the first terminal continuously changes (increasing or decreasing). The extinction coefficient is related to the distance value; the farther the target object is from the first terminal, the greater its extinction coefficient, and the closer it is, the smaller the extinction coefficient. Therefore, in a foggy environment, the extinction coefficient of the same target object varies significantly across multiple frames of environmental images. However, for scenes with fogged glass, the movement of the first terminal has little impact on the extinction coefficient of the same target object across multiple frames of environmental images.

[0023] In the second implementation, state information is determined based on the sharpness value of at least one target object in multiple frames of environmental images, including: determining that the glass of the first terminal is fogged up, wherein the difference in sharpness value of the same target object in any two frames of environmental images is less than or equal to a fifth threshold.

[0024] In this case, the fogging of the glass on the first terminal is determined by comparing the sharpness values ​​of the same target object in multiple frames of environmental images from an image acquisition device. If the difference in sharpness values ​​of the same target object in any two frames of the multiple environmental images is less than or equal to the fifth threshold, it indicates that the sharpness values ​​of the same target object remain basically unchanged in the multiple environmental images, and this feature corresponds to the fogging scene.

[0025] It should be understood that if the first terminal is in a foggy environment, the distance between the same target object and the first terminal changes continuously as the first terminal moves. The sharpness value is related to the distance; the farther the target object is from the first terminal, the lower its sharpness value, and the closer it is, the higher its sharpness value. Therefore, the sharpness value of the same target object varies significantly across multiple frames of environmental images in foggy conditions. However, for scenes with fogged glass, the movement of the first terminal has little impact on the extinction coefficient of the same target object across multiple frames of environmental images.

[0026] Optionally, the same target object is the same near-end target object, which includes objects outside the first terminal that are less than a preset distance from the first terminal.

[0027] Based on the above implementation methods, optionally, when it is determined that the glass of the first terminal is fogged up, the method may further include: controlling the defogging device in the vehicle to start, or controlling the window lifting device to start, or issuing an alarm message.

[0028] Optionally, when it is determined that the glass of the first terminal is not fogged, the method further includes: acquiring the saturation and brightness of any frame of environmental image from at least one image acquisition device; and determining the weather conditions of the environment in which the first terminal is located based on the ratio of brightness to saturation. This scheme is used to further determine whether the current environmental weather of the first terminal is normal weather or foggy weather (dense fog or light fog).

[0029] Optionally, the weather state of the environment where the first terminal is located is determined based on the ratio of brightness to saturation, including: when the weather state is dense fog, the ratio is greater than or equal to a second threshold; when the weather state is light fog, the ratio is greater than a third threshold and less than the second threshold; and / or, when the weather state is normal, the ratio is less than or equal to the third threshold.

[0030] Optionally, when the weather condition is determined to be dense fog, the driving status of the first terminal is controlled or control information is output to the vehicle controller; or, when the weather condition is determined to be light fog, the environmental image is defogging; or, when the weather condition is determined to be normal, road detection is performed based on the environmental image.

[0031] The above scheme calculates the saturation and brightness of the environmental image, and determines the fog concentration level based on the ratio of brightness to saturation. This enables the first terminal to detect foggy environments and execute corresponding control operations according to the fog concentration level. When the fog concentration is low, there is no need to switch the driving mode of the first terminal; the environmental image can be defogged using image processing algorithms, thus avoiding waste of control system resources.

[0032] Secondly, embodiments of this application provide a detection device, including: an acquisition module and a processing module. The acquisition module is used to acquire at least one frame of an environmental image from at least one image acquisition device, the environmental image being used to present information about the environment in which a first terminal is located; the processing module is used to determine state information of the first terminal based on the at least one frame of the environmental image, the state information including at least one of the following: whether the glass of the first terminal is fogged, or the weather conditions of the environment in which the first terminal is located.

[0033] Optionally, the weather condition includes dense fog, light fog, or normal.

[0034] Optionally, the status information includes whether the glass of the first terminal is fogged up, and the environment image presents an environment that includes at least two target objects and the sky.

[0035] In one possible implementation, the processing module is specifically used to determine state information based on the brightness information of at least two target objects in a first environment image in at least one frame of environment image, the brightness information of the sky, and the depth information of at least two target objects.

[0036] Optionally, the processing module determines that the glass of the first terminal is fogged up. In the first environmental image, there is at least one group of target objects among at least two target objects. The difference in the extinction coefficients of any two target objects in each group of target objects is greater than a first threshold. The extinction coefficient is determined by the brightness information of the target object, the brightness information of the sky, and the depth information of the target object. The extinction coefficient is used to indicate the degree of brightness loss of the target object in the atmosphere.

[0037] Optionally, the status information includes whether the glass of the first terminal is fogged up, and the environment presented in the environmental image includes at least one near-end target object, which includes an object outside the first terminal that is less than a preset distance from the first terminal.

[0038] In one possible implementation, the processing module is specifically used to determine state information based on the sharpness value of at least one near-end target object in a first environment image within at least one frame of environment images. The sharpness value of the at least one near-end target object is determined by the grayscale value of the image patch corresponding to the at least one near-end target object.

[0039] Optionally, the processing module determines that the glass of the first terminal is fogged up, and in the first environmental image, at least one near-end target object has a sharpness value that is less than or equal to a preset sharpness threshold.

[0040] Optionally, the acquisition module is specifically used to acquire multiple frames of environmental images from at least one image acquisition device. The processing module is specifically used to determine the status information of the first terminal based on the multiple frames of environmental images.

[0041] Optionally, the status information includes whether the glass of the first terminal is fogged up, and the environment image presented includes at least one target object.

[0042] In one possible implementation, the processing module is specifically used to determine state information based on the extinction coefficient or sharpness value of at least one target object in multiple frames of environmental images. Specifically, the extinction coefficient of at least one target object in each frame of the environmental image is determined by the brightness information of at least one target object, the brightness information of the sky, and the depth information of at least one target object; the sharpness value of at least one target object in each frame of the environmental image is determined by the grayscale value of the image patch corresponding to at least one target object.

[0043] Optionally, the processing module determines that the glass of the first terminal is fogged, wherein the difference in the extinction coefficient of the same target object in any two environmental images in the multi-frame environmental images is less than or equal to a fourth threshold, or the difference in the sharpness value of the same target object in any two environmental images in the multi-frame environmental images is less than or equal to a fifth threshold.

[0044] Optionally, the same target object is the same near-end target object, which includes objects outside the first terminal that are less than a preset distance from the first terminal.

[0045] Based on the above implementation methods, optionally, when the processing module determines that the glass of the first terminal is fogged up, it is also used to: control the defogging device in the vehicle to start, or control the window lifting device to start, or issue an alarm message.

[0046] Optionally, when the processing module determines that the glass of the first terminal is not fogged, the acquisition module is further configured to acquire the saturation and brightness of any frame of environmental image from at least one image acquisition device. The processing module is further configured to determine the weather conditions of the environment in which the first terminal is located based on the ratio of brightness to saturation.

[0047] Optionally, when the weather condition is dense fog, the ratio is greater than or equal to the second threshold; when the weather condition is light fog, the ratio is greater than the third threshold and less than the second threshold; and / or, when the weather condition is normal, the ratio is less than or equal to the third threshold.

[0048] Optionally, the processing module is also used to: control the driving state of the first terminal or output control information to the vehicle controller when the weather condition is determined to be dense fog; or, perform defogging processing on the environmental image when the weather condition is determined to be light fog; or, perform road detection based on the environmental image when the weather condition is determined to be normal.

[0049] Thirdly, embodiments of this application provide a detection device, including at least one processor and at least one memory; the at least one memory is used to store computer execution instructions, and when the detection device is running, the at least one processor executes the computer execution instructions stored in the at least one memory to cause the detection device to perform the detection method as described in any of the first aspects.

[0050] Fourthly, embodiments of this application provide a computer storage medium for storing a computer program, which, when executed on a computer, causes the computer to perform a detection method as described in any of the first aspects.

[0051] This application provides a detection method, apparatus, and storage medium. The method includes: acquiring single-frame or multi-frame environmental images from at least one image acquisition device; comprehensively analyzing the brightness variation pattern or image quality parameters of at least one target object in the single-frame or multi-frame environmental images; and determining the state information of a first terminal. The state information of the first terminal includes whether the glass of the first terminal is fogged, or the weather conditions of the environment in which the first terminal is located. The above detection process distinguishes between fogged glass and foggy weather conditions, improving the accuracy of environmental detection. Attached Figure Description

[0052] Figure 1 Functional block diagram of a vehicle provided in the embodiments of this application;

[0053] Figure 2 A flowchart of a detection method provided in an embodiment of this application;

[0054] Figure 3 This is a spatial diagram of the image acquisition device provided in the embodiments of this application within a vehicle.

[0055] Figure 4 A flowchart for determining the state information of a first terminal is provided in an embodiment of this application;

[0056] Figure 5 A flowchart for determining the state information of a first terminal is provided in an embodiment of this application;

[0057] Figure 6 A flowchart for determining the state information of a first terminal is provided in an embodiment of this application;

[0058] Figure 7 A flowchart for determining the weather status of a first terminal is provided in an embodiment of this application;

[0059] Figure 8 This is a schematic diagram of the structure of a detection device provided in an embodiment of this application;

[0060] Figure 9 This is a schematic diagram of the hardware structure of a detection device provided in an embodiment of this application. Detailed Implementation

[0061] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0062] The detection method provided in this application embodiment can be applied to any terminal with autonomous driving function. The terminal has a closed space, and the components forming the closed space include at least glass. The terminal can be a vehicle, ship, airplane, spacecraft, etc. This application embodiment does not impose any restrictions on this.

[0063] For ease of description, the following embodiments use a vehicle as an example.

[0064] As an example, the detection method provided in this application embodiment can be applied to vehicles with autonomous driving functions or to other devices (such as cloud servers) that control autonomous driving functions. The vehicle can implement the detection method provided in this application embodiment through its components (including hardware and / or software) to determine the vehicle's current state information (such as speed, position, road conditions, weather conditions, etc.) and generate control commands to control the vehicle. Alternatively, other devices (such as servers) can implement the detection method in this application embodiment to determine the vehicle's current state information, generate control commands to control the vehicle, and send the control commands to the vehicle.

[0065] Figure 1 This is a functional block diagram of a vehicle 100 provided in an embodiment of this application. In some embodiments, the vehicle 100 can be configured in a fully or partially autonomous driving mode. For example, the vehicle 100 can control itself while in autonomous driving mode, and can determine the state information of the vehicle and its surrounding environment through human operation, such as whether the vehicle's windows are fogged up, or the weather conditions of the environment where the vehicle is located, and control the vehicle 100 based on the determined state information. When the vehicle 100 is in autonomous driving mode, the vehicle 100 can be set to operate without human interaction.

[0066] Vehicle 100 may include various subsystems, such as a mobility system 102, a sensor system 104, a control system 106, one or more peripheral devices 108, and at least one of a power supply 110, a computer system 112, and a user interface 116. Optionally, vehicle 100 may include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and component of vehicle 100 may be interconnected via wired or wireless means.

[0067] The propulsion system 102 may include components that provide powered motion to the vehicle 100. In some embodiments, the propulsion system 102 may include an engine 118, an energy source 119, a transmission 120, and wheels / tires 121. The engine 118 may be an internal combustion engine, an electric motor, an air-compressed engine, or other types of engine combinations, such as a hybrid engine consisting of a gasoline engine and an electric motor, or a hybrid engine consisting of an internal combustion engine and an air-compressed engine. The engine 118 converts the energy source 119 into mechanical energy. Examples of the energy source 119 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. The energy source 119 may also provide energy to other systems of the vehicle 100. The transmission 120 transmits mechanical power from the engine 118 to the wheels 121. The transmission 120 may include a gearbox, a differential, and a drive shaft. In some embodiments, the transmission 120 may also include other components, such as a clutch. The drive shaft may include one or more axles that can be coupled to one or more wheels 121.

[0068] Sensor system 104 may include several sensors for sensing information about the environment surrounding vehicle 100. For example, sensor system 104 may include at least one of a positioning system 122 (which may be a GPS system, a BeiDou system, or another positioning system), an inertial measurement unit (IMU) 124, a radar 126, a laser rangefinder 128, and a camera 130. Sensor system 104 may also include sensors for the internal systems of the monitored vehicle 100 (e.g., an in-vehicle air quality monitor, fuel gauge, oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, orientation, speed, etc.). This detection and identification is a key function for the safe operation of the autonomous vehicle 100.

[0069] The positioning system 122 described above can be used to estimate the geographical location of vehicle 100. IMU 124 is used to sense changes in the position and orientation of vehicle 100 based on inertial acceleration. In some embodiments, IMU 124 can be a combination of an accelerometer and a gyroscope. Radar 126 can use radio signals to sense objects in the surrounding environment of vehicle 100, including millimeter-wave radar, lidar, etc. In some embodiments, in addition to sensing objects, radar 126 can also be used to sense the speed and / or direction of travel of objects. Laser rangefinder 128 can use lasers to sense the distance between vehicle 100 and objects in the surrounding environment. In some embodiments, laser rangefinder 128 may include one or more laser sources, a laser scanner, and one or more detectors, as well as other system components. Camera 130 can be used to capture multiple images of the surrounding environment of vehicle 100. Camera 130 can be a still camera or a video camera.

[0070] The control system 106 controls the operation of the vehicle 100 and its components. The control system 106 may include various elements such as a steering system 132, a throttle 134, a braking unit 136, a computer vision system 140, a route control system 142, and an obstacle avoidance system 144. The steering system 132 is operable to adjust the forward direction of the vehicle 100, such as a steering wheel system. The throttle 134 controls the operating speed of the engine 118 and thus the speed of the vehicle 100. The braking unit 136 controls the deceleration of the vehicle 100. The braking unit 136 may use friction to slow down the wheels 121. In other embodiments, the braking unit 136 may convert the kinetic energy of the wheels 121 into electrical current. The braking unit 136 may also take other forms to slow down the rotational speed of the wheels 121 to control the speed of the vehicle 100. The computer vision system 140 is operable to process and analyze images captured by the camera 130 to identify objects and / or features in the environment surrounding the vehicle 100. These objects and / or features may include traffic signals, road boundaries, and obstacles. Computer vision system 140 may use object recognition algorithms, structure from motion (SFM) algorithms, video tracking, and other computer vision techniques. In some embodiments, computer vision system 140 may be used to map the environment, track objects, estimate object velocities, etc. Route control system 142 is used to determine the driving route of vehicle 100. In some embodiments, route control system 142 may combine data from sensors, positioning system 122, and one or more predetermined maps to determine the driving route for vehicle 100. Obstacle avoidance system 144 is used to identify, evaluate, and avoid or otherwise traverse potential obstacles in the environment of vehicle 100. Of course, in some embodiments, control system 106 may add or alternatively include components other than those shown and described. Alternatively, some of the components shown above may be reduced.

[0071] Vehicle 100 can interact with external sensors, other vehicles, other computer systems, or users via peripheral devices 108. Peripheral devices 108 may include a wireless communication system 146, an onboard computer 148, a microphone 150, and / or a speaker 152. The wireless communication system 146 can communicate wirelessly with one or more devices directly or via a communication network. For example, the wireless communication system 146 can use 3G cellular communication, such as CDMA, EVDO, GSM / GPRS, or 4G cellular communication, such as LTE, or 5G cellular communication. The wireless communication system 146 can communicate using WiFi and a wireless local area network (WLAN). In some embodiments, the wireless communication system 146 can communicate directly with devices using an infrared link, Bluetooth, or ZigBee. Other wireless protocols, such as various vehicle communication systems, may also be used. For example, the wireless communication system 146 may include one or more dedicated short-range communications (DSRC) devices.

[0072] In some embodiments, peripheral device 108 provides a means for a user of vehicle 100 to interact with user interface 116. For example, on-board computer 148 may provide information to a user of vehicle 100. User interface 116 may also operate on-board computer 148 to receive user input. On-board computer 148 may be operated via a touchscreen. In other cases, peripheral device 108 may provide a means for vehicle 100 to communicate with other devices located within the vehicle. For example, microphone 150 may receive audio (e.g., voice commands or other audio input) from a user of vehicle 100. Similarly, speaker 152 may output audio to a user of vehicle 100. Power source 110 may provide power to various components of vehicle 100. In one embodiment, power source 110 may be a rechargeable lithium-ion or lead-acid battery. One or more such battery packs may be configured as a power source to provide power to various components of vehicle 100. In some embodiments, power source 110 and energy source 119 may be implemented together, as is the case in some all-electric vehicles.

[0073] Some or all of the functions of vehicle 100 are controlled by computer system 112. Computer system 112 may include at least one processor 113, which executes instructions 115 stored in a non-transitory computer-readable medium such as data storage device 114. Computer system 112 may also be multiple computing devices that control individual components or subsystems of vehicle 100 in a distributed manner.

[0074] Processor 113 can be any conventional processor, such as a commercially available Central Processing Unit (CPU). Alternatively, the processor can be a dedicated device such as an Application-Specific Integrated Circuit (ASIC) or other hardware-based processor. Although Figure 1 The processor, memory, and other components within the same physical housing are functionally illustrated; however, those skilled in the art will understand that the processor, computer system, or memory may actually include multiple processors, computer systems, or memories that may be stored within the same physical housing, or multiple processors, computer systems, or memories that may not be stored within the same physical housing. For example, memory may be a hard disk drive, or other storage media located in a different physical housing. Therefore, references to processors or computer systems will be understood to include references to collections of processors or computer systems or memories that may operate in parallel, or collections of processors or computer systems or memories that may not operate in parallel. Unlike using a single processor to perform the steps described herein, some components, such as steering and deceleration components, may each have their own processor that performs calculations only related to the component's specific function.

[0075] In the various aspects described herein, the processor may be located remotely from the vehicle and communicate wirelessly with the vehicle. In other aspects, some of the processes described herein are executed on a processor located within the vehicle, while others are executed by a remote processor, including taking the necessary steps to perform a single operation.

[0076] In some embodiments, the data storage device 114 may include instructions 115 (e.g., program logic) that can be executed by the processor 113 to perform various functions of the vehicle 100, including those described above. The data storage device 114 may also include additional instructions, including instructions to send data to, receive data from, interact with, and / or control one or more of the mobility system 102, sensor system 104, control system 106, and peripheral devices 108. In addition to instructions 115, the data storage device 114 may also store data such as road maps, route information, vehicle position, direction, speed, and other such vehicle data, as well as other information (e.g., weather conditions). This information can be used by the vehicle 100 and the computer system 112 during autonomous, semi-autonomous, and / or manual operation of the vehicle 100. As an example, the data storage device 114 can acquire environmental information from the sensor system 104 or other components of the vehicle 100. This environmental information may include, for example, whether there are green belts, traffic lights, pedestrians, etc., in the vicinity of the vehicle's current environment. The vehicle can calculate the presence of such elements using machine learning algorithms. The data storage device 114 can also store the vehicle's own state information, as well as the state information of other vehicles interacting with it. State information includes, but is not limited to, the vehicle's speed, acceleration, and heading angle. For example, the vehicle can obtain the distance between itself and other vehicles, and the speed of other vehicles, based on the speed and distance measurement functions of the radar 126. Thus, the processor 113 can acquire the aforementioned environmental or state information from the data storage device 114, and based on the environmental information of the vehicle's current environment, the vehicle's own state information, the state information of other vehicles, and traditional rule-based driving strategies, derive a final driving strategy to control the vehicle for autonomous driving (e.g., acceleration, deceleration, stopping).

[0077] User interface 116 is used to provide information to or receive information from a user of vehicle 100. Optionally, user interface 116 may include one or more input / output devices within a set of peripheral devices 108, such as one or more of wireless communication system 146, vehicle computer 148, microphone 150, and speaker 152.

[0078] Computer system 112 can control the functions of vehicle 100 based on input received from various subsystems (e.g., driving system 102, sensor system 104, and control system 106) and from user interface 116. For example, computer system 112 can utilize input from control system 106 to control steering system 132 to avoid obstacles detected by sensor system 104 and obstacle avoidance system 144. In some embodiments, computer system 112 is operable to provide control over many aspects of vehicle 100 and its subsystems.

[0079] Optionally, one or more of these components may be installed separately from or associated with vehicle 100. For example, data storage device 114 may exist partially or completely separate from vehicle 100. The components may be communicatively coupled together via wired and / or wireless means. It should be noted that the above components are merely examples; in practical applications, components in each of the above modules may be added or removed as needed. Figure 1 This should not be construed as a limitation on the embodiments of this application.

[0080] In some embodiments of this application, the vehicle may further include hardware structures and / or software modules to implement the above functions in the form of hardware structures, software modules, or a combination of hardware structures and software modules. Whether a particular function is implemented in the form of hardware structures, software modules, or a combination of hardware structures and software modules depends on the specific application and design constraints of the technical solution.

[0081] The aforementioned vehicle 100 can be a sedan, SUV, sports car, truck, bus, recreational vehicle, amusement park vehicle, construction equipment, tram, golf cart, train, etc., and this application embodiment does not impose any restrictions on this.

[0082] The vehicle 100 in this embodiment has intelligent driving or autonomous driving functions. The autonomous driving level of the vehicle is used to indicate the degree of intelligence and automation of the autonomous vehicle. Currently, according to the SAE standard, the autonomous driving level of vehicles is divided into 6 levels: no automation (L0), driver assistance (L1), partial automation (L2), conditional automation (L3), high automation (L4), and full automation (L5).

[0083] Based on the functional description of the aforementioned vehicles, autonomous / assisted driving vehicles can acquire environmental images through sensor systems and determine the current weather conditions based on image recognition algorithms. If the current weather conditions are unfavorable, such as rain, snow, or fog, the vehicle can be controlled accordingly, for example, by lowering the autonomous driving level or issuing warning messages. However, there is a special case: during winter driving, due to the large temperature difference between the interior and exterior, the windshield on the inside of the vehicle is prone to fogging. This scenario is similar to a foggy weather condition. To ensure driving safety, the vehicle will perform corresponding control operations as if the weather conditions were unfavorable, such as lowering the autonomous driving level. Those skilled in the art will understand that if the windshield is fogged, it can be cleared by activating a defogging device, such as turning on the air conditioning, without lowering the autonomous driving level. Therefore, failing to distinguish between foggy weather and windshield fogging scenarios could lead to a waste of system resources for autonomous vehicles.

[0084] To address the aforementioned technical problems, this application provides a detection method for an autonomous vehicle 100 or a computing device associated with the autonomous vehicle 100 (such as...). Figure 1 The computer system 112, computer vision system 140, and data storage device 114 can determine the current state information of the vehicle and execute corresponding control operations based on the image features of the acquired environmental images. The vehicle 100 can determine the weather condition (e.g., sunny, cloudy, rainy, foggy, snowy) of its current environment based on the image features of target objects (e.g., objects fixed outside the vehicle, other vehicles on the road, lane lines, traffic lights, sky, etc.) in the environmental images. Alternatively, the vehicle 100 can determine whether its windshield is fogged based on the image features of the aforementioned target objects in the environmental images. After determining the current state information, the vehicle 100 performs corresponding control operations based on the determined state information; different state information can correspond to different control operations. As an example, when the vehicle 100 determines that the current environment is foggy, it can perform defogging processing on the environmental image, or reduce the autonomous driving level, or reduce the vehicle speed, etc. As another example, when vehicle 100 determines that the windshield is fogged up, it controls other devices on the vehicle to start or stop, such as controlling the defogging device or window regulator to start.

[0085] The detection method of this application will be described in detail below using specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar content will not be described repeatedly in different embodiments.

[0086] Figure 2This is a flowchart illustrating a detection method provided in an embodiment of this application. Figure 2 As shown, the detection method provided in this embodiment includes the following steps:

[0087] Step 201: Acquire at least one frame of environmental image from at least one image acquisition device. The environmental image is used to present information about the environment in which the first terminal is located.

[0088] The first terminal in this embodiment can be a vehicle, ship, aircraft, spacecraft, etc., with autonomous driving capabilities, and is currently in a fully or partially autonomous driving mode. When in fully autonomous driving mode, the first terminal can achieve automatic control based on environmental images. Alternatively, when in partially autonomous driving mode, the first terminal can make a preliminary judgment based on environmental images and then obtain user control commands through human-computer interaction to achieve semi-automatic control.

[0089] It should be noted that the first terminal in this embodiment has an enclosed space, and the components of the enclosed space include at least glass, through which the user observes the surrounding environment of the first terminal. For example, the user can observe the road conditions, weather conditions, etc. around the vehicle through the vehicle's windshield, left and right side windshields, or rear windshield.

[0090] To achieve intelligent control functions in the first terminal, at least one image acquisition device can be installed in the enclosed space inside the first terminal. The image acquisition device can include a monocular camera, a stereo camera, a depth camera (RGB-D), and various combinations thereof. Monocular cameras are simple in structure and low in cost, but a single image cannot determine depth information. Stereo cameras consist of two monocular cameras; the distance between these two cameras is known, and the distance of an object from either camera can be determined based on the relative positions of the two cameras. Depth cameras can measure the distance from an object to the camera using infrared structured light or time-of-flight (ToF) ranging, by actively emitting light towards the object and receiving the reflected light. Optionally, the image acquisition device may also include a camera.

[0091] The environmental images acquired by the image acquisition device in this embodiment are visible light images, which are used to present information about the environment in which the first terminal is located, such as road conditions (obstacles such as other vehicles and pedestrians, lane lines, stop lines, lane edge lines, pedestrian crossings, traffic lights, traffic signs, green belts, etc.) and weather conditions (sunny, cloudy, rainy, snowy, foggy, etc.).

[0092] Taking the vehicle as the first terminal as an example. Figure 3 A schematic diagram of the image acquisition device inside the vehicle is shown. Figure 3As shown, the image acquisition device can be set in the front driving area 301 of the vehicle 100, such as the top of the driver's seat or the top of the passenger seat, to acquire environmental images in front of the vehicle 100; it can also be set in the rear passenger area 302 of the vehicle 100, such as the top of the left or right side of the rear seat, to acquire environmental images on the left or right side of the vehicle 100, or the top of the middle rear of the rear seat, to acquire environmental images behind the vehicle 100.

[0093] Step 202: Determine the status information of the first terminal based on at least one frame of environmental image.

[0094] The status information includes at least one of the following:

[0095] Is the glass of the first terminal fogged up? Or, what is the weather condition of the environment in which the first terminal is located?

[0096] The weather conditions of the environment where the first terminal is located include dense fog, light fog, or normal conditions.

[0097] Fog in the environment affects camera imaging and detection. It can be described by factors such as the fog's envelope, particle size, density, and visibility to determine whether it is dense fog, light fog, or normal fog. The envelope is a spatial volume concept without a fixed standard; fog particle size is typically 1µm to 15µm; density includes two indicators: water content and number density, with values ​​ranging from 0.1 g / m³. 3 ~0.5g / m 3 and 50 / cm 3 ~2500 pieces / cm 3 The range of visibility depends on the microscopic physical quantities of the fog, including the number density of fog droplets, particle size, and water content.

[0098] It should be noted that visibility is mainly determined by two factors: first, the difference in brightness between the target object and the background (e.g., the sky); the greater (smaller) the difference, the greater (smaller) the visibility. Second, atmospheric transparency; the air layer between the observer (or image acquisition device) and the target object can reduce the aforementioned difference in brightness; the worse (better) the atmospheric transparency, the smaller (greater) the visibility. Fog, smoke, dust storms, heavy snow, drizzle, and other weather phenomena can make the atmosphere turbid and reduce transparency.

[0099] Table 1 shows a qualitative description of the relationship between visibility and weather. As can be seen from Table 1, the lower the visibility value, the worse the visibility and the higher the fog concentration. It should be noted that Table 1 is merely an illustrative example to represent the correspondence, and this application does not limit the specific correspondence. The above correspondence can also be represented in other forms, not limited to tables.

[0100] Table 1

[0101] visibility Qualitative assessment of visibility Qualitative description of weather 20km-30km Excellent visibility and clear field of vision normal 15km-20km Good visibility and clear field of vision normal 10km-15km Visibility was average. normal 1km-10km Poor visibility and unclear field of vision mist 500m-1km Poor visibility and unclear field of vision fog 200m-500m Visibility was very poor. Dense fog 50m-200m extremely poor visibility Dense fog <50m Visibility was almost zero. Dense fog

[0102] For simplicity, weather with visibility above 10km can be defined as normal weather, and weather with visibility below 10km can be defined as foggy weather. Depending on the performance of the image recognition algorithm, different classifications can be set for foggy weather, such as dividing it into light fog and dense fog, or into light fog, fog, heavy fog, dense fog, and very dense fog. This embodiment does not impose any limitations on this. For example, weather with visibility between 10km and 1km can be defined as light fog, and weather with visibility below 1km can be defined as dense fog.

[0103] The first terminal determines the visibility of the environmental image using an image recognition algorithm, and then determines the weather conditions of the environment in which the first terminal is located based on the visibility. For example, the first terminal determines visibility based on image sharpness; another example is based on the brightness difference between the target object in the image and the sky background; yet another example is based on the grayscale level of the image. This is because fog is always grayscale, so in a foggy image, objects that should be dark will appear grayscale, and the higher the fog concentration, the higher the grayscale level.

[0104] As an example, the aforementioned normal weather conditions include sunny days and days with very low fog concentration (e.g., visibility above 10 km), while abnormal weather includes dense fog or light fog. This embodiment uses an image recognition algorithm to distinguish between these normal and abnormal weather conditions. It should be understood that abnormal weather conditions may also include rainy days, snowy days, etc. In some embodiments, the first terminal can update the image recognition algorithm to make a more detailed determination of the weather conditions of its environment, thereby distinguishing between rainy days, snowy days, foggy days, etc.

[0105] This step is mainly used to distinguish between two scenarios: fogging of the glass on the first terminal and the first terminal being located in a foggy environment. This embodiment provides the following three implementation methods for this:

[0106] The first implementation determines the current state information of the first terminal by analyzing the brightness change patterns of at least two target objects in a single frame of environmental image (e.g., the first environmental image in at least one frame of environmental image). Taking two target objects (two detection points) as an example, if the difference in brightness attenuation between the two target objects is greater than or equal to a preset threshold, it can be considered that the brightness change patterns of the two target objects are inconsistent, and it can be determined that the windshield of the first terminal is fogged up; if the difference in brightness attenuation between the two target objects is less than the preset threshold, it can be considered that the brightness change patterns of the two target objects are consistent, and the weather condition (light fog or dense fog) of the environment where the first terminal is located can be further confirmed.

[0107] It should be understood that any target object possesses an inherent brightness. Under the influence of the atmospheric environment, the surface brightness of the target object captured by the image acquisition device of the first terminal will be less than this inherent brightness, i.e., there is a certain degree of brightness attenuation. Brightness attenuation is mainly affected by the atmospheric environment; the worse the environmental quality, the greater the brightness attenuation. For example, in foggy conditions, the higher the fog concentration, the greater the brightness attenuation. If the brightness variation patterns of two target objects are inconsistent, meaning that the two target objects are affected differently by the atmospheric environment, it can be assumed that the two target objects are not in the same environment. This situation is highly likely caused by the in-vehicle environment (e.g., fogging of the in-vehicle windows), confirming that the windshield of the first terminal is fogged. If the brightness variation patterns of two target objects are consistent, meaning that the two target objects are affected equally by the atmospheric environment, it can be assumed that the two target objects are in the same environment, further allowing determination of the weather conditions of the environment where the first terminal is located.

[0108] The second implementation method determines the current state information of the first terminal by analyzing the image quality of near-end target objects in a single frame of environmental image (e.g., the first environmental image in at least one frame of environmental image). If the image quality parameters of the near-end target objects meet preset conditions, the weather condition of the environment in which the first terminal is located (light fog or dense fog) can be further confirmed; if the image quality parameters of the near-end target objects do not meet preset conditions, it is considered that the windshield of the first terminal is fogged up.

[0109] It should be noted that the image quality of an environmental image can be judged by various methods, including but not limited to sharpness, signal-to-noise ratio, color, white balance, distortion, and motion effects. Sharpness refers to the clarity of the textures and boundaries of various details in an image, and can be evaluated by various methods, as detailed below. This application's embodiments demonstrate determining the weather conditions or whether the windshield is fogged by analyzing the sharpness of near-field objects in an environmental image. Of course, the weather conditions or whether the windshield is fogged can also be determined based on other image quality factors mentioned above, and this application does not impose any limitations on this.

[0110] The third implementation method determines the current state information of the first terminal by analyzing the brightness variation pattern or image quality of the same target object in multiple environmental images acquired by the same image acquisition device. Taking two environmental images as an example, if the brightness variation pattern of the same target object in the two environmental images is consistent, or the difference in image quality of the same target object in the two environmental images falls within a certain numerical range, it can be considered that the image quality of the same target object in the two environmental images is consistent or the same, and it is then considered that the windshield of the first terminal is fogged up; if the brightness variation pattern of the same target object in the two environmental images is inconsistent, or the difference in image quality of the same target object in the two environmental images falls outside a certain numerical range, it can be considered that the image quality of the same target object in the two environmental images is inconsistent or different, and the weather condition (light fog or dense fog) of the environment where the first terminal is located can be further confirmed.

[0111] The following is in conjunction with the appendix Figure 4 The first implementation of step 202 will be described in detail. In this implementation, the environment presented in the environmental image includes at least two target objects and the sky. The at least two target objects can be moving objects such as vehicles and pedestrians, or fixed objects such as traffic lights, signs, green belts, and lane lines. This embodiment does not impose any restrictions on this.

[0112] Figure 4 A flowchart for determining the state information of a first terminal is provided in an embodiment of this application, such as... Figure 4 As shown, step 202 above specifically includes:

[0113] Step 301: Obtain the brightness information of at least two target objects, the brightness information of the sky, and the depth information of at least two target objects in a single frame of environmental image.

[0114] In this embodiment of the application, a single-frame environmental image can be a specific environmental image from at least one set of environmental images captured by at least one image acquisition device, such as the first environmental image. Here, "first" does not represent a temporal sequence, but rather any environmental image captured by an image acquisition device. Optionally, the first environmental image can also be a frame environmental image from the at least one set of environmental images that satisfies preset conditions or rules. Here, the preset conditions or rules are not specifically limited.

[0115] The brightness of the target object in the atmosphere satisfies Koschmieder's law:

[0116] L=L0e -kd +L f (1-e -kd )

[0117] In the formula, L represents the surface brightness of the target object, L0 represents the intrinsic brightness of the target object, and L f d represents the brightness of the sky, d represents the distance between the target object and the first terminal, and k represents the extinction coefficient (which is equal to the sum of the absorption coefficient and the diffusion coefficient).

[0118] The brightness information of the target object includes its surface brightness and intrinsic brightness. The first terminal obtains the surface brightness of the target object (L in the above formula) by extracting the brightness value of the image patch corresponding to the target object in the environmental image. Different types of target objects correspond to different intrinsic brightness values, and the first terminal can pre-store the intrinsic brightness values ​​of different types of target objects. Similarly, the first terminal obtains the brightness information of the sky (L in the above formula) by extracting the brightness value of the image patch corresponding to the sky in the environmental image. f The first terminal can acquire depth information of any target object in the environmental image based on monocular or binocular visual ranging methods. This depth information indicates the distance of any target object from the first terminal (or the image acquisition device of the first terminal), i.e., d in the above formula. One monocular ranging method utilizes the target contact point; the projection of the target contact point onto the camera forms a similar triangle with the optical axis. Based on the principle of similar triangles, the distance from the camera to the target contact point can be obtained. Binocular ranging, on the other hand, directly measures the distance to the target by calculating the parallax of the two images obtained from binocular vision.

[0119] It should be noted that the brightness information and extinction coefficient of the target object are not limited to the aforementioned Koschmieder law, but may also include Allard's law of atmospheric light illuminance transmission, Mie scattering theory, etc., which can all serve as the basis for calculating the extinction coefficient. This application does not impose any restrictions on these aspects.

[0120] Step 302: Determine the state information of the first terminal based on the brightness information of at least two target objects in a single frame of environmental image, the brightness information of the sky, and the depth information of at least two target objects.

[0121] For the same target object, the object's brightness information, the sky's brightness information, and the object's depth information collectively indicate the degree of brightness loss of the target object in the atmosphere. Specifically, when the object's brightness information, the sky's brightness information, and the object's depth information are known, the extinction coefficient corresponding to the target object can be calculated based on the aforementioned Koschmieder's law. The extinction coefficient is used to indicate the degree of brightness loss of the target object in the atmosphere.

[0122] In this step, the extinction coefficients of at least two target objects can be determined based on Koschmieder's law. Taking two target objects as an example, the first terminal can determine its state information based on the extinction coefficients of these two target objects. Specifically, it is determined whether the difference between the extinction coefficients of the two target objects is less than a first threshold. If the difference is less than or equal to the first threshold, the extinction coefficients of the two target objects are considered to be consistent (or the same), and it can be further determined whether the weather condition of the environment where the first terminal is located is foggy, whether it is dense fog or light fog. If the difference is greater than the first threshold, the extinction coefficients of the two target objects are considered to be inconsistent (or different), and it is determined that the glass of the first terminal is fogged. It should be noted that the above-mentioned first threshold can be set based on experience, or it can be fine-tuned based on the actual detection effect. This application embodiment does not impose any restrictions on the setting method.

[0123] For multiple target objects, the first terminal can determine its state information based on the extinction coefficients corresponding to the multiple target objects. Specifically, it determines whether the extinction coefficients corresponding to the multiple target objects are consistent. If the difference between any two of the extinction coefficients corresponding to the multiple target objects is less than a first threshold, then the extinction coefficients corresponding to the multiple target objects are considered consistent, and the weather conditions of the environment where the first terminal is located can be further determined. If there is at least one group of target objects among the multiple target objects, and the extinction coefficients corresponding to any two target objects in each group are greater than or equal to the first threshold, then the extinction coefficients corresponding to the multiple target objects are considered inconsistent, and the glass of the first terminal is determined to be fogged.

[0124] It should be understood that if the glass in a certain area of ​​the first terminal fogs up, the target object in that area will not conform to the brightness change law. Regardless of the distance of the target object in that area, the extinction coefficient calculated for the target object in that area will be significantly different from the extinction coefficient calculated for other target objects outside that area (i.e., the difference between the extinction coefficient corresponding to the target object in that area and the extinction coefficient corresponding to the target object in other areas is greater than the first threshold). Through the above-mentioned judgment process in this embodiment, the first terminal is equipped with the ability to detect whether the glass of the first terminal is fogged up, thereby improving the intelligence level of the first terminal.

[0125] The detection method provided in this embodiment acquires a single-frame environmental image from at least one image acquisition device and determines whether the extinction coefficients of two or more target objects in the environmental image are consistent. If the extinction coefficients of the two or more target objects are inconsistent, it can be determined that the glass of the first terminal is fogged. The above judgment process enables the first terminal to detect whether its glass is fogged, thus distinguishing between foggy weather and fogged glass, and improving the intelligence level of the first terminal.

[0126] The following is in conjunction with the appendix Figure 5 The second implementation of step 202 is described in detail. In this implementation, the environment presented by the environmental image includes at least one near-end target object, which includes objects outside the first terminal that are within a preset distance of the first terminal. Taking a vehicle as an example, the near-end target object can be the front or rear hood of the vehicle, any object fixed on the hood, or the rearview mirrors on the left and right sides of the vehicle, etc. Specifically, markers can be set on the near-end target object, such as red dots or crosses on the front hood of the vehicle. The first terminal determines the near-end target object by identifying the markers in the environmental image based on the environmental image captured by the image acquisition device (e.g., a camera).

[0127] Figure 5 A flowchart for determining the state information of a first terminal is provided in an embodiment of this application, such as... Figure 5 As shown, step 202 above specifically includes:

[0128] Step 401: Obtain the sharpness value of at least one near-end target object in a single frame of the environment image.

[0129] and Figure 4 Similar to the embodiments shown, the single-frame environmental image in the embodiments of this application can also be a certain frame of environmental image in at least one frame of environmental image acquired by at least one image acquisition device, such as the first frame of environmental image. Here, "first frame" does not represent a temporal relationship and can be any frame of environmental image acquired by an image acquisition device.

[0130] In quality assessment without a reference image, image sharpness is a crucial indicator of image quality, as it corresponds well to human subjective perception; low image sharpness manifests as blurriness. The first terminal can obtain the sharpness value of at least one near-end target object in the environmental image based on any sharpness algorithm. Several commonly used and representative sharpness algorithms can be employed, including: Brenner gradient function, Tenengrad gradient function, Laplacian gradient function, SMD (grayscale variance) function, variance function, and energy gradient function. This application does not impose specific limitations, but rather focuses on obtaining a sharpness value.

[0131] As an example, the first terminal can obtain the sharpness value of at least one near-end target object in the environment image through the Brenner gradient function, which is used to calculate the square of the gray-level difference between two adjacent pixels, and can be expressed as:

[0132] D = ∑ y ∑ x |f(x+2,y)-f(x,y)| 2

[0133] In the formula, x,y represent pixel coordinates, f(x,y) represents the gray value of the corresponding point (x,y), and D represents the image sharpness value.

[0134] The first terminal can determine the sharpness value of at least one near-end target object by obtaining the grayscale value of the image block corresponding to at least one near-end target object based on the Brenner gradient function mentioned above.

[0135] Step 402: Determine the state information of the first terminal based on the sharpness value of at least one near-end target object in a single frame of environmental image.

[0136] Taking a near-end target object as an example, the first terminal determines its status information by comparing the sharpness value of the near-end target object with a preset sharpness threshold. If the sharpness value of the near-end target object is less than or equal to the preset sharpness threshold, it is determined that the glass of the first terminal is fogged up; if the sharpness value of the near-end target object is greater than the preset sharpness threshold, the weather conditions of the environment in which the first terminal is located can be further determined. It should be noted that the above-mentioned sharpness threshold can be set based on experience, or it can be fine-tuned based on the actual detection effect. This application embodiment does not impose any restrictions on the setting method.

[0137] The detection method provided in this embodiment acquires a single-frame environmental image from at least one image acquisition device, determines the image clarity of near-end target objects in the environmental image, and determines whether the glass of the first terminal is fogged based on the image clarity. This determination process enables the first terminal to detect whether its glass is fogged, distinguishing between foggy weather and fogged glass, thus improving the intelligence level of the first terminal.

[0138] Optionally, based on the above embodiments, when the first terminal determines that its glass is fogging, it can control the activation of the defogging device inside the first terminal (e.g., a vehicle-mounted fresh air system or air conditioner), or control the activation of the window lift device, or issue an alarm message (which can be issued through screen display, voice broadcast, or vibration). The first two methods can directly eliminate the fog on the glass of the first terminal, achieving a balance in the temperature difference between the inside and outside of the terminal and ensuring the driving safety of the first terminal. In the latter method, the user can manually intervene based on the alarm message to ensure the driving safety of the first terminal.

[0139] The above embodiments all involve detecting the terminal environment based on single-frame environmental images captured by an image acquisition device. The following embodiment illustrates detecting the terminal environment based on multiple frames of environmental images captured by an image acquisition device.

[0140] The following is in conjunction with the appendix Figure 6The third implementation method of step 202 is described in detail. By analyzing multiple frames of environmental images, it is determined whether the glass of the first terminal is fogged up, or the process is carried out according to the environmental weather detection method.

[0141] Figure 6 This is a flowchart illustrating how to determine the state information of a first terminal, as provided in an embodiment of this application. Figure 6 As shown, the detection method provided in this embodiment includes the following steps:

[0142] Step 501: Acquire multiple frames of environmental images from at least one image acquisition device. The environmental images are used to present information about the environment in which the first terminal is located.

[0143] The implementation process of this step is the same as step 201 in the above embodiment, except that the acquired environmental image is multiple frames. For details, please refer to the above embodiment, which will not be repeated here.

[0144] Step 502: Determine the status information of the first terminal based on multiple frames of environmental images.

[0145] In this step, multiple environmental images are all from the same image acquisition device. Based on these multiple environmental images from the same image acquisition device, the status information of the first terminal is determined. In practical applications, the specific number of frames can be set according to different needs; for example, five consecutive environmental images can be acquired according to a preset sampling interval.

[0146] The status information includes at least one of the following:

[0147] Is the glass of the first terminal fogged up? Or, what is the weather condition of the environment in which the first terminal is located?

[0148] In this embodiment, the environment presented by the environmental image includes at least one target object. The first terminal determines its state information based on multiple frames of environmental images, including the following two implementation methods:

[0149] The first implementation determines the current state information of the first terminal by analyzing the brightness change pattern of at least one target object in multiple environmental images. If the difference in brightness attenuation of the same target object in multiple environmental images is less than a preset threshold, it can be considered that the brightness change pattern of the target object is consistent in the multiple environmental images, and it can be determined that the windshield of the first terminal is fogged up; if the difference in brightness attenuation of the same target object in multiple environmental images is greater than or equal to the preset threshold, it can be considered that the brightness change pattern of the target object is inconsistent in the multiple environmental images, and the weather condition (light fog or dense fog) of the environment where the first terminal is located can be further confirmed.

[0150] The second implementation method determines the current state information of the first terminal by analyzing the image quality of at least one target object in multiple frames of environmental images. If the difference in image quality parameters of the same target object in multiple frames of environmental images is less than a preset threshold, it can be considered that the image quality of the target object is consistent across multiple frames of environmental images, and it can be determined that the windshield of the first terminal is fogged up. If the difference in image quality parameters of the same target object in multiple frames of environmental images is greater than or equal to the preset threshold, it can be considered that the image quality of the target object is inconsistent across multiple frames of environmental images, and the weather condition (light fog or dense fog) of the environment where the first terminal is located can be further confirmed. The aforementioned image quality parameters include, but are not limited to, sharpness, signal-to-noise ratio, color, white balance, distortion, and motion effects.

[0151] Specifically, in the first implementation of this embodiment, step 502 specifically includes:

[0152] State information is determined based on the extinction coefficient of at least one target object in multiple frames of environmental images.

[0153] For the same target object in multiple frames of environmental images, the extinction coefficient of the target object in each frame of the environmental image is determined by the brightness information of the target object, the brightness information of the sky, and the depth information of the target object. The calculation process of the extinction coefficient is the same as step 301 in the above embodiment, which can be referred to the above embodiment for details, and will not be repeated here.

[0154] Taking an environment image containing a target object as an example, after determining the extinction coefficient of the same target object in multiple environmental images, the first terminal determines its state information based on the difference in the extinction coefficients of the same target object in any two environmental images. If the difference in the extinction coefficients of the same target object in any two environmental images is less than or equal to a fourth threshold, it can be considered that the extinction coefficients of the target object are consistent or the same in the multiple environmental images, and it can be determined that the glass of the first terminal is fogged. If the difference in the extinction coefficients of the same target object in two environmental images is greater than the fourth threshold, it can be considered that the extinction coefficients of the target object are inconsistent or different in the multiple environmental images, and it can be further determined whether the weather condition of the environment where the first terminal is located is foggy, whether it is dense fog or light fog, for details please refer to Figure 7 Example. It should be noted that the above-mentioned fourth threshold can be set based on experience, or it can be fine-tuned based on the actual detection effect. This application embodiment does not impose any restrictions on the setting method.

[0155] In some embodiments, the first terminal can determine the changes in the extinction coefficients of multiple target objects in multiple frames of environmental images. If the extinction coefficients of each of the multiple target objects are consistent or the same in the multiple frames of environmental images, it is considered that the glass of the first terminal is fogged. Compared with determining a single target object in multiple frames of environmental images, this detection method is more accurate.

[0156] Specifically, in the second implementation of this embodiment, step 502 specifically includes:

[0157] State information is determined based on the sharpness value of at least one target object in multiple frames of environmental images.

[0158] For the same target object in multiple frames of environmental images, the sharpness value of the target object in each frame of the environmental image is determined by the grayscale value of the image block corresponding to the target object. The calculation process of the sharpness value is the same as step 401 in the above embodiment, which can be referred to the above embodiment for details, and will not be repeated here.

[0159] Taking an environment image containing a target object as an example, after determining the sharpness value of the same target object in multiple environmental images, the first terminal determines its status information based on the difference in sharpness values ​​of the same target object in multiple environmental images. Specifically, if the difference in sharpness values ​​of the same target object in any two environmental images is less than or equal to a fifth threshold, it can be considered that the sharpness values ​​of the target object are consistent or the same in the multiple environmental images, and it can be determined that the glass of the first terminal is fogged. If the difference in sharpness values ​​of the same target object in two environmental images is greater than the fifth threshold, it can be considered that the sharpness values ​​of the target object are inconsistent or different in the multiple environmental images, and it can be further determined whether the weather condition of the environment where the first terminal is located is foggy, whether it is dense fog or light fog, for details please refer to Figure 7 Example. It should be noted that the above-mentioned fifth threshold can be set based on experience, or it can be fine-tuned based on the actual detection effect. This application embodiment does not impose any restrictions on the setting method.

[0160] Optionally, the target object selected in the multi-frame environmental image can be a near-end target object, which includes objects outside the first terminal that are less than a preset distance from the first terminal.

[0161] Optionally, the number of environmental images to be acquired and the time interval between acquiring multiple environmental images can be preset according to actual needs. For example, if the interval is set to acquire one environmental image at 0.1s, the first terminal can perform environmental detection based on 5 consecutive environmental images.

[0162] It should be noted that, regardless of the execution of the first terminal... Figure 4 , Figure 5 , Figure 6 The judgment process shown all involve the following situation: It is necessary to further determine the weather conditions of the environment where the first terminal is located, which can be handled according to the environmental weather detection methods. The following is in conjunction with the appendix... Figure 7 The process of judging foggy weather conditions is explained in detail.

[0163] It should be understood that different image acquisition devices have different shooting angles. Therefore, as an optional approach to determining the state information of the first terminal, a comprehensive judgment can be made based on multiple frames of environmental images acquired by image acquisition devices from different angles to determine the state information of the first terminal. Taking two image acquisition devices as an example, namely the first image acquisition device and the second image acquisition device, step 502 may include: determining the first state information of the first terminal based on the multiple frames of environmental images from the first image acquisition device; determining the second state information of the first terminal based on the multiple frames of environmental images from the second image acquisition device; and determining the state information of the first terminal based on the first state information and the second state information.

[0164] As an example, if the first state information and the second state information are the same (e.g., glass fogging), then the state information of the first terminal is glass fogging; if the first state information and the second state information are different (e.g., the first state information is glass fogging, and the second state information is ambient fog), the state information of the first terminal can be determined according to the weight of the image acquisition device (e.g., if the weight of the first image acquisition device is greater than the weight of the second image acquisition device, then the state information of the first terminal is determined to be the first state information (glass fogging)). The weight of the image acquisition device is related to its hardware performance; the stronger the hardware performance, the larger the weight value. It should be noted that the above scheme is merely an exemplary description, and the embodiments of this application do not limit the above judgment rules.

[0165] Figure 7 This is a flowchart illustrating how to determine the weather status of a first terminal, as provided in an embodiment of this application. Figure 7 As shown, the detection method provided in this embodiment includes:

[0166] Step 601: Obtain the saturation and brightness of any frame of environmental image from at least one image acquisition device.

[0167] In this embodiment, the first terminal obtains the overall saturation (S) and brightness (V) of the environmental image using the HSV color model (Hue, Saturation, Value). Saturation (S) represents how closely the colors of the environmental image resemble spectral colors. For a given color, it can be considered as a mixture of a spectral color and white; the greater the proportion of the spectral color, the closer the color is to the spectral color, and the higher the color saturation. Brightness (V) represents the lightness or darkness of the colors in the environmental image. Both S and V range from 0% to 100%.

[0168] Step 602: Determine the weather conditions of the environment where the first terminal is located based on the ratio of brightness to saturation.

[0169] Compared to normal weather conditions, the ratio of brightness to saturation (V / S) of an environmental image is larger in foggy conditions. Therefore, this ratio can be used to determine whether the weather is foggy, and whether it is light fog or dense fog. Specifically, if the ratio is greater than or equal to a second threshold, the weather condition of the environment where the first terminal is located is determined to be dense fog; if the ratio is greater than a third threshold but less than the second threshold, the weather condition of the environment where the first terminal is located is determined to be light fog; if the ratio is less than or equal to the third threshold, the weather condition of the environment where the first terminal is located is determined to be normal weather (little or no fog). It should be noted that the second and third thresholds can be set empirically or fine-tuned based on actual detection results. This embodiment of the application does not impose any restrictions on the setting method.

[0170] Optionally, the first terminal can perform corresponding control operations based on the determined weather conditions. In this embodiment, different control operations can be corresponding to different foggy weather conditions with different concentration levels.

[0171] When the first terminal determines that the weather is dense fog, the first terminal controls its own driving state (for example, switching the driving state from fully automated driving state to semi-automated driving state, that is, reducing the level of automated driving of the first terminal), or the first terminal outputs information to the vehicle controller, and the vehicle controller sends control commands to the relevant devices of the first terminal, such as sending a command to the fog lights of the first terminal to turn them on.

[0172] When the first terminal determines that the weather is light fog, it performs defogging on the environmental image and then sends the defogging image data to the detection module for road detection. Based on the road detection results, it executes the corresponding driving strategy (e.g., acceleration, deceleration, or stopping). Alternatively, it may not perform any operation (e.g., maintain the current driving state) or handle the situation in the manner described above for dense fog (e.g., reduce the autonomous driving level of the first terminal).

[0173] When the first terminal determines that the weather is normal, it can directly perform road detection based on the environmental image. For details, please refer to the above text, which will not be repeated here.

[0174] Optionally, the first terminal can update its image recognition algorithm to make more detailed judgments about the weather conditions of its environment, such as adding the recognition of abnormal weather and distinguishing between foggy, rainy, and snowy weather. Different control operations can be performed for different abnormal weather conditions. For example, when the first terminal determines that the weather is rainy, it can output information to the vehicle controller, which will then send an activation command to the windshield wipers. The controller can also intelligently adjust the wiper frequency based on the amount of rainfall.

[0175] The detection method provided in this embodiment calculates the saturation and brightness of an environmental image, and determines the fog concentration level based on the ratio of brightness to saturation. This judgment process enables the first terminal to detect foggy environments and execute corresponding control operations based on the fog concentration level. When the fog concentration is low, there is no need to switch the driving state of the first terminal; the environmental image can be defogged using image processing algorithms, thereby avoiding waste of control system resources.

[0176] In summary, if the distinction between foggy weather and fogged windows cannot be effectively made, the first terminal may apply defogging algorithms or downgrade the autonomous driving level when the windows are fogged, resulting in a waste of control system resources. Alternatively, it may mistake foggy weather for fogged windows, rendering the defogging device ineffective. Based on the detection method provided in the above embodiments, the first terminal can quickly identify its status information and execute corresponding control operations based on different status information, thereby improving the intelligence level of the first terminal.

[0177] It should be noted that the execution entity of the above-described method embodiments can be a first terminal (e.g., an autonomous vehicle) or a component on the first terminal (e.g., a detection device, chip, controller, or control unit), or a cloud device communicatively connected to the first terminal. This application does not impose any limitations on these embodiments. As an example, the detection device can be an image acquisition device (e.g., a camera device), the controller can be a multi-domain controller (MDC), and the control unit can be an electronic control unit (ECU), also known as a vehicle computer.

[0178] Taking the detection device on the first terminal as an example, the embodiments of this application can divide the detection device into functional modules according to the above method embodiments. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware or software functional modules. It should be noted that the module division in the embodiments of this application is illustrative and is only a logical functional division. In actual implementation, there may be other division methods. The following description uses the division of each functional module according to each function as an example. The detection device described below can also be replaced by possible execution entities such as chips, controllers, or control units.

[0179] Figure 8 This is a schematic diagram of a detection device provided in an embodiment of this application. Figure 8 As shown, the detection device 700 provided in this application embodiment includes:

[0180] Acquisition module 701 is used to acquire at least one frame of environmental image from at least one image acquisition device, the environmental image being used to present information about the environment in which the first terminal is located;

[0181] Processing module 702 is configured to determine the state information of the first terminal based on the at least one frame of environmental image, the state information including at least one of the following:

[0182] Is the glass of the first terminal fogged up, or

[0183] The weather conditions of the environment in which the first terminal is located.

[0184] Optionally, the weather condition includes any one of dense fog, light fog, or normal.

[0185] Optionally, the status information includes whether the glass of the first terminal is fogged up, and the environment image shows that the environment includes at least two target objects and the sky;

[0186] The processing module 702 is specifically used to determine the state information based on the brightness information of at least two target objects in the first environment image of the at least one frame of environment image, the brightness information of the sky, and the depth information of the at least two target objects.

[0187] Optionally, the processing module 702 determines that the glass of the first terminal is fogged up, wherein in the first environmental image, there is at least one group of target objects among the at least two target objects, and the difference in the extinction coefficients of any two target objects in each group of target objects is greater than a first threshold. The extinction coefficients are determined by the brightness information of the target objects, the brightness information of the sky, and the depth information of the target objects. The extinction coefficients are used to indicate the degree of brightness loss of the target objects in the atmosphere.

[0188] Optionally, the status information includes whether the glass of the first terminal is fogged up, and the environment presented by the environmental image includes at least one near-end target object, which includes an object outside the first terminal that is less than a preset distance from the first terminal.

[0189] The processing module 702 is specifically used to determine the state information based on the sharpness value of at least one near-end target object in the first environmental image in the at least one frame of environmental image;

[0190] The sharpness value of the at least one near-end target object is determined by the grayscale value of the image block corresponding to the at least one near-end target object.

[0191] Optionally, the processing module 702 determines that the glass of the first terminal is fogged up, and in the first environmental image, at least one near-end target object has a sharpness value that is less than or equal to a preset sharpness threshold.

[0192] Optionally, the acquisition module 701 is specifically used to acquire multiple frames of environmental images from at least one image acquisition device;

[0193] The processing module 702 is specifically used to determine the status information of the first terminal based on the multi-frame environmental images.

[0194] Optionally, the status information includes whether the glass of the first terminal is fogged up, and the environment presented by the environmental image includes at least one target object;

[0195] The processing module 702 is specifically used to determine the state information based on the extinction coefficient or sharpness value of the at least one target object in the multi-frame environmental images;

[0196] The extinction coefficient of the at least one target object in each frame of the environmental image is determined by the brightness information of the at least one target object, the brightness information of the sky, and the depth information of the at least one target object; the sharpness value of the at least one target object in each frame of the environmental image is determined by the grayscale value of the image block corresponding to the at least one target object.

[0197] Optionally, the processing module 702 determines that the glass of the first terminal is fogged, wherein the difference in the extinction coefficient of the same target object in any two frames of the multi-frame environmental images is less than or equal to a fourth threshold.

[0198] Optionally, the processing module 702 determines that the glass of the first terminal is fogged up, wherein the difference in sharpness value of the same target object in any two frames of the multi-frame environmental images is less than or equal to a fifth threshold.

[0199] Optionally, the same target object refers to the same near-end target object, which includes objects outside the first terminal that are less than a preset distance from the first terminal.

[0200] Optionally, when the processing module 702 determines that the glass of the first terminal is fogged up, it is further configured to:

[0201] The system can activate the defroster, activate the window lift, or issue an alarm.

[0202] Optionally, when the processing module 702 determines that the glass of the first terminal is not fogged, the acquisition module 701 is further configured to:

[0203] Acquire the saturation and brightness of any frame of environmental image from at least one image acquisition device;

[0204] The processing module 702 is further configured to determine the weather conditions of the environment in which the first terminal is located based on the ratio of the brightness to the saturation.

[0205] Optionally, when the weather condition is dense fog, the ratio is greater than or equal to the second threshold; when the weather condition is light fog, the ratio is greater than the third threshold and less than the second threshold; and / or

[0206] When the weather condition is normal, the ratio is less than or equal to the third threshold.

[0207] Optionally, the processing module 702 is further configured to:

[0208] When the weather condition is determined to be dense fog, control the driving state of the first terminal or output control information to the vehicle controller; or

[0209] When the weather condition is determined to be fog, the environmental image is dehazed; or

[0210] When the weather condition is determined to be normal, road detection is performed based on the environmental image.

[0211] Optionally, the detection device provided in this application embodiment may further include a communication module. The communication module is used to send control commands to the defogging device or window regulator on the first terminal. The control commands are used to control the defogging device inside the first terminal vehicle to start, or to control the window regulator of the first terminal to start. Alternatively, the communication module is used to send alarm information to the display device, voice device, or vibration device of the first terminal, and the alarm can be issued through screen display, voice broadcast, or vibration.

[0212] The detection device provided in this application is used to execute the detection scheme of any of the foregoing method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0213] Figure 9 This is a schematic diagram of the hardware structure of a detection device provided in an embodiment of this application. Figure 9 As shown, the detection device 800 provided in this application embodiment includes:

[0214] At least one processor 801 ( Figure 9 Only one processor is shown in the image) and at least one memory 802 ( Figure 9 Only one memory is shown in the image.

[0215] The at least one memory 802 is used to store computer execution instructions. When the detection device 800 is running, the at least one processor 801 executes the computer execution instructions stored in the at least one memory 802 to cause the detection device 800 to execute the detection scheme of any of the aforementioned method embodiments.

[0216] It should be noted that the detection device 800 provided in this application embodiment can be set on the first terminal or on a cloud device, and this application embodiment does not impose any restrictions on this.

[0217] This application also provides a computer storage medium for storing a computer program, which, when run on a computer, causes the computer to execute the detection method in any of the foregoing method embodiments.

[0218] This application also provides a computer program product that, when run on a computer, causes the computer to execute the detection method in any of the foregoing method embodiments.

[0219] This application also provides a chip, including: at least one processor and an interface, for calling and running a computer program stored in at least one memory to execute the detection method in any of the foregoing method embodiments.

[0220] This application also provides an autonomous driving system, which includes one or more first terminals as described above, and one or more cloud devices, wherein the first terminals are equipped with the aforementioned detection device, or the cloud devices are equipped with the aforementioned detection device, so that the autonomous driving system can distinguish between windshield fogging and ambient weather, thereby improving the accuracy of the system's environmental detection.

[0221] This application also provides a vehicle that includes the aforementioned detection device. The detection device enables the vehicle to distinguish between windshield fogging and ambient weather conditions, thereby controlling the activation or deactivation of other devices on the vehicle (e.g., defoggers, window regulators, display devices, vibration devices, voice devices, etc.). Furthermore, the vehicle also includes at least one camera device and / or at least one radar device. The radar device includes at least one of millimeter-wave radar, lidar, or ultrasonic radar.

[0222] Optionally, the vehicle can be a sedan, SUV, sports car, truck, bus, recreational vehicle, amusement park vehicle, construction equipment, tram, golf cart, train, etc., and this application embodiment does not impose any limitations on this. It should be understood that the processor mentioned in this application embodiment can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0223] It should also be understood that the memory mentioned in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0224] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, the memory (storage module) is integrated into the processor.

[0225] It should be noted that the memories described herein are intended to include, but are not limited to, these and any other suitable types of memories.

[0226] It should also be understood that the use of the terms "first," "second," and various numerical designations in this document is merely for descriptive convenience and is not intended to limit the scope of this application.

[0227] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0228] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0229] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A detection method, characterized in that, The method includes: Acquire at least one frame of environmental image from at least one image acquisition device, the environmental image being used to present information about the environment in which the first terminal is located; Based on the at least one frame of environmental image, the state information of the first terminal is determined, and the state information includes at least one of the following: Is the glass of the first terminal fogged up, or The weather conditions of the environment in which the first terminal is located; When the status information includes whether the glass of the first terminal is fogged up, the environment image presented includes at least two target objects and the sky. Based on the at least one frame of environmental image, the state information of the first terminal is determined, including: The first terminal's glass is determined to be fogged, wherein in the first environmental image of the at least one frame of environmental image, there is at least one group of target objects among the at least two target objects, and the difference in the extinction coefficients of any two target objects in each group of target objects is greater than a first threshold. The extinction coefficient is determined by the brightness information of the target object, the brightness information of the sky, and the depth information of the target object. The extinction coefficient is used to indicate the degree of brightness loss of the target object in the atmosphere.

2. The method according to claim 1, characterized in that, The weather conditions include any one of dense fog, light fog, or normal.

3. The method according to claim 1 or 2, characterized in that, The status information includes whether the glass of the first terminal is fogged up, and the environment presented by the environmental image includes at least one near-end target object, which includes an object outside the first terminal that is less than a preset distance from the first terminal. Based on the at least one frame of environmental image, the state information of the first terminal is determined, including: The state information is determined based on the sharpness value of at least one near-end target object in the first environment image in the at least one frame of environment image; The sharpness value of the at least one near-end target object is determined by the grayscale value of the image block corresponding to the at least one near-end target object.

4. The method according to claim 3, characterized in that, Determining the state information based on the sharpness value of at least one near-end target object in the first environmental image includes: If the glass of the first terminal is found to be fogged up, then in the first environmental image, at least one near-end target object has a sharpness value that is less than or equal to a preset sharpness threshold.

5. The method according to claim 1, characterized in that, The acquisition of at least one frame of environmental image from at least one image acquisition device includes: Acquire multiple frames of environmental images from at least one image acquisition device; Determining the status information of the first terminal based on the environmental image includes: The status information of the first terminal is determined based on the multi-frame environmental images.

6. The method according to claim 5, characterized in that, The status information includes whether the glass of the first terminal is fogged up, and the environment presented by the environmental image includes at least one target object; Determining the state information of the first terminal based on the multi-frame environmental images includes: The state information is determined based on the extinction coefficient or sharpness value of the at least one target object in the multi-frame environmental images; The extinction coefficient of the at least one target object in each frame of the environmental image is determined by the brightness information of the at least one target object, the brightness information of the sky, and the depth information of the at least one target object; the sharpness value of the at least one target object in each frame of the environmental image is determined by the grayscale value of the image block corresponding to the at least one target object.

7. The method according to claim 6, characterized in that, Determining the state information based on the extinction coefficient of the at least one target object in the multi-frame environmental images includes: It is determined that the glass of the first terminal is fogged up, wherein the difference in the extinction coefficient of the same target object in any two frames of the multi-frame environmental images is less than or equal to the fourth threshold.

8. The method according to claim 6, characterized in that, Determining the state information based on the sharpness value of the at least one target object in the multi-frame environmental images includes: It is determined that the glass of the first terminal is fogged up, wherein the difference in sharpness value of the same target object in any two frames of the multi-frame environmental images is less than or equal to a fifth threshold.

9. The method according to claim 7 or 8, characterized in that, The same target object refers to the same near-end target object, which includes objects outside the first terminal that are less than a preset distance from the first terminal.

10. The method according to any one of claims 1-2 and 4-8, characterized in that, When determining that the glass of the first terminal is fogged up, the method further includes: The system can activate the defroster, activate the window lift, or issue an alarm.

11. The method according to any one of claims 1-2 and 4-8, characterized in that, When it is determined that the glass of the first terminal is not fogged up, the method further includes: Acquire the saturation and brightness of any frame of environmental image from at least one image acquisition device; The weather conditions of the environment in which the first terminal is located are determined based on the ratio of brightness to saturation.

12. The method according to claim 11, characterized in that, Determining the weather condition of the environment where the first terminal is located based on the ratio of brightness to saturation includes: When the weather condition is dense fog, the ratio is greater than or equal to the second threshold. When the weather condition is foggy, the ratio is greater than the third threshold and less than the second threshold; and / or When the weather condition is normal, the ratio is less than or equal to the third threshold.

13. The method according to claim 12, characterized in that, The method further includes: When the weather condition is determined to be dense fog, control the driving state of the first terminal or output control information to the vehicle controller; or When the weather condition is determined to be fog, the environmental image is dehazed; or When the weather condition is determined to be normal, road detection is performed based on the environmental image.

14. A detection device, characterized in that, include: An acquisition module is used to acquire at least one frame of environmental image from at least one image acquisition device, the environmental image being used to present information about the environment in which the first terminal is located; The processing module is configured to determine the state information of the first terminal based on the brightness variation pattern or image quality parameters of at least one target object in the at least one frame of environmental image, wherein the state information includes at least one of the following: Is the glass of the first terminal fogged up, or The weather conditions of the environment in which the first terminal is located; When the status information includes whether the glass of the first terminal is fogged up, the environment image presented includes at least two target objects and the sky; The processing module is specifically used to determine that the glass of the first terminal is fogged up. In the first environmental image of the at least one frame of environmental image, there is at least one group of target objects among the at least two target objects. The difference in the extinction coefficients of any two target objects in each group of target objects is greater than a first threshold. The extinction coefficient is determined by the brightness information of the target object, the brightness information of the sky, and the depth information of the target object. The extinction coefficient is used to indicate the degree of brightness loss of the target object in the atmosphere.

15. The apparatus according to claim 14, characterized in that, The weather conditions include any one of dense fog, light fog, or normal.

16. The apparatus according to claim 14 or 15, characterized in that, The status information includes whether the glass of the first terminal is fogged up, and the environment presented by the environmental image includes at least one near-end target object, which includes an object outside the first terminal that is less than a preset distance from the first terminal. The processing module is specifically used to determine the state information based on the sharpness value of at least one near-end target object in the first environmental image of the at least one frame of environmental image; The sharpness value of the at least one near-end target object is determined by the grayscale value of the image block corresponding to the at least one near-end target object.

17. The apparatus according to claim 16, characterized in that, The processing module determines that the glass of the first terminal is fogged up, and in the first environmental image, there is at least one near-end target object whose sharpness value is less than or equal to a preset sharpness threshold.

18. The apparatus according to claim 14, characterized in that, The acquisition module is specifically used to acquire multiple frames of environmental images from at least one image acquisition device; The processing module is specifically used to determine the status information of the first terminal based on the multi-frame environmental images.

19. The apparatus according to claim 18, characterized in that, The status information includes whether the glass of the first terminal is fogged up, and the environment presented by the environmental image includes at least one target object; The processing module is specifically used to determine the state information based on the extinction coefficient or sharpness value of the at least one target object in the multi-frame environmental images; The extinction coefficient of the at least one target object in each frame of the environmental image is determined by the brightness information of the at least one target object, the brightness information of the sky, and the depth information of the at least one target object; the sharpness value of the at least one target object in each frame of the environmental image is determined by the grayscale value of the image block corresponding to the at least one target object.

20. The apparatus according to claim 19, characterized in that, The processing module determines that the glass of the first terminal is fogged up, wherein the difference in the extinction coefficient of the same target object in any two environmental images in the multi-frame environmental images is less than or equal to a fourth threshold, or the difference in the sharpness value of the same target object in any two environmental images in the multi-frame environmental images is less than or equal to a fifth threshold. The same target object refers to the same near-end target object, which includes objects outside the first terminal that are less than a preset distance from the first terminal.

21. The apparatus according to any one of claims 14-15 and 17-20, characterized in that, When the processing module determines that the glass of the first terminal is fogged up, it is further configured to: The system can activate the defroster, activate the window lift, or issue an alarm.

22. The apparatus according to any one of claims 14-15 and 17-20, characterized in that, When the processing module determines that the glass of the first terminal is not fogged, the acquisition module is further configured to: Acquire the saturation and brightness of any frame of environmental image from at least one image acquisition device; The processing module is further configured to determine the weather conditions of the environment in which the first terminal is located based on the ratio of the brightness to the saturation.

23. The apparatus according to claim 22, characterized in that, When the weather condition is dense fog, the ratio is greater than or equal to the second threshold. When the weather condition is foggy, the ratio is greater than the third threshold and less than the second threshold; and / or When the weather condition is normal, the ratio is less than or equal to the third threshold.

24. The apparatus according to claim 23, characterized in that, The processing module is further configured to: When the weather condition is determined to be dense fog, control the driving state of the first terminal or output control information to the vehicle controller; or When the weather condition is determined to be fog, the environmental image is dehazed. or When the weather condition is determined to be normal, road detection is performed based on the environmental image.

25. A detection device, characterized in that, Includes at least one processor and at least one memory; The at least one memory is used to store computer execution instructions. When the detection device is running, the at least one processor executes the computer execution instructions stored in the at least one memory to cause the detection device to perform the detection method as described in any one of claims 1-13.

26. A computer storage medium, characterized in that, Used to store a computer program, which, when executed on a computer, causes the computer to perform the detection method according to any one of claims 1-13.

27. A computer program product, characterized in that, Used to store computer programs, which, when run on a computer, cause the computer to perform the detection method of any one of claims 1-13.

28. A chip, characterized in that, include: At least one processor and interface are configured to call and run a computer program stored in at least one memory to perform the detection method of any one of claims 1-13.

29. An autonomous driving system, characterized in that, It includes one or more first terminals and one or more cloud devices, wherein the first terminal is provided with the detection device of claim 25, or the cloud device is provided with the detection device of claim 25.

Images