An apparatus for capturing a vehicle image in a parking spot scenario

Abstract

This application provides a device for capturing vehicle images in a parking space scenario, comprising: an image sensor; a distance sensor for acquiring and generating multiple distance data; a processor for calculating distance estimates based on the multiple distance data; a first memory for storing distance estimates when the parking space is detected to be empty; and a second memory for storing distance estimates when the parking space is detected to be parked. The processor is further configured to: automatically trigger the image sensor to capture vehicle images when the distance estimate stored in the first memory falls within a preset distance range; and automatically trigger the image sensor to capture vehicle images when the distance estimate stored in the second memory is greater than a vehicle exit threshold or equal to 0. This technical solution eliminates the need for real-time operation of the image sensor, saving power resources.

Description

Technical Field

[0001] This application relates to the field of intelligent transportation technology, and in particular to a device for capturing vehicle images in parking space scenarios. Background Technology

[0002] As human society continues to develop, future cities will accommodate an ever-increasing population. To achieve sustainable urban development and enhance the overall competitiveness of cities, the construction of smart cities is imperative. The concept of "smart" is to enable humans to manage production and life in a more refined and flexible way by applying next-generation information technologies. Sensors are embedded or equipped in various facilities such as power supply systems, water supply systems, and transportation systems, forming an Internet of Things (IoT) that connects with the Internet, achieving the integration of human society and physical systems. Then, through computers and cloud computing, the IoT is integrated, and smart cities can be realized.

[0003] Intelligent transportation is an important component of smart cities, and its essence lies in vehicle management, such as the management of vehicles in motion and vehicles in parking. For the management of vehicles in motion, images of vehicles can be captured by cameras, and vehicle behavior can be analyzed based on these images to achieve effective vehicle management.

[0004] For the management of parked vehicles, such as those within parking spaces, manual detection can be used to determine when a vehicle enters and leaves the space, and then charge the vehicle accordingly. However, this method requires staff involvement; if staff are not present, vehicle charging cannot be performed.

[0005] Alternatively, cameras can be deployed to capture real-time images of parking spaces. These images can then be analyzed to determine when a vehicle enters or leaves the parking space, and subsequently, charges can be applied. However, this method requires the cameras to be constantly powered on to capture images, necessitating a connection to a power supply system. This connection method is complex, involves numerous cables, and consumes significant power. Summary of the Invention

[0006] This application provides a device for capturing vehicle images in a parking space scenario, wherein the parking space includes both an empty vehicle state and a parked vehicle state, and the device includes:

[0007] Image sensor;

[0008] A distance sensor is used to collect and generate multiple distance data points. The detection direction of the distance sensor forms an acute angle with the long side of the parking space, so that the distance sensor can detect the distance between the vehicle near the parking space and the distance sensor.

[0009] One or more processors are configured to: calculate a distance estimate based on the plurality of distance data;

[0010] The device also includes a first memory and a second memory;

[0011] The first memory is used for:

[0012] When the parking space is detected to be vacant, the distance estimate is stored.

[0013] The second memory is used for:

[0014] When the parking space is detected to be in a parked state, the distance estimate is stored;

[0015] The processor is further used for:

[0016] When the distance estimate stored in the first memory is detected to fall within a preset distance range, the image sensor is automatically triggered so that it can capture vehicle images and mark the parking space as being in a parking state.

[0017] When the distance estimate stored in the second memory is detected to be greater than the vehicle exit threshold or equal to 0, the image sensor is automatically triggered to capture vehicle images and mark the parking space as empty.

[0018] As can be seen from the above technical solutions, in this embodiment, an image sensor and a distance sensor can be deployed on the device. The distance sensor detects multiple distance data points of the parking space, calculates distance estimates based on these data points, and automatically triggers the image sensor when a vehicle entry or exit is detected. The image sensor is then used to capture vehicle images, and is turned off after image acquisition, thus eliminating the need for real-time image sensor activation and saving power. Even without real-time image sensor activation, vehicle images of vehicles entering or leaving the parking space can be acquired. These images are then analyzed to determine when a vehicle enters or leaves the parking space, and billing is performed accordingly. This method does not require connecting the device to a power supply system; batteries can be used to power the device, simplifying the connection process. By leveraging data processing technology to detect vehicle entry and exit behaviors in parking spaces, the accuracy and sensitivity of these detections are improved, significantly enhancing various metrics for identifying entry and exit behaviors, increasing the accuracy of entry and exit recognition, and exhibiting strong anti-interference capabilities. With power consumption control as the core, the system controls overall power consumption by minimizing the operating time of the image and distance sensors, resulting in excellent vehicle entry and exit behavior recognition capabilities. By employing a process of first acquiring vehicle images and then verifying the data, it balances the timeliness and accuracy of vehicle image acquisition while reducing the false alarm rate of vehicle entry and exit behavior detection. Attached Figure Description

[0019] Figure 1A and Figure 1B This is a schematic diagram of the structure of an Internet of Things (IoT) device according to one embodiment of this application;

[0020] Figures 2A-2C This is a schematic diagram illustrating the deployment location of an IoT device in one embodiment of this application;

[0021] Figure 3 This is a flowchart illustrating a vehicle behavior detection method according to one embodiment of this application;

[0022] Figure 4 This is a flowchart illustrating a vehicle behavior detection method according to one embodiment of this application;

[0023] Figure 5 This is a schematic diagram of the data processing unit in one embodiment of this application;

[0024] Figure 6 This is a schematic diagram of the vehicle entry determination unit in one embodiment of this application;

[0025] Figure 7 This is a schematic diagram of the distance calibration unit in one embodiment of this application;

[0026] Figure 8 This is a schematic diagram of the vehicle dispatch determination unit in one embodiment of this application;

[0027] Figure 9 This is a schematic diagram of the timing selection unit in one embodiment of this application;

[0028] Figure 10 This is a schematic diagram of the state switching unit in one embodiment of this application. Detailed Implementation

[0029] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “the,” and “the” as used in this application and claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to any and all possible combinations comprising one or more of the associated listed items.

[0030] It should be understood that although the terms first, second, third, etc., may be used to describe various information in embodiments of this application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" may also be interpreted as "when," "when," or "in response to a determination."

[0031] This application proposes a device for capturing vehicle images in a parking space scenario. This device can be an IoT device or other types of devices; the type is not limited, but will be used as an example below. The IoT device can be battery-powered or power-powered; the power supply type is not limited. The IoT device can communicate with the management device (i.e., the server) via a wireless network or a wired network; the communication method is not limited.

[0032] For example, a parking space may be in an empty state and in a parked state, where the empty state indicates that no vehicle is parked in the parking space, and the parked state indicates that a vehicle is parked in the parking space.

[0033] In one possible implementation, see Figure 1A As shown, IoT devices may include, but are not limited to:

[0034] Image sensors (such as cameras) are used to capture vehicle images of a scene (i.e., a parking space scene).

[0035] For example, the image sensor is configured to capture vehicle images of a scene only when triggered, but the image sensor is also configured not to capture vehicle images of a scene when not triggered.

[0036] The distance sensor is used to collect and generate multiple distance data. The detection direction of the distance sensor forms an acute angle with the long side of the parking space so that the distance sensor can detect the distance between the vehicle near the parking space and the distance sensor. That is, the distance data is the distance between the vehicle near the parking space and the distance sensor.

[0037] For example, a distance sensor can periodically collect distance data, acquiring multiple data points in each collection cycle. The total number of distance data points collected by a distance sensor (such as an ultrasonic radar) in each cycle is limited by the sensor's power consumption and the battery performance of the IoT device. For instance, the distance sensor is activated only when data collection is needed, not in real-time, conserving power while still being able to collect data. Since the distance sensor is a major power consumer, the duration of each data collection cycle should be strictly limited to achieve a balance between accuracy and power consumption. Therefore, the duration of each data collection cycle can be determined based on the sensor's power consumption and the IoT device's battery performance. For example, if the sensor can collect K data points within its operating time (e.g., 200ms), then after activating the sensor, it collects K data points, then deactivates, waiting for the next activation. This process continues until the sensor collects K more data points, and so on.

[0038] When determining the working time of a distance sensor for a single distance data collection based on the power consumption of the distance sensor and the battery performance of the IoT device, the lower the power consumption of the distance sensor, the longer the working time can be, thus collecting more distance data and obtaining more accurate distance data. Conversely, the higher the power consumption of the distance sensor, the shorter the working time can be, thereby saving power resources and reducing power consumption.

[0039] Similarly, the better the battery performance of an IoT device, the longer its operating time can be, thus allowing it to collect more distance data; conversely, the worse the battery performance of an IoT device, the shorter its operating time can be.

[0040] For example, the IoT device can be located at the front of the car in the parking space, or at the rear of the car, or even between two parking spaces; there are no specific restrictions on the deployment location of the IoT device. Based on the deployment location of the IoT device, it is necessary to ensure that the detection direction of the distance sensor forms an acute angle with the long side of the parking space, so that the distance sensor can detect the distance between the vehicle near the parking space and the distance sensor. Preferably, this acute angle can be less than 45 degrees.

[0041] One or more processors are used to: calculate distance estimates based on multiple distance data.

[0042] In one possible implementation, any one of the aforementioned distance data points is defined as the distance data generated by the distance sensor that is less than a preset threshold. For example, assuming the distance sensor's ranging range is 0-255cm, when there are no obstacles obstructing the ranging range, the distance data generated by the distance sensor is 0. When there are obstacles obstructing the ranging range, the distance data generated by the distance sensor represents the distance value of the nearest obstacle to the distance sensor, which is within the 0-255cm range, meaning the preset threshold can be 255cm. Based on this, when the distance data is less than the preset threshold, it is considered valid distance data, and a distance estimate can be calculated based on the valid distance data; when the distance data is not less than the preset threshold, it is considered invalid distance data, and a distance estimate will not be calculated based on the invalid distance data. In summary, the processor calculates the distance estimate based on the valid distance data.

[0043] For example, each time the distance sensor is turned on, it can collect K distance data points. The processor can find the valid distance data (i.e., the distance data points less than a preset threshold among the K distance data points) from the K distance data points and calculate the distance estimate based on all the valid distance data points.

[0044] In one possible implementation, the distance estimate (i.e., the distance estimate between the vehicle and the distance sensor) is obtained after filtering multiple distance data points (i.e., valid distance data). For example, the standard deviation of the multiple distance data points (i.e., multiple distance values) is determined, and the noise covariance is determined based on this standard deviation. Then, based on the initial estimate, the multiple distance data points for the current acquisition period, and the noise covariance, the distance estimate for the current acquisition period is determined. For example, if the parking space is empty, the initial estimate can be a valid distance data point from the multiple distance data points; if the parking space is occupied, the initial estimate can be the distance estimate from the previous acquisition period.

[0045] The determination of noise covariance based on the standard deviation may include, but is not limited to: if the standard deviation is greater than a standard deviation threshold, the noise covariance can be determined based on the standard deviation and a preset weight, where the preset weight can be a value between 0 and 1; if the standard deviation is not greater than the standard deviation threshold, the noise covariance can be determined based on a fixed covariance. For example, the noise covariance determined based on the fixed covariance may be less than the noise covariance determined based on the standard deviation and the preset weight.

[0046] In one possible implementation, the processor calculates the distance estimate only when the number of multiple distance data points is greater than a first quantity or a second quantity, where the first quantity corresponds to the parking space being empty and the second quantity corresponds to the parking space being parked, and the first quantity is greater than the second quantity.

[0047] For example, each time the distance sensor is turned on, it can collect K distance data points. The processor can then find the valid distance data from these K data points, which are denoted as P distance data points, where P can be less than or equal to K. The processor will only use the P distance data points to calculate the distance estimate if the number of P distance data points is greater than a first number or a second number.

[0048] For example, when the parking space is empty, it needs to determine if the number of P distance data points is greater than a first quantity. If so, the processor will use them to calculate the distance estimate; otherwise, it will not. When the parking space is occupied, it needs to determine if the number of P distance data points is greater than a second quantity. If so, the processor will use them to calculate the distance estimate; otherwise, it will not.

[0049] See Figure 1A As shown, IoT devices may also include, but are not limited to, a first memory and a second memory. The first memory and the second memory may be the same storage medium or two separate storage media; there is no limitation on this. Figure 1A In this example, we take the first memory and the second memory as two storage media.

[0050] The first memory is used to store distance estimates when a parking space is detected to be empty. In other words, when a parking space is empty, the processor will store the distance estimate in the first memory each time it calculates the distance estimate. The first memory stores all distance estimates.

[0051] The second memory is used to store distance estimates when the parking space is detected to be in a parking state. In other words, when the parking space is in a parking state, the processor will store the distance estimate in the second memory each time it calculates the distance estimate. The second memory stores the distance estimates.

[0052] The processor is further configured to: automatically trigger the image sensor when it detects that the distance estimate stored in the first memory falls within a preset distance range, so that the image sensor can capture vehicle images and mark the parking space as being in a parked state, that is, changing from an empty vehicle state to a parked state.

[0053] The processor is further configured to: automatically trigger the image sensor when it detects that the distance estimate stored in the second memory is greater than the vehicle exit threshold or equal to 0, so that the image sensor is used to capture vehicle images and mark the parking space as empty, that is, change from being in a parked state to being in an empty state.

[0054] In one possible implementation, the first memory is configured to store multiple distance estimates. The processor is further configured to: automatically trigger the image sensor only when it detects that all distance estimates stored in the first memory (e.g., all distance estimates) fall within a preset distance range. For example, when the parking space is empty, the first memory is used to store distance estimates for multiple consecutive periods. If all the multiple distance estimates stored in the first memory fall within the preset distance range, it is determined that a vehicle has entered the parking space (i.e., a vehicle entry has occurred), and the image sensor is automatically triggered to capture an image of the vehicle, marking the parking space as parked, and controlling the image sensor to stop capturing vehicle images.

[0055] For example, when a parking space is empty, the calculated distance estimate can be stored in a first memory. When the processor detects that the distance estimate stored in the first memory falls within a preset distance range, it determines that a vehicle has entered the parking space (i.e., a vehicle entry has occurred). It can then automatically trigger the image sensor to capture images of the vehicle, mark the parking space as parked, and control the image sensor to stop capturing images of the vehicle in the scene. Alternatively, it can trigger the image sensor to send the vehicle image to the server, and then shut down the image sensor after the image transmission is complete.

[0056] In one possible implementation, the second memory is configured to store at least one distance estimate; that is, it may store only one distance estimate or at least two distance estimates. Hereinafter, storing only one distance estimate will be used as an example. The processor is further configured to: mark the parking space as empty only when it detects that the distance estimate stored in the second memory is greater than or equal to the vehicle exit threshold and that the next nearest stored distance estimate is greater than or equal to the vehicle exit threshold. The processor is further configured to: increase the vehicle exit threshold when it detects that the distance estimate stored in the second memory is greater than or equal to the vehicle exit threshold, while the next nearest stored distance estimate is less than the vehicle exit threshold and not equal to 0.

[0057] For example, when a car is parked in a parking space, the calculated distance estimate can be stored in a second memory. Since the second memory only stores one distance estimate, each time a new distance estimate needs to be stored, the existing distance estimate in the second memory is deleted, and the new distance estimate is stored in the second memory. When the processor detects that the distance estimate stored in the second memory (for ease of distinction, this distance estimate will be denoted as distance estimate A in subsequent embodiments) is greater than the vehicle exit threshold (the vehicle exit threshold represents the distance between the vehicle and the distance sensor when the vehicle is already parked in the parking space) or equal to 0, it determines that a vehicle may be leaving the parking space (i.e., a vehicle exit may occur). Therefore, it automatically triggers the image sensor to capture vehicle images. After the vehicle image acquisition is complete, the image sensor can be controlled to stop capturing vehicle images, i.e., the image sensor is turned off.

[0058] After a preset time interval (starting from when the image sensor captures the vehicle image, i.e., after a preset time interval following the image sensor capturing the vehicle image), the distance sensor can be triggered to re-collect multiple distance data points. The processor then calculates a distance estimate based on these multiple distance data points (i.e., the next stored distance estimate adjacent to distance estimate A; for ease of distinction, in subsequent embodiments, this distance estimate will be denoted as distance estimate B, meaning distance estimate B is the next stored distance estimate adjacent to distance estimate A), and stores the distance estimate in the second memory (since the parking space is still in a parked state, the distance estimate is stored in the second memory). It should be noted that within the preset time interval, the distance sensor no longer collects multiple distance data points, nor does it need to calculate distance estimates.

[0059] In summary, when the distance estimate A stored in the second memory is greater than or equal to the vehicle exit threshold, and the next nearest stored distance estimate B is also greater than or equal to the vehicle exit threshold, the processor can mark the parking space as empty. For example, when distance estimate A is detected to be greater than or equal to the vehicle exit threshold, it is determined that a vehicle exit may occur. Therefore, the image sensor is automatically triggered to capture vehicle images. After the vehicle images are acquired, the image sensor can be controlled to stop capturing vehicle images, i.e., the image sensor is turned off. After this, when distance estimate B is detected to be greater than or equal to the vehicle exit threshold, it is determined that a vehicle exit has occurred. Therefore, the parking space is marked as empty, and the vehicle image (i.e., the vehicle image captured when a vehicle exit may occur) is sent to the server.

[0060] Alternatively, if the distance estimate A stored in the second memory is greater than the exit threshold or equal to 0, while the next nearest stored distance estimate B is less than the exit threshold but not equal to 0, the exit threshold is increased. However, the processor does not mark the parking space as empty; that is, the parking space remains parked. For example, when a distance estimate A is detected to be greater than the exit threshold or equal to 0, the image sensor is automatically triggered to capture vehicle images. After the vehicle images are acquired, the image sensor is turned off. Subsequently, when a distance estimate B is detected to be less than the exit threshold but not equal to 0, it is determined that no exit behavior has occurred (i.e., no vehicle has left the parking space). Therefore, the parking space remains parked, the vehicle images (i.e., images captured when an exit behavior might have occurred) are deleted, and the exit threshold is increased (this indicates that determining the possibility of an exit behavior based on the previous exit threshold leads to an incorrect conclusion; therefore, increasing the exit threshold and determining whether an exit behavior has occurred based on the changed exit threshold improves accuracy).

[0061] In the above embodiments, regarding the period for the distance sensor to collect multiple distance data points, when the parking space is empty, the distance sensor can be configured to collect data for a first duration, meaning that the distance sensor can collect multiple distance data points at intervals of the first duration. When the parking space is occupied, the distance sensor can be configured to collect data for a second duration, meaning that the distance sensor can collect multiple distance data points at intervals of the second duration. The first duration can be longer than the second duration.

[0062] In one possible implementation, see Figure 1B As shown, IoT devices may also include a third memory. The third memory and the first memory can be the same storage medium or two separate storage media. Similarly, the third memory and the second memory can be the same storage medium or two separate storage media; there are no restrictions on this. Figure 1B The example given is that the first memory, the second memory, and the third memory are all different.

[0063] A third memory is used to: store a distance estimate when a parking space is detected to be in a parked state; the processor is further used to: determine a vehicle departure threshold based on the distance estimates stored in the third memory, and clear the third memory. For example, the third memory is limited to storing multiple distance estimates; the processor is further used to: determine the maximum value among the distance estimates stored in the third memory as the vehicle departure threshold.

[0064] For example, when the parking space is empty, the processor will store the distance estimate in the first memory after each calculation. The first memory stores the distance estimates.

[0065] When a car is parked in a parking space, the following two situations can be handled:

[0066] Scenario 1: After each distance estimate is calculated, the processor stores it in the third memory. This process continues until the third memory contains M distance estimates (configurable empirically, such as 200), at which point no further distance estimates are stored. The processor determines the vehicle dispatch threshold based on the M distance estimates stored in the third memory, for example, by setting the maximum value among the M estimates as the dispatch threshold, and then clears the third memory. After this, distance estimates are no longer stored in the third memory, but instead are stored in the second memory, starting from the (M+1)th distance estimate.

[0067] Scenario 2: After each distance estimate is calculated, the processor stores it in both the second and third memory. The second memory stores all distance estimates, and the third memory stores all distance estimates. For the third memory, the maximum number of stored distance estimates is M. Once the number of stored distance estimates in the third memory reaches M, no further distance estimates are stored. The processor can determine the vehicle dispatch threshold based on the M stored distance estimates. For example, the maximum value among the M distance estimates can be used as the vehicle dispatch threshold, and the third memory can be cleared. After this, no further distance estimates are stored in the third memory.

[0068] For the second memory, when the parking space is in a parking state, starting from the first distance estimate, each distance estimate is stored in the second memory. For the specific storage method, please refer to the above embodiment.

[0069] As can be seen from the above technical solutions, in this embodiment, image sensors and distance sensors can be deployed on IoT devices. The distance sensors detect multiple distance data points of the parking space, calculate distance estimates based on these data points, and automatically trigger the image sensor when a vehicle enters the parking space (i.e., enters the parking space) or leaves the parking space (i.e., exits the parking space). The image sensor is then used to capture vehicle images, and is turned off after image acquisition, thus eliminating the need for real-time image sensor activation and saving power. Even without real-time image sensor activation, vehicle images of entering or leaving the parking space can be collected. These images can be analyzed to determine when a vehicle enters or leaves the parking space, and then billing can be performed accordingly. This method does not require connecting the IoT devices to a power supply system; batteries can be used to power the IoT devices, making the connection method relatively simple. This system utilizes data processing technology to detect vehicle entry and exit behaviors in parking spaces, improving the accuracy and sensitivity of detection. It significantly enhances various metrics for identifying vehicle entry and exit behaviors, increasing the accuracy of both. It also exhibits strong anti-interference capabilities. With power consumption control as its core principle, it minimizes the operating time of the image and distance sensors, thus controlling overall power consumption and providing excellent vehicle entry and exit behavior recognition capabilities. By employing a process of first acquiring vehicle images and then verifying the data, it balances the timeliness and accuracy of vehicle image acquisition while reducing the false alarm rate for exit behavior detection.

[0070] The technical solutions described above in the embodiments of this application will be explained below in conjunction with specific application scenarios.

[0071] IoT devices can include processors (such as MCUs (Micro Controller Units), image sensors, and distance sensors. Image sensors can be activated when a vehicle enters or leaves a parking space, not in real-time, saving power by capturing images of the vehicle entering or leaving the parking space. Distance sensors can be activated when distance data is needed, also saving power by capturing distance data. The MCU can be activated in real-time, deciding when the distance sensor should be activated and triggering it to start and collect distance data. Based on the distance data collected by the distance sensor, the MCU decides when the image sensor should be activated and triggers it to start and collect images when needed.

[0072] Image sensors include cameras, which are not limited as long as they can capture images of parking spaces. Distance sensors include ultrasonic radar, which is not limited as long as it can capture distance data to parking spaces.

[0073] In summary, in this embodiment of the application, the IoT device can detect vehicle entry and exit behavior through ultrasonic radar and cameras, and use data processing technology to detect vehicle entry and exit behavior in parking spaces (such as roadside parking spaces). Vehicle entry behavior refers to the act of a vehicle (any vehicle) entering a parking space (such as a roadside parking space), and vehicle exit behavior refers to the act of a vehicle leaving a parking space (such as a roadside parking space).

[0074] For example, ultrasonic radar can be used to collect distance data to parking spaces, and this data can be used to detect when a vehicle enters or leaves a parking space. When a vehicle enters or leaves a parking space, a camera can capture an image of the vehicle, and based on this image, it can be determined when the vehicle entered or left the parking space.

[0075] For example, since neither the image sensor nor the proximity sensor is turned on in real time, only the MCU is turned on in real time. Therefore, the power consumption of IoT devices is relatively low, so they can be powered by batteries instead of power supplies. In other words, IoT devices can be battery-powered or power-supplyed; there is no limitation on this.

[0076] For example, to simplify the deployment environment of IoT devices and avoid complex wiring scenarios, IoT devices can communicate with management devices (i.e., servers) via wireless networks. This means sending vehicle images to the management device via a wireless network, where the management device stores the vehicle images. Alternatively, IoT devices can also communicate with management devices via wired networks, sending vehicle images to the management device via a wired network.

[0077] See Figure 2A The diagram illustrates the deployment location of an IoT device. The IoT device can be positioned at the front of the car in a parking space, with ultrasonic radar collecting distance data to the front of the car. Alternatively, see [link to relevant documentation]. Figure 2B As shown, IoT devices can also be located at the rear of the vehicle in a parking space, with ultrasonic radar collecting distance data to the rear of the vehicle within the parking space. Alternatively, see... Figure 2C As shown, IoT devices can also be located between two parking spaces, i.e., two IoT devices are deployed together. Each IoT device includes an MCU, a camera, and an ultrasonic radar. The ultrasonic radar of the left IoT device collects distance data of the rear position of the car in the left parking space, and the ultrasonic radar of the right IoT device collects distance data of the front position of the car in the right parking space.

[0078] In practical applications, the relative horizontal angle between the main detection direction of the ultrasonic radar and the driving direction needs to be between 0 and 180 degrees. As long as this relative relationship is met, it is acceptable; there is no need to distinguish the specific usage scenario of the parking space. For example, if the relative horizontal angle between the main detection direction of the ultrasonic radar and the driving direction is between 0 and 45 degrees, and the ultrasonic radar is installed on the rear side of the vehicle, see [reference needed]. Figure 2B As shown. Alternatively, the relative horizontal angle between the main detection direction of the ultrasonic radar and the driving direction is between 135 and 180 degrees, and the ultrasonic radar is installed on the front side of the vehicle. See [reference needed]. Figure 2A As shown. Alternatively, two ultrasonic radars can be installed at the center line of adjacent parking spaces, i.e., two ultrasonic radars are integrated. The horizontal angle between the main detection direction of one ultrasonic radar and the driving direction is between 0 and 45 degrees, and the horizontal angle between the main detection direction of the other ultrasonic radar and the driving direction is between 135 and 180 degrees. See [reference needed]. Figure 2C As shown.

[0079] In this embodiment, the ultrasonic radar is a distance sensor. Based on the working principle of ultrasonic ranging, it can return the distance value of the nearest obstacle within its detection range. In other words, the ultrasonic radar returns the distance value of the obstacle closest to it. Assuming the ranging range of the ultrasonic radar is 0-255cm, when there are no obstacles obstructing the ranging range (i.e., the distance value of the nearest obstacle to the ultrasonic radar is greater than 255cm), the ultrasonic radar returns a distance value of 0. When there are obstacles obstructing the ranging range (i.e., the distance value of the nearest obstacle to the ultrasonic radar is not greater than 255cm), the ultrasonic radar returns a distance value between 0 and 255cm. For example, if the ultrasonic radar returns a distance value of 100cm, it means that the distance value of the nearest obstacle to the ultrasonic radar is 100cm. Of course, the above values ​​are merely examples and are not intended to limit the range.

[0080] In the above application scenarios, this application proposes a vehicle behavior detection method. This vehicle behavior detection method can be applied to Internet of Things (IoT) devices. An image sensor collects vehicle images when a vehicle enters or leaves a parking space and sends the vehicle images to a management device (such as a cloud platform, which can also be called a server). A distance sensor collects distance data of the parking space. An MCU implements the data processing flow of the vehicle behavior detection method. For example, based on the distance data of the parking space, the MCU can identify the time when a vehicle enters or leaves the parking space and trigger the image sensor to collect vehicle images of the parking space.

[0081] In one possible implementation, the MCU may include, but is not limited to, the following functional units: a data processing unit, a vehicle entry determination unit, a timing selection unit, a distance calibration unit, a vehicle exit determination unit, and a state switching unit. Of course, the above functional units are merely examples; a certain functional unit can be split into more functional units, or multiple functional units can be merged into a single functional unit, without limitation.

[0082] The state switching unit is used to maintain the state of the parking space, such as whether it is in a parked state (i.e., a vehicle is parked) or an empty state (i.e., no vehicle is parked). The parked state indicates that a vehicle is already parked in the parking space. For example, when the vehicle entry determination unit determines that a vehicle has entered the parking space (an entry action has occurred), the state of the parking space is marked. For example, this mark can be a first value to indicate that it is in a parked state. The empty state indicates that no vehicle is parked in the parking space. For example, when the vehicle exit determination unit determines that a vehicle has left the parking space (an exit action has occurred), or when no entry action has occurred, the state of the parking space is marked. For example, this mark can be a second value to indicate that it is in an empty state.

[0083] The data processing unit is used to acquire multiple distance data of the parking space from the distance sensor, and determine the estimated distance between the vehicle and the distance sensor at the current moment based on the multiple distance data.

[0084] The timing selection unit is used to determine the data acquisition interval of the distance sensor, and based on the data acquisition interval, determines the acquisition time, thereby triggering the distance sensor to acquire distance data at each acquisition time.

[0085] The vehicle entry determination unit is used to determine whether a vehicle has entered a parking space based on a distance estimate, i.e., whether there is a vehicle entry behavior. If so, it can trigger the image sensor to collect the vehicle image of the parking space.

[0086] The vehicle departure determination unit is used to determine whether a vehicle has left the parking space based on the distance estimate, i.e., whether there is a vehicle departure behavior. If so, it can trigger the image sensor to collect the vehicle image of the parking space.

[0087] The distance calibration unit is used to calibrate and store the vehicle exit threshold between the vehicle and the distance sensor based on the distance estimate. For example, the initial value of the vehicle exit threshold is used to represent the distance between the vehicle and the distance sensor when the vehicle is already parked in the parking space, and the vehicle exit threshold can be dynamically adjusted.

[0088] In one possible implementation, a flowchart of the vehicle behavior detection method can be found here. Figure 3 As shown.

[0089] Step 301: The distance sensor collects distance data of the parking space, and the data processing unit obtains this distance data from the distance sensor. If it is determined that a vehicle exists in the parking space based on the distance data, the data processing unit determines an estimated distance between the vehicle and the distance sensor based on the distance data.

[0090] Step 302: When the vehicle is in an empty state, the data processing unit inputs the distance estimate to the vehicle entry determination unit. The vehicle entry determination unit determines whether the vehicle has entered the parking space, i.e., whether there is an entry behavior, based on the distance estimate. If yes, step 303 can be executed; otherwise, the system waits for the next data collection moment. The data processing unit then re-acquires distance data from the distance sensor, determines the distance estimate based on the re-acquired distance data, and inputs the distance estimate to the vehicle entry determination unit. The vehicle entry determination unit then re-determines whether the vehicle has entered the parking space based on the distance estimate, and so on.

[0091] In step 303, when the vehicle entry determination unit determines that a vehicle has entered the parking space, it sends a wake-up command to the image sensor. The image sensor acquires an image of the vehicle in the parking space and sends the image to the management device. After the image is sent, the image sensor is turned off to save power. The state switching unit changes the state of the parking space to "parked," that is, from "empty" to "parked."

[0092] Step 304: After changing the parking space status to "parked," the data processing unit acquires multiple distance data points from the distance sensor, determines a distance estimate based on each distance data point, and inputs all the obtained distance estimates to the distance calibration unit. The distance calibration unit determines the vehicle exit threshold between the vehicle and the distance sensor based on the multiple distance estimates and stores the exit threshold.

[0093] For example, the distance calibration unit determines the most accurate parking distance of the vehicle by statistically analyzing a preset number (e.g., 200) of distance estimates, i.e., the distance between the vehicle and the distance sensor. This distance is used as the vehicle departure threshold to reduce false alarms caused by data fluctuations in the vehicle departure determination unit.

[0094] Step 305: When the vehicle is in a parked state, the data processing unit inputs the distance estimate to the vehicle departure determination unit. The vehicle departure determination unit determines whether the vehicle has left the parking space, i.e., whether there is a departure behavior, based on the distance estimate. If yes, step 306 can be executed; otherwise, the system waits for the next data collection moment. The data processing unit re-acquires distance data from the distance sensor, determines the distance estimate based on the re-acquired distance data, and inputs the distance estimate to the vehicle departure determination unit. The vehicle departure determination unit then re-determines whether the vehicle has left the parking space based on the distance estimate, and so on.

[0095] For example, the distance calibration unit can send a vehicle departure threshold to the vehicle departure determination unit, and based on the vehicle departure threshold, the vehicle departure determination unit determines whether the vehicle has left the parking space based on the distance estimate.

[0096] In step 306, when the vehicle departure determination unit determines that a vehicle has left the parking space, it sends a wake-up command to the image sensor. The image sensor acquires an image of the vehicle in the parking space and sends the image to the management device. After the image is sent, the image sensor is turned off to save power. The state switching unit changes the state of the parking space to an empty state, that is, from a parked state to an empty state.

[0097] In summary, this demonstrates the complete process of a vehicle entering and leaving a parking space. The process is similar when another vehicle enters and leaves a parking space, and will not be repeated here.

[0098] For example, the timing selection unit can also determine the data acquisition interval of the distance sensor and determine the acquisition time based on the data acquisition interval, and trigger the distance sensor to acquire distance data at each acquisition time, that is, the distance sensor acquires the distance data of the parking space at each acquisition time.

[0099] In one possible implementation, a flowchart of the vehicle behavior detection method can be found here. Figure 4 As shown.

[0100] Step 401: The IoT device starts running.

[0101] Step 402: The data processing unit collects the distance data of the parking space acquired by the distance sensor. Based on the distance data, the data processing unit determines the distance estimate between the vehicle and the distance sensor. This distance estimate is used as the distance estimate at time t1, which is the optimal estimate of the obstacle ahead.

[0102] In step 403, the data processing unit inputs the distance estimate at time t1 to the vehicle entry determination unit. The vehicle entry determination unit determines whether the vehicle has entered the parking space, i.e., whether there is a vehicle entry behavior, based on the distance estimate. If yes, step 404 can be executed; otherwise, the timing selection unit determines the next acquisition time, triggers the distance sensor to acquire distance data at the next acquisition time, and repeats step 402.

[0103] In step 404, when the vehicle entry determination unit determines that a vehicle has entered the parking space, it triggers the image sensor to acquire an image of the vehicle in the parking space. The vehicle entry behavior information (used to indicate the garage image at the time of vehicle entry) and the vehicle image are sent to the management device. After the vehicle image is sent, the image sensor is turned off to save device power consumption. The state switching unit changes the parking space status to "parked".

[0104] Step 405: The data processing unit collects the distance data of the parking space acquired by the distance sensor, determines the distance estimate based on the distance data, uses the distance estimate as the distance estimate at time t2, and inputs the distance estimate at time t2 (such as multiple distance estimates) to the distance calibration unit.

[0105] The distance calibration unit determines the vehicle exit threshold between the vehicle and the distance sensor based on the distance estimate at time t2, and stores this exit threshold. The distance calibration unit determines the most accurate parking distance for the vehicle, i.e., the vehicle exit threshold between the vehicle and the distance sensor, by statistically analyzing a preset number of distance estimates.

[0106] In step 406, after inputting multiple distance estimates at time t2 to the distance calibration unit, the data processing unit can also collect distance data of the parking space acquired by the distance sensor, determine a distance estimate based on this distance data, and use this distance estimate as the distance estimate at time t3. The data processing unit inputs the distance estimate at time t3 to the vehicle departure determination unit, and the distance calibration unit inputs the vehicle departure threshold to the vehicle departure determination unit. The vehicle departure determination unit determines whether the vehicle has left the parking space, i.e., whether there is a vehicle departure behavior, based on the distance estimate and the vehicle departure threshold. If yes, step 407 can be executed; otherwise, the timing selection unit can determine the next acquisition time, trigger the distance sensor to acquire distance data at this acquisition time, and repeat step 406, i.e., redetermine the distance estimate at time t3 based on the distance data, and so on.

[0107] In step 407, when the vehicle departure determination unit determines that a vehicle has left the parking space, it triggers the image sensor to acquire an image of the vehicle in the parking space. The departure behavior information (used to indicate that the vehicle image is the image of the vehicle at the time of departure) and the vehicle image are sent to the management device. After the vehicle image is sent, the image sensor is turned off to save power. The state switching unit changes the state of the parking space to an empty vehicle state.

[0108] In summary, after the IoT device is started, it can repeatedly execute steps 402-407 to achieve cyclic detection of the entry and exit behavior of all vehicles in the parking space, which will not be elaborated further here.

[0109] In one possible implementation, the data processing unit can acquire distance data, determine whether a vehicle exists in the parking space based on the distance data, and determine an estimated distance between the vehicle and the distance sensor based on the distance data. See [link to relevant documentation]. Figure 5 The diagram shown is a schematic of the data processing unit.

[0110] Step 501: The data processing unit obtains the distance data of the parking space from the distance sensor. That is, after the distance sensor collects K distance data of the parking space each time (that is, the distance sensor acquires K distance data after each start-up, and the K distance data includes valid distance data and / or invalid distance data), it sends the K distance data to the data processing unit. The number of K distance data can be selected based on experience.

[0111] For example, since the distance sensor is the main part of power consumption, the working time of the distance sensor to collect distance data at one time should be strictly limited in order to achieve a balance between accuracy and power consumption. Assuming that the distance sensor collects K distance data within the working time of the distance sensor (e.g., 200ms), the data processing unit will obtain K distance data from the distance sensor each time. Hereafter, we will take 3 distance data as an example.

[0112] For each distance data point, assuming the distance sensor's ranging range is 0-255cm, when there are no obstacles obstructing the ranging range, the distance data is 0. When there are obstacles obstructing the ranging range, the distance data represents the distance value of the nearest obstacle to the distance sensor, which is located within the 0-255cm range.

[0113] Step 502: The data processing unit determines the valid distance data among the K distance data and counts the number of valid distance data. For example, for each distance data, if the distance data is 0, then the distance data is valid; if the distance data is greater than the upper limit (e.g., 255cm), then the distance data is invalid; if the distance data is between 0 and 255cm, then the distance data is valid. In summary, the data processing unit can count the number of valid distance data among the K distance data.

[0114] Step 503: The data processing unit determines whether the number of valid distance data points out of the K distance data points is greater than the quantity threshold. If yes, proceed to step 504. If no, end the process and wait for the next cycle to re-acquire the K distance data points for the parking space from the distance sensor, and so on.

[0115] For example, the quantity threshold can be related to the parking space's status. When the space is empty, the quantity threshold can be a first quantity; when the space is parked, the quantity threshold can be a second quantity, and the first quantity is greater than the second quantity. For instance, when the space is empty, the first quantity can be 2 / 3 of the total number of data points (K). Assuming K is 3, the first quantity would be 2. When the space is parked, the second quantity can be 1 / 3 of the total number of data points (K). Assuming K is 3, the second quantity would be 1. Of course, the above are just examples of the first and second quantities. As long as the first quantity is less than or equal to K, the second quantity is less than K, and the first quantity is greater than the second quantity, there are no restrictions.

[0116] For example, the reason for using the first and second quantities mentioned above is as follows: When the vehicle is in an empty state, it is necessary to detect whether a vehicle has entered the parking space. After the vehicle enters the parking space, it is stationary, and the distance sensor receives strong reflections from the object, resulting in stable values. Therefore, the first quantity of distance data is required to be valid distance data, and this first quantity can be 2 / 3 of the total number of distance data K. When the vehicle is parked, it is necessary to detect whether a vehicle has left the parking space. However, leaving the parking space is a rapid movement process, and the distance sensor receives weak reflections from the object, reducing the stability of the values. Therefore, the second quantity of distance data is required to be valid distance data, and this second quantity can be 1 / 3 of the total number of distance data K.

[0117] Step 504: Based on multiple distance data collected by the distance sensor in the current acquisition cycle, such as the valid distance data among K distance data points (hereafter, we will take K valid distance data points as an example, meaning all K distance data points are valid distance data points), the data processing unit determines the standard deviation of the multiple distance data points. For example, the standard deviation of the multiple distance data points can be determined using the following formula: In the above formula, S represents the standard deviation, K represents the total number of distance data points, and X represents the mean of all distance data points. i This represents distance data, and the standard deviation reflects the stability of this set of distance data; the smaller the value, the higher the stability.

[0118] Step 505: Determine the noise covariance based on the standard deviation. For example, if the standard deviation is greater than a standard deviation threshold, the noise covariance is determined based on the standard deviation and a preset weight, where the preset weight is between 0 and 1; if the standard deviation is not greater than the standard deviation threshold, the noise covariance is determined based on a fixed covariance. The noise covariance determined based on the fixed covariance is less than the noise covariance determined based on the standard deviation and the preset weight.

[0119] For example, in a vehicle behavior detection scenario, a set of distance data (i.e., K distance data points) collected by a distance sensor is acquired within a short time, such as within 200ms. Therefore, it can be assumed that the relative position between the vehicle and the distance sensor remains unchanged, meaning that all distance data points are identical. However, in practical applications, discrepancies and jumps in distance data may occur. In such cases, standard deviation can be introduced to calculate the fluctuations between individual distance data points within the same set.

[0120] Based on this, if the standard deviation is greater than the standard deviation threshold (which can be configured empirically), it indicates that the distance data in this group is significantly affected by noise. The noise covariance R of this group of distance data is then determined as the standard deviation multiplied by the weights, i.e., the noise covariance is determined based on the standard deviation and preset weights. If the standard deviation is not greater than the standard deviation threshold, it indicates that the distance data in this group is relatively stable and not significantly affected by noise. In this case, a fixed covariance (i.e., the default value) is used to determine the noise covariance R of this group of distance data. The noise covariance determined based on the fixed covariance can be less than the noise covariance determined based on the standard deviation and preset weights.

[0121] Step 506: Based on the initial estimate, multiple distance data points in the current acquisition period, and the noise covariance, determine the distance estimate for the current acquisition period. For example, when the vehicle is empty, the initial estimate can be any valid distance data point among multiple distance data points; when the vehicle is parked, the initial estimate can be the distance estimate from the previous acquisition period.

[0122] For example, the initial estimate, multiple distance data points for the current acquisition period, and the noise covariance can be input into a Kalman filter. The Kalman filter then outputs the distance estimate for the current acquisition period. In other words, the data processing unit uses the Kalman filter's calculation formula to determine the distance estimate for the current acquisition period. Of course, the Kalman filter is just an example for determining the distance estimate and is not a limitation.

[0123] Since the processing of multiple distance data is only one-dimensional, a simplified one-dimensional Kalman filter can be used, as shown in the following formula: In the above formula, E k This represents the optimal estimate at the current moment, i.e., the distance estimate for the current acquisition period. E k-1 R represents the initial estimate, K represents the noise covariance, and X represents the total number of distance data points. i This represents distance data. In summary, distance estimates can be determined based on initial estimates, multiple distance data points, and noise covariance.

[0124] In the above formula, the initial estimate E k-1The initial estimate E depends on the state of the parking space. When the car is parked, the initial estimate is... k-1 It can be the distance estimate from the previous acquisition cycle, i.e., the E value from the previous acquisition cycle. k , representing the optimal estimate at the previous moment. When the vehicle is in an empty state, the initial estimate is E. k-1 It can be a valid distance data X1 from multiple distance data. X1 can be any valid distance data from all distance data (such as K distance data), such as the first valid distance data from all distance data, that is, the first valid distance data X1 is used as the optimal estimate of the previous time step.

[0125] Using the above initial estimate E k-1 The reason is that when the car is parked, the collection interval between adjacent sets of distance data is short and the correlation between them is strong. Therefore, the optimal estimate of the previous collection period (time tn-1) can be used as the initial estimate E of the current collection period (time tn). k-1 When the vehicle is empty, to reduce the high power consumption caused by the distance sensor and interference from pedestrians or other environmental factors, the distance data detection frequency is low, and the correlation between adjacent sets of distance data is low. Therefore, the first valid distance data X1 in this set of distance data is used as the initial estimate E. k-1 .

[0126] In summary, the distance estimate for the current acquisition cycle can be obtained and output. For example, the data processing unit can obtain the distance estimate for each acquisition cycle and input the distance estimate for each acquisition cycle into the vehicle entry determination unit, vehicle exit determination unit, distance calibration unit, etc.

[0127] In one possible implementation, the vehicle entry determination unit can determine whether a vehicle has entered a parking space based on a distance estimate, see [link to relevant documentation]. Figure 6 The diagram shown is a schematic of the vehicle entry determination unit.

[0128] Step 601: After receiving the distance estimate, the vehicle entry determination unit stores the distance estimate in the first memory, which is used to store the distance estimates of the most recent N acquisition cycles.

[0129] For example, the first memory (which can be implemented through a data buffer pool) is used to store the distance estimates for the most recent N acquisition cycles. N can be configured empirically, such as N=12, 16, etc. Taking N=12 as an example. After the vehicle entry determination unit stores the distance estimate for the 13th acquisition cycle in the first memory, it will delete the distance estimate for the 1st acquisition cycle from the first memory, and so on. The vehicle entry determination unit will only store the distance estimates for 12 acquisition cycles in the first memory.

[0130] For example, when the parking space is empty, the vehicle entry determination unit can receive the distance estimate input by the data processing unit. Each time the vehicle entry determination unit receives a distance estimate, it stores the distance estimate in the first memory, and the first memory can store up to 12 distance estimates.

[0131] Step 602: If the parking space is empty and the distance estimates (i.e., all distance estimates) for N collection cycles in the first memory are within the preset distance range, the vehicle entry determination unit determines that the vehicle has entered the parking space, i.e., there is a vehicle entry behavior, and proceeds to step 603. If the distance estimate for any collection cycle in the first memory is not within the preset distance range, then there is no vehicle entry behavior.

[0132] For example, the preset distance range can also be called the vehicle entry distance range. The preset distance range can be configured based on experience. For example, the preset distance range can be [30, 200]. Of course, the preset distance range [30, 200] is just an example and is not limited thereto. As long as the minimum value of the preset distance range is greater than 0 and the maximum value of the preset distance range is less than 255 (i.e., the maximum value of the distance value), it is acceptable.

[0133] The reason for using a preset distance range [30, 200] is that when the relative position between the vehicle and the distance sensor is less than 30cm, the vehicle's acquisition angle is incomplete, so the preset distance range should not be less than 30cm. When the relative position between the vehicle and the distance sensor is greater than 200cm, the reflection intensity becomes unstable due to the increased distance, which can easily cause numerical fluctuations and interference, so the preset distance range should not be greater than 200cm.

[0134] Step 603: The vehicle entry determination unit sends a wake-up command to the image sensor so that the image sensor can acquire the vehicle image of the parking space (i.e., the vehicle image of the vehicle entering the parking space), send the vehicle image to the management device, and turn off the image sensor after the vehicle image is sent to save power.

[0135] In one possible implementation, after the vehicle entry determination unit determines that the vehicle has entered the parking space, i.e., after the vehicle entry action has occurred, the distance calibration unit can determine the distance between the vehicle and the distance sensor (i.e., the distance between the two after the vehicle has parked in the parking space) based on multiple distance estimates. This distance is used as a vehicle exit threshold, and the vehicle exit threshold is stored. See also Figure 7 The diagram shown is a schematic of the distance calibration unit's processing.

[0136] Step 701: After receiving the distance estimate, the distance calibration unit determines whether the vehicle has completed distance calibration, i.e., whether the vehicle departure threshold between the vehicle and the distance sensor has been obtained. If yes, the departure threshold determination process exits; otherwise, the departure threshold determination process is executed, i.e., step 702 is executed.

[0137] For example, when the parking space is in a parked state (i.e., changing from an empty state to a parked state), after the data processing unit obtains the distance estimate for each collection cycle, it can send the distance estimate to the distance calibration unit. After receiving the distance estimate, the distance calibration unit first determines whether the vehicle has completed distance calibration. If not, it proceeds to step 702; if so, it exits. Figure 7 The process.

[0138] Step 702: The distance calibration unit stores the distance estimate in the third memory. The third memory is used to store the distance estimate for M acquisition cycles. That is, the third memory can store the distance estimate for a maximum of M acquisition cycles. M can be a positive integer, such as 200, 210, etc., and there is no restriction on it.

[0139] For example, the third memory (which can be implemented through a data buffer pool) is used to store the distance estimates for M acquisition cycles. M can be configured based on experience. Therefore, each time the distance calibration unit receives a distance estimate, it can store the distance estimate in the third memory.

[0140] Step 703: The distance calibration unit determines whether the distance estimates in the third memory have reached M values.

[0141] For example, after each distance calibration unit stores a distance estimate in the third memory, it can determine whether there are M distance estimates in the third memory. If so, it executes step 704; otherwise, it waits for the next distance estimate to be stored in the third memory and continues to execute step 703.

[0142] Step 704: The distance calibration unit determines the maximum distance estimate among the M distance estimates in the third memory as the vehicle exit threshold between the vehicle and the distance sensor.

[0143] For example, after determining the maximum distance estimate as the departure threshold, the vehicle can be marked as having completed calibration, and subsequent distance calibration will not be performed on the same vehicle. The departure threshold is then sent to the departure determination unit. Furthermore, the distance calibration unit can clear the third memory, so that when the next vehicle enters after the current vehicle has departed, completely new accumulated data is used for distance calibration. It is important to note that after clearing the third memory, no further distance estimates are stored in the third memory before the current vehicle departs; that is, the third memory only stores the M distance estimates for the current vehicle.

[0144] In one possible implementation, the vehicle departure determination unit can determine whether the vehicle has left the parking space based on a distance estimate, see [link to relevant documentation]. Figure 8 The diagram shown is a schematic of the vehicle departure determination unit.

[0145] Step 801: When the parking space is in a parked state, the vehicle exit determination unit can receive the distance estimate input by the data processing unit. After receiving the distance estimate, the vehicle exit determination unit stores the distance estimate in a second memory. The second memory (which can be implemented through a data buffer pool) stores the distance estimate for the most recent acquisition cycle; that is, the second memory only stores the distance estimate for one acquisition cycle. The vehicle exit determination unit then determines whether the distance estimate in the second memory is greater than the vehicle exit threshold. If yes, step 802 is executed; if no, it determines whether the distance estimate is 0. If it is 0, step 802 can be executed; if it is not 0, it determines that the vehicle has not left the parking space and waits for the next received distance estimate. Based on this newly received distance estimate, step 801 is executed again.

[0146] For example, if the distance estimate is greater than the vehicle departure threshold, it means that the distance between the vehicle and the distance sensor has increased, and the vehicle may leave the parking space, which means that there may be a vehicle departure behavior.

[0147] Step 802: The vehicle departure determination unit sends a wake-up command to the image sensor so that the image sensor can acquire vehicle images of the parking space (i.e., vehicle images of vehicles that have departed) and store the vehicle images of vehicles that have departed. After the vehicle images are stored, the image sensor is turned off to save power.

[0148] For example, in order to acquire vehicle images (which may include license plates) of a parking space during the process of a vehicle leaving, the requirements for timeliness and data stability are very high. Therefore, when a vehicle is likely to leave the parking space, a wake-up command needs to be sent to the image sensor immediately so that the image sensor can acquire vehicle images of the parking space. However, after the vehicle image is acquired, it is not immediately sent to the management device. Instead, the vehicle image is stored first, and the process awaits the exit verification result in subsequent steps.

[0149] The reason for adopting the above method is to ensure that vehicle images are captured during the vehicle's departure process. If the distance estimate is greater than the departure threshold, or if the distance estimate is 0, indicating that the vehicle may have left the parking space, a wake-up command needs to be sent to the image sensor immediately to enable the image sensor to capture the vehicle image of the parking space; otherwise, the vehicle may leave quickly. However, in real-world scenarios, noise data may cause an abnormal increase in the distance estimate, leading to a false departure alarm. Therefore, after the vehicle image is captured, it is not immediately sent to the management device. Instead, the vehicle image is stored first, awaiting the departure verification result in subsequent steps. It is important to note that a false alarm is an incorrect result; for example, the vehicle did not leave, but it was incorrectly identified as having departed.

[0150] Step 803: After a preset time interval, the vehicle departure determination unit can receive the distance estimate input by the data processing unit and store the distance estimate in the second memory. If the vehicle departure determination unit determines that the distance estimate in the second memory is greater than the vehicle departure threshold, or if the distance estimate is 0, it determines that the vehicle has left the parking space and executes step 804. If the vehicle departure determination unit determines that the distance estimate in the second memory is not greater than the vehicle departure threshold and the distance estimate is not 0, it determines that the vehicle has not left the parking space, i.e., a false alarm has occurred. In this case, step 805 can be executed.

[0151] For example, after triggering the image sensor to acquire vehicle images of the parking space, a timer can be started. The timer's timeout period is a preset duration, which can be configured empirically, such as 3 seconds or 4 seconds, and is not limited thereto. After the timer's timeout period expires, the data processing unit acquires the distance data of the parking space, re-determines the distance estimate based on this data, and sends the distance estimate to the vehicle exit determination unit. After receiving the distance estimate, the vehicle exit determination unit stores the distance estimate in a second memory. If the distance estimate in the second memory is greater than the vehicle exit threshold, or if the distance estimate is 0, then a vehicle exit action is determined to have occurred. If the distance estimate is not greater than the vehicle exit threshold and the distance estimate is not 0, then no vehicle exit action is determined to have occurred, i.e., a false alarm occurs.

[0152] Step 804: The vehicle departure determination unit sends a wake-up command to the image sensor so that the image sensor sends the vehicle image to the management device, and turns off the image sensor after the vehicle image is sent.

[0153] Step 805: The vehicle departure determination unit sends a wake-up command to the image sensor, causing the image sensor to discard the stored vehicle images and turn off the image sensor. For example, since the aforementioned vehicle departure threshold has already led to false alarms, the vehicle departure determination unit can also adjust the vehicle departure threshold, such as increasing its value. In subsequent processes, the above procedures are executed based on the adjusted vehicle departure threshold.

[0154] In one possible implementation, the timing selection unit can determine the data acquisition interval of the distance sensor and determine the acquisition time based on the data acquisition interval, triggering the distance sensor to acquire distance data at each acquisition time. See [link to relevant documentation]. Figure 9 The diagram shown is a schematic of the timing selection unit's processing.

[0155] Step 901: The timing selection unit determines the status of the parking space.

[0156] Step 902: If the parking space is empty, the timing selection unit determines the data collection interval as the first duration, which is the interval between two adjacent collection times. For example, after waking up the distance sensor, collecting distance data of the parking space through the distance sensor, and then turning off the distance sensor, after the first duration, the distance sensor is woken up again, collecting distance data of the parking space through the distance sensor, and then turning off the distance sensor, and so on.

[0157] Step 903: If the parking space is in a parked state, the timing selection unit determines whether the vehicle exit threshold between the vehicle and the distance sensor has been obtained. If not, proceed to step 904; if yes, proceed to step 905.

[0158] Step 904: If the vehicle departure threshold is not obtained, the timing selection unit determines the data collection interval as the third duration, that is, the interval between two adjacent collection times is the third duration.

[0159] For example, after collecting distance data of parking spaces through a distance sensor and turning off the distance sensor, the distance sensor is woken up again after a second period of time to collect distance data of parking spaces, and then turned off again, and so on.

[0160] Step 905: If the vehicle departure threshold has been obtained, the timing selection unit determines the data collection interval as the second duration, that is, the interval between two adjacent collection times is the second duration.

[0161] For example, after collecting distance data of parking spaces through a distance sensor and turning off the distance sensor, the distance sensor is woken up again after a second period of time to collect distance data of parking spaces, and then turned off again, and so on.

[0162] In the above embodiments, the first duration can be greater than the second duration, the second duration can be greater than or equal to the third duration, and the second duration can be inversely proportional to the vehicle departure threshold. That is, the larger the vehicle departure threshold, the smaller the second duration, and vice versa. The reason for using the above duration is as follows:

[0163] If the parking space is empty, image acquisition needs to be triggered after the next vehicle enters the parking space and comes to a complete stop. The relative position of the vehicle and the distance sensor after the vehicle comes to a complete stop is random and fixed. Therefore, the timeliness requirement for triggering image acquisition is much lower than that for triggering image acquisition when the vehicle leaves. Considering factors such as timeliness and power consumption, the data acquisition interval is relatively large when no vehicle is parked. Therefore, the first time interval will be longer than the second time interval, and the first time interval will be longer than the third time interval. For example, the first time interval can be 5 seconds. Of course, 5 seconds is just an example of the first time interval and there is no limitation on it.

[0164] If the parking space is in a parked state and the exit threshold has not been obtained, it means that the vehicle has just entered the parking space. It is necessary to quickly calibrate the exit threshold between the vehicle and the distance sensor. Therefore, by speeding up the data iteration, data that truly reflects the exit threshold can be obtained, that is, the data collection interval is relatively small. Therefore, the third duration will be shorter than the first duration, and the third duration will be shorter than the second duration. For example, the third duration can be 500ms. Of course, 500ms is just an example and there is no limitation on it.

[0165] If the parking space is in a parked state and the exit threshold has been obtained, the data collection interval can be selected based on the exit threshold. This data collection interval is the second duration, and the second duration is inversely proportional to the exit threshold. For example, when the exit threshold is 0-120cm, the second duration can be 1 second; when the exit threshold is 120-150cm, the second duration can be 800ms; and when the exit threshold is greater than 150cm, the second duration can be 600ms. Of course, the above values ​​for the exit threshold and the second duration are just examples and are not considered limitations.

[0166] The reason for selecting different data acquisition intervals based on the vehicle departure threshold is that, in real-world scenarios, the vehicle's departure speed is constant, and the distance sensor's ranging range is limited. Therefore, the larger the vehicle departure threshold, the shorter the time window during which the vehicle can be detected. Conversely, the smaller the vehicle departure threshold, the larger the time window during which the vehicle can be detected. Thus, different data acquisition intervals (i.e., the second duration) can be selected based on the vehicle departure threshold to improve the image acquisition rate during the vehicle departure process.

[0167] In one possible implementation, when the parking space is in an vacant state, the state switching unit can change the state of the parking space to a parked state; or, when the parking space is in a parked state, the state switching unit can change the state of the parking space to an vacant state. See [link to relevant documentation]. Figure 10 The diagram shown is a schematic of the state switching unit's processing.

[0168] Step 1001: The state switching unit determines the state of the parking space.

[0169] Step 1002: If the parking space is empty and the vehicle entry determination unit determines that the vehicle has entered the parking space (i.e., there is a vehicle entry behavior), the state switching unit changes the state of the parking space to a parking state and marks the vehicle as uncalibrated. The distance calibration unit then calibrates the vehicle and determines the exit threshold.

[0170] Step 1003: If the parking space is in a parked state and the vehicle departure determination unit determines that the vehicle has left the parking space (i.e., there is a vehicle departure behavior), then the state switching unit changes the state of the parking space to an empty state.

[0171] As can be seen from the above technical solutions, the embodiments of this application significantly improve various metrics for identifying vehicle entry and exit behaviors using distance sensors. The accuracy of vehicle entry recognition is increased to over 95%, and vehicle exit recognition is calculated from the moment the vehicle actually leaves, enabling the identification of exit behavior within a short time with an accuracy of over 90%. Image acquisition can be triggered instantly upon exiting the vehicle. The algorithm has strong anti-interference capabilities, with power consumption control as its core. By minimizing the working time of the distance sensor, the overall power consumption is controlled, while maintaining excellent vehicle entry and exit behavior recognition capabilities, achieving better recognition rates and timeliness. Based on discrete distance data, the standard deviation of the same group of distance data is correlated with the noise covariance of the Kalman filter to suppress data noise. The process of acquiring images first and then verifying data ensures compatibility with the timeliness and accuracy of image acquisition; the feedback mechanism for data verification failure reduces the false alarm rate of detecting vehicle exit behavior. By identifying data characteristics in empty and parked states, and differentiating three elements—the optimal estimate of adjacent time points, the standard deviation of the same group of sampled data, and the noise covariance of the Kalman filter—better data noise reduction is achieved. Through high-frequency data collection and iteration, accurate estimation of the vehicle departure threshold is achieved, suppressing the impact of noise on the departure threshold, reducing the false alarm rate of vehicle departure detection, and improving the sensitivity of vehicle departure detection. Based on the relative distance between the actual parked vehicle and the distance sensor, the sampling interval frequency of the raw data is differentiated, balancing power consumption control and the timeliness of vehicle departure triggering.

[0172] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0173] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0174] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0175] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0176] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0177] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.

Claims

1. A device for capturing vehicle images in a parking space scenario, wherein the parking space includes both an empty vehicle state and a parked vehicle state, the device comprising: Image sensor; A distance sensor is used to collect and generate multiple distance data points. The detection direction of the distance sensor forms an acute angle with the long side of the parking space, so that the distance sensor can detect the distance between the vehicle near the parking space and the distance sensor. One or more processors are configured to: calculate a distance estimate based on the plurality of distance data; The device also includes a first memory and a second memory; The first memory is used for: When the parking space is detected to be vacant, the distance estimate is stored. The second memory is used for: When the parking space is detected to be in a parked state, the distance estimate is stored; The processor is further used for: When the distance estimate stored in the first memory is detected to fall within a preset distance range, the image sensor is automatically triggered so that it can capture vehicle images and mark the parking space as being in a parking state. When the distance estimate stored in the second memory is detected to be greater than the vehicle departure threshold or equal to 0, the image sensor is automatically triggered so that the image sensor can be used to capture vehicle images. The processor calculates the distance estimate only when the number of the plurality of distance data is greater than a first quantity or greater than a second quantity, wherein the first quantity corresponds to the parking space being empty and the second quantity corresponds to the parking space being parked, and the first quantity is greater than the second quantity; The parking space is marked as empty only when the distance estimate stored in the second memory is greater than or equal to the vehicle exit threshold and the next nearest stored distance estimate is greater than or equal to the vehicle exit threshold. Furthermore, if it is detected that the distance estimate stored in the second memory is greater than the vehicle departure threshold or equal to 0, while the next nearest stored distance estimate is less than the vehicle departure threshold and not equal to 0, then the vehicle departure threshold is increased.

2. The device according to claim 1, wherein, When the parking space is empty, the distance sensor is configured to collect data for a first duration. When the parking space is in a parked state, the distance sensor is configured to collect data for a second duration. Wherein, the first duration is longer than the second duration.

3. The device according to claim 1, wherein, The distance estimate is obtained after filtering the multiple distance data.

4. The device according to claim 1, wherein, The distance data is defined as the distance data generated by the distance sensor that is less than a preset threshold.

5. The device according to claim 1, wherein, The first memory is configured to store a plurality of the distance estimates; The processor is further used for: The image sensor is automatically triggered only when it is detected that all distance estimates stored in the first memory fall within a preset distance range.

6. The device according to claim 1, wherein, The device also includes a third memory; The third memory is used for: When the parking space is detected to be in a parked state, the distance estimate is stored; The processor is further configured to: determine the vehicle departure threshold based on the distance estimate stored in the third memory, and clear the third memory.

7. The device according to claim 6, wherein, The third memory is configured to store a plurality of the distance estimates; The processor is further used for: The maximum value among the distance estimates stored in the third memory is determined as the vehicle departure threshold.

Images